Simple OSS Cloud Computing Platform Initiative

Cloud computing platform is an environment that allows to build, deploy and manage systems in the cloud. Such systems may benefit from on-demand resources (computing, storage and network) provided at competitive prices, elastic scaling and absence of low-level management tasks (handled by the infrastructure provider and the platform).

Sounds cool? Let's try to think what it would take to build OSS Cloud Computing Platform a la Alt style.

The implementation idea is simple - allow developers to start building, deploying and managing cloud systems, while keeping everything as simple and frictionless as possible. If things are complex - the project will just stall before even having a chance to kick off. If there is a lot of friction - the project will not have a chance to evolve.

If we shamelessly borrow ideas from good initiatives from all sides of the fence, then here's how the typical usage might look like:

Let's walk through the steps, that are absolutely necessary to be performed in order to have our solution running in the cloud:

• First, we configure the platform
• Then we build the solution
• Last step is to deploy the solution

Last step should have the least amount of friction involved, since this is something you would do frequently (we are aiming for the continuous deployment from the start).

Create Deployment Slot

We need to pass the deployment management information to the cloud platform (whether it is hosted locally or shared project server is used). Logically required management information includes:

• Cloud Provider API keys.
• Version control repository credentials.
• Base image type.
• Private settings (something you would not want to keep in the source code)

Once we have this information in place it should be enough for the scripts behind the platform to be ready to deploy and monitor. Just like the continuous integration server works.

If you have used Lokad.Cloud, the deployment slot was created there by deploying a new Azure solution and then uploading project dlls via the web console.

Build The Solution

Next step is to actually build the solution. Developer could check out one of the pre-configured templates for his solution. These templates would be open-sourced, just anything else and would include things like:

• proper solution structure with all the ignore files;
• cloud interface libraries;
• script templates;
• various resources specific to the project type.

The template solution could be immediately compiled and started locally. No fabrics are needed.

Deploy the Solution

Deploying solution should be as simple as "git push". Afterwards the simple magic will start:

• platform will detect change in the DVCS and will start one of the pre-configured VM images, using the API key.
• VM image will download the source code, build it, initialize the cloud process and report to duty.
• Platform will stop the previous version(s) of VM, replacing it with the new one. Then it will tell the second instance to start actually running.

If something goes wrong - we abort the process, while raising an alert with error logs included.

This might look like a simple continuous cloud deployment server. In a sense, it is.

Implementation

Implementation-wise, the whole idea is just to have:

• a simple web UI with a bunch of server-side scripts;
• one or two pre-configured cloud VM images for one-or-two cheapest cloud providers;
• interface libraries that provide uniform (more-or-less) access to the cloud resources;
• one or two solutions templates/samples that match these cloud VM images and include interface libraries.

Obviously, there are multiple ways to improve the experience, while going beyond the initial idea (plugging monitoring and maintenance feedback back to the platform UI is one of them). Yet, keeping the project scope to a bare minimum for a start is one of the simplest ways to guarantee that it hits the real world fast and gets a chance to evolve based on the feedback (ivory tower plans tend to be less lucky with the natural selection process).

Primary stack focus would be:

• AMQP for the messaging (open source, have enough performance for MMORPG, is pleasure to use);
• NoSQL persistence (CQRS Architectures do not care much about persistence anyway, plus these things are easier to scale);
• custom light-weight service bus implementation (it's easy to do this with AMQP anyway) borrowed from Lokad.CQRS (that's for .NET implementation, other platforms are welcome as well).

We are targeting two audiences here:

• developers that just want to learn the thing without spending a lot of resources and time on getting started;
• small/medium teams that want to give it a try, but don't have resources and time to work through all the challenges and problems (there are a few).

Actually, theoretically it should be possible to fit the whole "hello world" experience in 5-10 minutes. This would require a few shortcuts, though.

The project will provide:

• the initial guidance that could improve with the help from community - script templates, image versions, tutorials;
• architectural and design references that fit this kind of projects (a big link to CQRS with some bridging material);
• overlaying logic that is based on the practical experience and enforces good development practices.

So basically, that just: some experience and practices, a few running scripts, some templates and guidance. Almost too simple for a name of "Cloud Computing Platform". Yet as long at it helps to build, deploy and manage - why not?

What do you think? Are there ways to reduce complexity and friction even further?

Comments

[Dmitri Nesteruk]

Not quite sure what you're trying to create here - your own Azure-like cloud infrastructure or just guidance packages for existing cloud infrastrucutures?

[Rinat Abdullin]

I'm considering to create a simple OSS cloud computing platform that could simplify cloud app development (based on CQRS architectures) and deployment to cost-effective cloud infrastructure providers.

When a project is OSS, it is also a guidance of it's own.

[Ken Egozi]

That's a great initiative.

It sounds though like you're packaging a few things together, that might benefit more (or the same) if done separately. One is easy deployment of a .net app (a-la Heroku or even AppEngine), and the other is a simple and frictionless setup of CQRS environment. The latter can be built on top of the first, which has a value of its own.

[Rinat Abdullin]

Ken, yes, you got the point here.

The only subtle thing is that app deployment might need to be aware of certain things within the deployed package (for management, monitoring and maintenance). Hence, it is easier to build these two in a single project.

Of course, since this is an open source project, it will be possible to take any piece and reuse on its own.

[Peter Mounce]

What you're describing sounds very similar to heroku.com in the Ruby world. Having such a thing for .NET would be a lovely thing!

[Rinat Abdullin]

Then there might be some exciting news coming for you soon))

[Troels Thomsen]

I've been holding on this comment for weeks, and finally we've just launched our private beta of AppHarbor, which is a Git-enabled .NET cloud platform. If you are interested in checking it out, request an invite at our site and I'll be happy to send it in your direction...

In regard to your post, the "Enforces good dev practices" argument is essentially our focus in all this. We don't want developers to write code for our platform. We want to embrace great code quality, which allows you to run everywhere.

[Rinat Abdullin]

Troels, Thanks for leaving a comment. I've signed up for the invite and look forward for giving it a try!