Follow me
Latest replies
Categories
Thursday
25Jun

Testing MVC Elements and Interactions with Mock Container

Lokad Shared Libraries project includes implementation of the MockContainer that significantly simplifies testing of Model-View-Controller elements in separation. It does this by automatically generating and injecting mock objects, should the tested class ask for them in its constructor.

These mock objects are persisted in the local container instance as singleton, so that additional expectations could be recorded against them. Expectations are verified when the container is disposed.

There is a little overhead of creating and disposing container and instances every test, since we are using Autofac IoC Container internally here, which has a really good performance.

Interface and internals of this MockContainer have been polished heavily since its introduction a few months ago. This (with the help from a base fixture class providing a nice syntax) has allowed to write tests like this one:

[Test]
public void Display()
{
  View.SetTitle("Some title");

  // we expect the view to ask IViewspace to display itself
  WhenViewIsShown
     // you always set Form.CancelButton  for user's pleasure
    .CancelButtonShouldBeSet()
     // assert that title as as expected
    .TitleShouldBe("Some title");

    // create 0 to 20 random models and display them in view
    View.BindModel(RandClientModels.NextEvents(0, 20));
}

This is an actual unit test against actual Windows.Forms.UserControl component. It does what it says. Ability to write unit tests like this brings following benefits for free:

  • In this unit test all calls are executed against the interface. So should we need add Silverlight or WPF view implementations to our library, we would already have tests for the first step of Test Driven Development.
  • Ability to write simple unit tests really pays off in scenarios, where a single component is reused in multiple projects. Finding a bug in one of these projects, expressing it in a unit test and fixing, immediately makes all projects downstream more stable.
  • Using randomly-generated Models allows to cover a wider range of the possible failure scenarios. Should some case go wrong, we'll be able to reproduce it by checking out rand seed used in the integration run (Nondeterministic reproducible testing will be the subject of another article).

Here's another type of unit test. It could be invoked explicitly just to see how our view looks alone (without launching an entire application and clicking our way to this View):

[Test, Explicit]
public void Show_view_for_debugging()
{
  // when view asks to show itself, actually display it
  WhenViewIsShown.ShowAsDialog();
  // load random values and display
  View.BindModel(RandClientModels.NextEvent());
  // display view, while using ValidEvent rule 
  // to validate all input
  View.GetModel(EditEventController.ValidEvent);
}

ExplicitAttribute in NUnit tells us that unit test is excluded from the usual unit testing run and has to be launched manually.

This kind of debugging unit test for the reusable View components speeds up all the testing and UI tweaking process. Do you need to check out how View validates data, scales or displays that recent UI tweak? This is just one click away:

Using MockContainer to test MVC Views

Sometimes we may want to go deeper and check out how various components integrate together. This is not a normal unit test (in the sense of Unit Testing), so it is marked with ExplicitAttribute as well.

[Test, Explicit]
public void Test_for_debugging_with_controller()
{
  // view is registered by test fixture
  WhenViewIsShown.ShowAsDialog();

  // simplified syntax for registering additional components
  // that we don't want to be replaced with mock objects.
  Container.Build(builder =>
  {
    builder.Register<ConnectionEditController>();
    builder.Register<WorkScopeView>().As<IWorkScopeView>();
  });

  // resolving controller and launching it with a random connection
  var controller = Container.Resolve<ConnectionEditController>();
  controller.Startup();
  controller.Edit(RandApiModels.NextServiceConnection());
}

Although code like this does not participate in the Continuous Integration process, having it around speeds up the development and simplifies debugging, should something go wrong.

This specific test creates UI elements like this:

Using MockContainer to test complex component interactions

Unit tests samples come from open source project Lokad SDK that allows developers to introduce forecasting capabilities of Lokad.com into their applications.

Saturday
20Jun

Cloud Bursting Scenarios For Small Companies

Almost every software developer dreams about starting up his own MicroISV company that would keep bringing him money even during the sleep. Let’s see how cloud computing can help us out here by making it easier to succeed.

This article continues the Cloud Computing series.

These days it became so easy to start a one-man company in the IT industry, like it had never been before. A lot of people are doing that, while the MicroISV and Micropreneur are becoming rather hot topics on self-development blogs.

One of the reasons for this is that you no longer need to have a physical hardware or other resources in order to be able to provide software and services. Almost everything could be virtualized and hosting could be simply rented. It's quite flexible and convenient, isn't it?

As turns out, Cloud Computing can add even more flexibility and market maturity to small companies at a fraction of the cost. That’s because one of the major driving forces behind the Cloud Computing hype is the actual ability to buy hardware capacities and resources on demand without paying for them upfront.

Let’s have a look at first business scenario of pushing CPU-intensive computing into the cloud and the technological availability of implementing it today.

Note: all figures, names and numbers are purely imaginative. They are used just to show you the logic behind the cloud bursting and scale of savings it can bring.

Let’s imagine a small and young one-man company providing some software as a service. For example IdealDotNET that specializes in providing custom code quality reports and optimization recommendations for .NET development projects, based on some proprietary algorithms. These algorithms are running against assemblies being uploaded by the customers. First 10 assemblies are analyzed for free.

We’ll put aside financial feasibility of this endeavor and concentrate on the technological implementation side. It could be as simple as this:

ISV company without a cloud computing capabilities

As we all know, running complex introspection rules against any assembly is a rather CPU-intensive process, especially if these rules include some extensive checks. So this system will be able to handle only a limited amount of users per hour. Let’s assume that this setup can process 30 .NET assemblies per hour on a deployment that costs 100 EUR per month (entire system is located on one semi-dedicated server).

As long as the number of uploads does not exceed 30 per hour, this setup will provide solution that just works for its money. Here’s how the usage statistics might look like:

Sample upload statistics for an ISV company without a cloud computing capabilities

Everything will go just fine till IdealDotNET starts some marketing campaigns or simply gets mentioned by Scott Hanselman in his blog (he once twitted about the power to take down a small site by linking to it).

We’ll get into a nasty situation, when this happens. Number of upload attempts per hour will exceed our capacities:

Small ISV company is not capable of handling the Slashdot effect

If the system had been designed to handle such a situation, then it would not crash down everything. Instead, all excessive visitors would be simply turned away by saying something like "We are sorry, but upload limit has been reached, please come later".

This approach means lost profit, because some of these visitors had a chance of liking the service and becoming customers. More than that, some of the existing customers may hit the limit, get discouraged by it and turn away to the competitors.

What could we do?

One option is to have more servers from the start. This way IdealDotNet will have more capacity to handle usage spikes before they happen. Let’s say that the company got two more servers from the very start, boosting upload limit to 100 per hour. Then the Slashdot effect would look much better for us:

Handling Slashdot effect by buying servers before it happens

That’s an easy solution, yet it comes with two problems:

  • Additional servers come at a cost (for example, additional 200 EUR per month) and they will still sit idle most of the time. If IdealDotNET had existed for mere 2 months, then total cost (simplified) of handling such a spike is 400 EUR.
  • Buying exactly the right processing capacities the start is impossible. Even with the best estimate you’ll either have excessive capacities or will miss some spikes.

That’s where the technological opportunities of Cloud computing come to the rescue. With some tweaking infrastructure of IdealDotNET could be modified to use only one server most of the time, while gracefully scaling up to handle the spikes.

ISV company enhanced with the capabilities to burst into the cloud

We’ll talk about the implementation details later. Let’s focus on the economical benefits first. With this configuration our processing limits start looking more interesting - they elastically adapt to handle the demand:

Sample usage statistics for an ISV company capable of bursting into the cloud

If IdealDotNET had used Amazon EC2 Medium CPU Virtual Machines to handle these spikes (their price is 0.20USD per hour each), then the total cost of handling this spike is around 40 EUR. That looks better than 400EUR, does not it? Plus we get the flexibility of elastically scaling to 1000 uploads per hour (or more) as needed.

This scenario is called Cloud Bursting or Bursting into the cloud.

Note: theoretically IdealDotNET could get a better deal from Mosso (their cloud machines start at 0.015 USD per hour). But it does not look like they have all the technological pieces in place for implementing cloud bursting scenario efficiently.

How hard is it to implement a system that can elastically handle as much stress as it is needed? Answer is: it is not extremely hard, if you change your way of thinking a little bit towards elastic capacities.

We start by adding a Manager component to our system (it could be a separate process or simply a loop in a background thread). It will continuously watch for the number of uploads pending to be processed. Should it become obvious that our processing capacities are not enough to keep up with the demand; it will issue requests to the Cloud Computing API to deploy new virtual machines (Amazon EC2 already has such REST-based API available, while Windows Azure claims to release it later).

Each virtual machine usually comes preconfigured with some worker role. This pre-configuration can be done by:

  • deploying an application or scripts to the cloud (if this cloud service is a Platform as a Service provider like Google App Engine or Windows Azure);
  • uploading preconfigured VM image (if the cloud is an Infrastructure as a Service provider like Amazon EC2).

When this worker boots up, it only needs the addresses of the task and result queues in order to start processing jobs. These parameters are usually passed as arguments along with the deployment calls.

In order to let the cloud workers access these queues in a secure way we will need to expose them via some encrypted service API, implemented in a flavor of communication framework of your choice.

When the Manager detects that we no longer need the some processing power from the cloud, it would gracefully shut down these workers, saving energy for the society and money for IdealDotNET.

Note: obviously, there are few more technological challenges that have to be dealt with. These might include: reliable cloud management, remote exception handling and monitoring. However these problems do not have any black holes in their domain space (at least from my perspective) and thus are almost guaranteed to resolve to a finite and defined work time which could be used in project management schedules. This work time could be further reduced by applying established frameworks and development principles.

Given all these advantages of the cloud computing why wouldn’t IdealDotNET move everything to the cloud? There are several reasons for that:

  • Cloud computing solutions are generally more expensive when it comes to long-term consumption (compared to existing offerings by hosters).
  • This would be a more complex solution to implement (from the delivery and maintenance standpoints).
  • This comes with a certain lockdown cost. Once you have your entire infrastructure in a single cloud, it is not that easy to move it to a different cloud (especially since we don’t have well-established cloud computing market, yet). It is safer to keep primary resources closer to you, while using the cloud only to handle the spikes.

In fact, in order to reduce the dependency on a single cloud provider, IdealDotNET company could use several cloud computing providers, picking whichever is better for the situation.

Does not this example of cloud bursting create an itch to use such a great resource to solve some business problems? There are more examples to follow in the Cloud Computing series.

You can subscribe to the updates, if you are interested.

Related links:

Tuesday
16Jun

Software Development Body of Knowledge - ABC 02

In this Software Development ABC article we'll walk over the development terms that were quite a paradigm shift for me in the first years of my software-related career.

There will be a short intro, followed by the compact definitions of 3 terms with the links leading to full articles (and even more links).

The first paradigm shift has happened while I was coding forecasting algorithms for my diploma in economics (back in Delphi and Pascal these days). It was about Version Control Systems. They have allowed to stop worrying about manual backups or doing something wrong. Ability to instantly roll back to the previous working copy (or any other previous moment in history) gave an invaluable sense of confidence and flexibility. I also was finally able to get rid of incremental archives scattered across the hard drives (all these "copy2.zip" and "copy2version4.zip").

The second paradigm shift was about Unit Testing which is an extremely simple concept on its own - split the code in units and write assertions to rapidly verify correct behavior of these units. Unit testing has allowed to concentrate on the development by significantly reducing the risk of breaking something with a change (which is often the case with mathematical code). Green became my favorite color afterwards.

The third paradigm shift took place some time later, when I started working remotely in my first development project. It was .NET 1.1 with a lot of fragile and complex legacy code. Changes there rarely went without breaking something else.

Yet, the sales people didn't care about things like development friction and constantly demanded new features. They often went directly to the developers in order to get them. Planning and schedules were often sacrificed for the sake of moving forward with an important customer (which actually made sense in that project).

The only way to deliver reliable functionality in such a brittle and fast paced environment was about using some methods from the Agile Software Development. And, as it has turned out later, they did work and made development process more efficient and stable.

So let's get into these three terms in a more detail.

You can either continue reading through expanded definitions (which have links leading to even more material) or subscribe to receive updates on this topic.

Software Development Body of Knowledge ABC

Version Control System - VCS


Version Control System (VCS) is a software that allows to manage changes of documents, programs, images and other information that is stored in form of computer files. Changes are usually identified by an incrementing number or letter code also known as revision number or revision.

The simplest usage of versioning is - you can easily go back to the previous working version of your files, should you mess something up with the latest changes.

Changes could range from fixing a typo in a text file up to a huge refactoring in a software project, spanning hundreds of files. Each change usually has name of the person introduced it, time of the change and an optional description message.

Proceed to the full article on Version Control Systems

Unit Testing


Unit Testing in software development is a way to quickly verify that smallest blocks of software (units) behave as expected even as the software changes and evolves.

Here's how it works.

Any program could be logically separated into distinct units (in object-oriented programming the smallest unit usually being a class). Developers, while coding these program units, also create tests for them (code blocks containing some assertions and expectations about units). These tests could be used to rapidly verify behavior of the code being tested.

When some other developer introduces new units or changes something in existing units, he can run all the tests available for the program and verify that everything is still operating as expected. Usually running unit tests is a fast operation (less than 30 seconds), so developers are encouraged to do that often.

Proceed to the full article on Unit Testing

Agile Software Development


Agile Software Development is an iterative and incremental approach in software development, where requirements and solutions evolve through a tight collaboration of self-organizing cross-functional teams.

Agile works quite well in situations when some unique software project has to be delivered really fast in order to seize the market opportunity. This works because Agile software methodologies allow development teams to adapt quickly to the rapidly changing business requirements. This happens because of:

  • short development iterations allowing to start getting feedback early in the project's life-cycle and incorporate it into the product fast by delivering working software repeatedly and in a timely manner;
  • continuous integration helping to reduce development friction and improve quality of a rapidly developing project in self-organizing environment;
  • feature-driven development focusing development on delivering features (functionality) bringing real business value to the client (and the project);
  • lean software development laying out common principles for efficient software development process;
  • eXtreme Programming pushing lightweight software development methodologies to the extreme.

Proceed to the full article on Agile Software Development

Interested in the series? You can stay tuned by subscribing to the RSS feed of this Journal.