Making Sense of Insane Software Requirements

Writing is a thought you can analyze (c) Tim Ferris

Have you ever found yourself looking at a pile of software requirements without any clue what to do with them? Some of them could make sense, while others - little sense or even contradict to each other. All in all this would leave this nagging feeling of "big ball of mud", that you have to sort out.

This happens quite often to me. I used to hate such situations before. However these days such problems are simply tasks that usually have step-by-step solution.

Let's see why these big piles of requirements (or specifications) even show up. They are probably caused by the fact that it's rare for a single person to have a clear vision of all requirements in his head. Usually these come from different people at different points in time:

• Different people specify different "I want to have this awesome feature" ideas for the same project.
• Somebody might provide one suggestion ("Make it green, it'll look awesome!"), forget about it, then provide completely contradicting solution a week later ("Make it red, it'll look awesome!"). This happens to busy people all the time.
• People might provide feature requests without full knowledge about how things are already implemented ("Please add this helicopter landing pad on top of our building", while the building is simply a bungalo on the beach)

The most interesting part is that it's duty of software developer to accomplish all that in the best possible way, while turning down things that are too expensive or risky ("You know, adding underground parking garage to this existing building would probably make it collapse").

One approach to accomplish that is to simply start organizing these requirements, giving them some sort of structure. You would try to bring closely related concepts together and look at them in isolation from the others (hey, these 2 requirements try to give button "X" different colors).

You might need to try different approaches to structure and represent requirements, before you find the one that fits your sutiation (put them on cards and shuffle around, add as lines to Excel, draw UI mockups etc).

The big goal here is not to try build links and relations in your head - it'll take a lot of energy just to remember them. You can instead try to bring them some place out (even if the first attempt is a mess) and then improve on that structure on a step-by-step basis. At some point in the process you might even discover contradicting requirements (e.g.: making the button red and green at the same time), and this time you will be able to reason about them and explain "why" it is so.

So next time you encounter a bunch of complex and contradicting requirements - just try putting them on a sheet of paper and shuffling around.

Ultimately, in the world of complex business relations and behaviors, you can go further and try to code your domain model while exploring numerous specifications. Code can provide much stronger logical and reasoning support than simple writing. You'll be able to clarify your understanding by encapsulating some complexity inside value objects, hiding business behaviors inside aggregates and isolating UI-related details inside projected views and client-side interactions.

This will give you rather structured high-level representation with the ability to dive into any single detail. Better than that, you could send it for somebody to review and improve. If coded domain model happens to capture everything perfectly - it can serve as a basis for production software that will do something useful with it. Otherwise, it's just a written model that you can improve on.

Tim Ferriss once said "Writing is a thought that you can analyze". In the world of software design we can push it even further and say that "Code is a representation of the problem that you can analyze". If you happen to encounter complex business problem that needs solving, then may be capturing it in the code would help you to understand it and reason about it. Just give it a try, even if the code itself would never run in production.

Use Software Design to Aid Your Development Process

Each project is unique and requires different development methodology (a repeatable process for delivering software aka "whom to blame if deployment breaks and how to avoid it"). At Lokad we adopted a number of different methodologies for different types of projects.

In each specific case the choice was an attempt to balance between:

• complexity of adding new features;
• cost of failure and number of people that can be allocated for testing;
• number of developers working together on the project;
• required speed of development;

It was interesting to see different methodologies (basically just a collection of rules) emerge and evolve, as our understanding of the process grew and project specifics changed.

On one extreme we have relatively rigorous process, where almost all changes are tested as they go through the scheduled regular releases to Windows Azure. Development teams have multiple developers (even though they can be moving between projects on an hourly basis) and some testing people. This happens for projects where a missed bug could seriously hurt the business.

On the other extreme we have some projects with almost no testing and continuous deployment to Windows Azure, where each commit to the repository is immediately pushed to the production (web sites are updated and backend services are redeployed). This can be applied to supporting projects, where a bug would be a mere inconvenience.

There are some projects at Lokad which are located somewhere between these extremes (e.g.: somewhat important). During their lifetime, projects can change their location on this scale as well (e.g.: prototype turning into a commercial product or commercial product being discontinued).

In each case, software design and methodology serve as a powerful enabling factor, which reduces risks and costs at multiple levels. Here are some tech-specific examples:

• append-only storage (as in event sourcing) significantly reduces risk of loosing data in case of faulty deployment. There is still some risk of getting corrupt data in, but this problem is usually limited by the short time window.

append-only storage does not mean that data grows indefinitely. It just means that during the normal operation data can only be appended and never erased.

• by applying basic techniques from object-oriented software programming, messaging and domain-driven design, one can build rather decoupled software that is relatively simple to reason about. Low coupling will lead:

  • reduced risk of cascading failures;
  • lower cost of development - developers don't step on each others toes while changing code or merging changes;
  • less expensive teams - if certain portions of the code are rather simple and decoupled, then you don't need a brilliant and expensive development team to handle them;

• usage of domain-driven design and event sourcing can simplify data persistence, especially managing it between software upgrades. In the very extreme, software would have either event streams (transaction logs) schema of which rarely changes or persisted views (cached read models), which can be cleared upon the deployment. This can simplify deployment process to the point of reliable continuous delivery. No fear of irreversibly breaking your database with an SQL upgrade script.

• changes in tech requirements that would otherwise be relatively expensive are now taken for granted (e.g. moving software from on-premises to the cloud or back, scaling out client UI or backend processing). This allows software to evolve such turning points without noticeable spikes in complexity and development effort (which would otherwise require change of development methodologies).

Having that said, good software design is merely one of the necessary factors required for successful delivery. Good team collaboration, strategic analysis and adequately disciplined development methodology are also good-to-have.

Travel Schedule in the USA

Here is some additional information about travel schedule for USA in April, May. Mostly we'll be travelling in California and through national parks around, however there are a few opportunities for things related to software. I would pass through some cities in the area and would definitely love to meet local developers for a few beers (as in #CQRSBeers), if there is any interest.

Please, drop me an email to rinat.abdullin@gmail.com, if you would like to help in arranging such beer get-togethers. What is needed - to negotiate day, pick some nice and quiet bar favoured by developers and announce the event to local dev communities. Then, we'll just dring beers (or whatever you prefer), talk development, design and all other things that interest.

Here are the hard dates:

• April 27-28: San Diego. Available for CQRS Beers.
• May 1-3: Los Angeles. Available for CQRS Beers and still have up to 4 hours available for consulting/workshop.
• May 16-18: San Francisco. Available for CQRS Beers, all the other dev-related time looks booked out, ATM.
• May 19 - May 22: Portland, ME. Busy with DDD Summit 2013, but available for a dinner or beers.
• May 23 - May 27: Slowly driving from Portland to NY. No definite plans for these dates, yet, so something can be arranged in the area.

If you have any questions, please don't hesitate to drop an email.

Domain-Driven Design and Organizational Politics

Domain-Driven Design can often lend a helping hand in diffusing a tight political situation in a conservative organisation (to our benefit).

Consider case, when business processes of company are backed up by a bunch of IT teams responsible for different software systems. More often than not, these IT teams would be not so friendly to each other, trying to protect their data and software from any unfreindly influence, changes or even access to the data. This creates noticeable friction for any new initiatives, which need access to this data or simply integration with the software. At Lokad we've seen this pattern on more than one occasion.

One way of solving this situation is to leverage Domain-Driven Design methodology to identify most important factors, risks and stakeholders at play (see context mapping). Then, once we identified and prioritized separate areas, specific development methodologies can be applied to shift the odds each battle to our favor.

For example, teams fighting for their projects, can reduce cost of change and focus on real problems by evolving their domain models in collaboration with domain experts. Scalability issues (quite common in legacy CRUD domains) can be worked around by applying patterns like Domain Events and Command-Query Responsibility Segregation. This would put such teams in more favorable position, compared to other teams. Business owners like those who deliver fast and build trust. More favorable position can be leveraged to gain more influence, personal freedom or other bonuses.

Another example would be about fighting off teams of SQL Database Administrators who resist any change in database schema. Let them have their database for reporting purposes, while persisting everything internally with event sourcing. We'll simply propagate our own changes to their databases via projection of events to SQL tables. It will not be our fault if SQL database can't keep up with the performance of event-sourced backend. Then, at some point, we could simply offer to replace entire pretty expensive Oracle cluster with a bunch of Redis servers running on some commodity hardware without the need of expensive SQL tuning. Such approach can massively reduce costs in a company, which is a strong polical leverage for further improvements.

In other words, DDD helps to come up with consistent strategy for dealing with complexity, friction and inefficiency in organisations. Various tactical patterns and architectural styles could be applied locally to support this strategy in different specific situations.

I'm Open for On-site Consulting in USA in May

Peculiar travelling suggestions are dancing lessons from God. (c) Kurt Vonnegut.

I'll be travelling in USA with my girlfriend in May, visiting both coasts (NY and Portland on East and California on the West). If somebody is interested in on-site consulting or mentoring within these days, please drop me an email to rinat.abdullin@gmail.com and we could work something out.

NY and California are preferred, but other travelling suggestions are also interesting.

Does your event store only serve one node?

In reply to the question from Stacy:

Does your event store only serve one node type? Or does your event store serve any and all nodes?

I did it in various ways, since different scenarios might require different deployment strategies. Here are some cases that worked out.

1 event store for multiple subdomains, hosted within the same worker process. Event Store is hosted in the same worker process as well (with in-memory cache) and accessible via direct calls to application services from these subdomains. Worked nicely.

1 event store per node for our scalable FTP proxy for Azure project. Each node writes to it's own event store (hosting the engine). Event consumers (if there are any) join events together. Technically, in this case one could have one central event store, accessible by multiple nodes, but I didn't want to bother with complex deployment at this phase of the project. Plus, more dependent servers you have - higher risk of failure and problems.

Multiple distinct applications, hosted in different nodes, with each of their own event stores. They can exchange information by sending commands or by pushing views with well-known schema to a well-known location. Event stores are kept private.

So generally I treat event store as something private to the application. In single-node projects - one event store per application, in multi-node environments - one event store per node (or have multiple nodes share the same event store in case of some partitioning and load balancing).

Obviously, this ratio does not involve various replication scenarios (e.g. master-slave), where replicas don't count towards the event store score (they still contain the same information).

Best Infrastructure for Event Sourcing and CQRS

Disclaimer: this thing does not exist - I'm just laying out possible improvements for Lokad.CQRS.

It's interesting how some experience can change your perspective on things. Some time ago, when Lokad.CQRS was started, I considered ideal development framework to be about message bus that could be run on premises or on the cloud alike.

Quite a few projects were delivered with this approach, including ones with (relatively) big data processing, (relatively) complex business workflows and (relatively) fast development iterations. Some experience highlighted strong parts of the Lokad.CQRS approach, while the other - indicated potentially possible improvements for making the experience better.

For example, known shortcomings are:

• inefficient key-value storage model for projected views, which does not deal well with complex queries or large lists;
• inability to execute application server commands synchronously (in addition to async commands);
• complicated manually coded event subscriptions and ports;
• need to redeploy everything, upon each configuration change (including load balancing scenarios);
• we don't abuse cloud capabilities as efficiently as we could.

Right now my ideal and imaginary Lokad.CQRS would look like this:

Let's go through the concepts to see, if they make sense.

We host server-side functionality (like aggregates) in Command Handlers - methods which are executed whenever a command message arrives. A command message can be passed to server either synchronously via Command Endpoint (essentially a web interface) or enqueued in Persistent Queue for async processing. Obviously, the second approach is more scalable and reliable.

Command handlers can do all sorts of things, including generation of events (just like in any CQRS architecture). These events are pushed to the event store, from which they are published to all interested event handlers. These event handlers can be reached only by publishing event to the store (no direct calls or messaging). This provides efficient environment for having huge event streams (gigabytes of data are OK) and simple event replays, should this needed. This works well for view projections, business processes and subscriptions.

Both command handlers and event handlers can be redeployed individually and dynamically, without affecting the rest of the infrastructure. Actual infrastructure is represented by Virtual Nodes (normally one VNode per machine), which host command handlers, event handlers and command endpoints. Naturally, VNodes are the place where scaling out, dynamic upgrades and reconfiguration happens. All this functionality can be carried out by with the help of a remote interactive shell (much like shells in BeingTheWorst.com podcast or EventStore), which is accessible by connecting to any VNode via telnet/ssh client. It's good for scripting, realt-time debugging and live management.

View storage is not really abstracted in this scenario, since it's hard to find common denominator for diverse storage engines that are currently used by the Lokad.CQRS community (Redis, Lucene, SQL, Azure blobs etc). It might be better to avoid coupling altogether, while offering capability to efficiently rebuild views from the event history, should this be needed.

This sounds like something that could noticeably simplify development of cloud-capable systems with Event Sourcing, CQRS and Domain-Driven Design (in a really opinionated and methodological way).

PS: Design outlined above works well with big-data and business analytics scenarios as well (tested).