Emergent Design Faces Reality

Last two weeks were packed. We are working hard to have a limited prototype of the application ready and available for a demo in June. So far things look really good for the schedule!

Collaborative design process

We chat frequently with Pieter, discussing things small and big: starting from component design to a naming choice of some variable or just a weird gut feeling about some code.

I found out that disagreements with Pieter are especially productive and exciting. I’m really glad that he has patience to put through with my stupid questions.

Here is one example.

A few days ago Pieter started working on profile functionality and began introducing there PhotoUrl fields. That immediately gave me the big shivers, since I considered this to be a misleading design. Profile service was responsible for managing and providing published user information like gender, birthday or name. Photo urls have got to be a different concern! Bleeding them into the component responsible for creating and providing profile info felt like an over-complication, compared to the other components (they are clean and focused).

I tried to explain these reasons to Pieter, but that didn’t get us far. He replied that it was ok to denormalize and mock some data within the profile service, since it would help him to get the profile viewing page faster. In response I tried to suggest to create mock stubs for photo urls in a dedicated photo component.

This went on for a while. Looking at the code together through ScreenHero didn’t help much either.

Some progress started only when we started talking about things in terms “this gives me shivers”, while trying to understand why each other sees things differently.

As it turned out, we had different perspectives on decomposition of the components. I had in mind purely vertical responsibility for the profile component, where it would have all layers of an N-layered app along with full responsibilities : creating data, persisting it locally, publishing events, providing HTTP handler for the UX. All that, while focusing on a small and coherent set of behaviors around public user profiles.

At the same time Pieter was working with the UX. He was interested in a design decomposition which would give him the component that would focus only on maintaining a cache of all user-related information for the purpose of serving profile pages and providing that information to the other components. That component would have a lot of data, but it would not contain any complex business rules

• mostly event subscriptions and denormalized read models.

Seeing this difference was a huge step. I also needed that component (e.g. when you have a news feed and need to enrich entries in it with beautiful profile photos along with name, gender and age for each user). However, since I wasn’t aware of such distinction in our domain, I actually misused a bunch of components for this purpose.

While flushing out boundaries and contracts of this new profile component we also touched it’s interactions with the future components, which are not even available in the current code (e.g.: review and draft). We talked about naming, responsibilities, contracts

• all things except for the implementation (which would be trivial at that point). We even made explicit things like :

Ok, so we don’t have draft and review components in our codebase this week, however we will pretend that they exist by manually publishing events from them in our ‘prepopulateDB’ script. Since the other components subscribe to these events, they will not even notice any change when we introduce actual implementations. And since we model events from the perspective of the consumers, they will be useful.

A better and more clear design emerged through this process, things clicking into the place like pieces of a puzzle.

I find this process truly astonishing : you use codebase to drive exploration of the domain and also capture a deeper insight that is obtained during that process. Emerging design is a beautiful side-effect of that process.

Design constraints

Such process would not be possible without the design constraints which fuel and direct creativity. Here are a few that are important in our case:

• Distributed development team of three people, working remotely on the same codebase in a single github repository;
• mentality of golang, which forces us to think in terms of tiny packages with short and clear names;
• requirement to have a demo version in June and a working Beta in September;
• shared belief in the power of simplicity;
• high performance and scalability requirements, which we must not optimize for right now (since that put us behind the schedule for the June demo).

Optimize for future performance

I find it particularly interesting to optimize design for future performance optimizations, while consciously writing code that is designed for short-term evolvability (and hence is hacky and slow). This forces you to think about isolating this hacky code, preparing it for future replacement and possible optimization strategies.

It is almost as if that non-existent better code was written behind the lines and continuously evolved every time you touch the component or think about it. It is impossible to forget about that, since actual code is so inefficient, just like the caterpillar.

After a few iterations you end up with the component that is designed:

• to have high evolvability in the short term
• to be optimized in the longer term, making a bunch of strategies available (starting from a denormalized read model up to a in-memory cache across all nodes in the cluster, invalidated by messages).

Making it all real

All this process is not only fun, but it also tightly tied to the real world. Tomas makes sure of that. First of all, he acts as the domain expert and the stakeholder in the product, setting constraints and priorities, sharing insight. He also works on the vision of the product from the user perspective, capturing concepts in a tangible form of HTML templates which we started merging into the codebase.

These HTML templates started showing up a few ago. They made Pieter and me feel as if New Year came early this year:

• it is awesome to see a real product instead of hacky UI;
• UX easily communicates important requirements that could be missed otherwise (e.g. “gender” symbol and “is online” highlight for every author in the newsfeed entry).

In the end

We keep saying: "let's see how much our approach will hold before it becomes a problem", however so far it holds up pretty well. Architecture, technology and other irrelevant implementation details have changed more than once during this period (e.g.: during the last weeks we switched from FDB to CRUD with shared transactions to event-driven CRUD (no event-sourcing, though). Design still supports growth of understanding and product through these minor perturbations.

Published: May 12, 2014.

Next post in HappyPancake story: Minimal Viable Product

🤗 Check out my newsletter! It is about building products with ChatGPT and LLMs: latest news, technical insights and my journey. Check out it out