I love software engineering: designing prototypes, writing code, building beautiful solutions. It is a way of thinking and expressing yourself.
I spent more than a decade in this area, only to realize that I've been approaching that from a completely wrong angle.
There is nothing wrong with building a beautiful software solution. However, the beauty doesn't necessarily have anything to do with the usefulness and practicality of the software.
How well does your software solve the problems?
How to doom a project?
Let's say that I want to build something useful for others (for money, reputation, or karma). I spend time to make it perfect. Code should be clean, CI/CD pipelines should be convenient, APIs - understandable. Plus, I may want to exercise a design pattern or two that I've learned.
There is a big reveal of my project after weeks of hard work. Then, the world suddenly gives a cold shoulder. Friends don't use it, and customers don't fully comprehend it.
Without positive feedback, there isn't much enthusiasm to continue developing. The project gets abandoned. I failed to make a difference.
Perhaps, the world wasn't ready for my solution? Let's not dig into the details, but try something new.
What about a link aggregator or a new highly scalable event sourcing library? Ooh, great idea! I'll make it open source and user-friendly. Maybe a large corporation will start using it. Then people will ask for support, and I could offer a commercial offer. There are so many exciting options that I need to start writing them down!
I can already tell that this second project is also likely to be doomed. It is already in love with its failure - the solution.
Rooting the project in problems
Outstanding software engineering doesn't necessarily lead to a successful software product or project.
The project will be successful when it is practical and rooted in real problems.
I ignored the problems. The solutions were well-engineered but eventually withered and died.
Focus on the real problems, and the solution will have a greater chance of survival and growth. What I should've done: study the real world, listen to people's problems, and dig underneath them. Then, find the ways to validate my solution before writing too many lines of code.
That, I think, was the main issue with my projects in the past. I focused too much on engineering, while I should've focused on identifying problems and iteratively shaping the solutions.
Coding should start after the solution design. Design should start after talking to people and identifying the problems. All three should iterate.
There is an immense field of study around "Product Design" and "Customer Development". By ignoring it for many years, I've learned about the cost of building solutions that aren't rooted in people's problems.
That is a good thing. Now there is a lot to catch up :)
Update: there is a follow-up article: Problem-driven vs Domain-driven