Features, Use Cases, Rendr

Last two weeks went fast.

I mostly focused on the backend, rewriting some of our older modules (register, draft, review and auth) to match JSON API requirements and covering them with use-cases. Missing edge cases were addressed as well.

There were also a few technical features, affecting the entire codebase.

First of all, I added role-based security to the system, extending authorization to support various levels of super-users, like admins and reviewers. This required a slight adjustment of use case infrastructure.

Second, Tomas asked to get rid of all stored procedures in the code. So I simply reconfigured use case verifier to throw errors on any module that declares stored procedures. A few more hours to clean up the code, and we are guaranteed not to have this problem any more.

Third, I introduced a quick way to stress test a single module in isolation by running a special stress utility. This is faster than generating Finland/Sweden events and passing them through the system. Stress utility works by taking all use cases of a module and using them to generate events over and over again. It works automatically and provides really fast feedback.

Fourth, I went through the entire codebase, replacing backend-side upsert statements with PSQL upsert queries, while also switching queries to use prepared statements. This helped to resolve a lot of concurrency issues (detected by stress tool) and speed up the execution by 40% on local machine and by 5-7% at the glesys environment.

Fifth, I made all event handlers idempotent. This is needed to handle message duplication in cases of network partitions (something that might happen with RabbitMQ). The solution was to simply extend our use case verifier. After performing a normal run, it goes through all the use cases, duplicating one event at a time and making sure that all expectations still match. Then I simply went through the solution, fixing all tests that appeared to be broken. Afterwards this logic was added to the build process, to ensure that new code will stay idempotent.

In this case we handle only idempotency, since this was a "low hanging fruit". Out-of-order messages is something we don't deal with, yet.

Pieter spent most of his time working on the performance and stability of the codebase. He used our Finland and Sweden datasets as a tool to make the system more robust. This also involved improving our RabbitMQ bus to make it reconnect on network failures (golang client doesn't do that out of the box).

Tomas was busy with maintenance of HPC1, tweaking it and moving the entire system to new servers on Glesys. This included migration of all 3 countries, SQL DBs, media files and app servers. Lot's of work to do.

User Interface

Tomas has a vision of a Single-Page Application (SPA), where we would have rich experience on the clients with the ability to support older browsers. This SPA could even be shipped as a native app to desktops and mobile phones (via something like PhoneGap), offering really nice user experience.

At the end of the last week I finally started working in this direction. Our primary choice of technology is rendr from AirBNB. It is a relatively small library that allows to run the same JS code on the server and the client, rendering HTML at either one, depending on the state.

Working with rendr actually involves touching multiple technologies: node.js, expresso stack, backbone.js, handlebars.js. So I started by picking these technologies one at a time and learning more about them. That's what I plan to focus on during the next week.

Published: September 07, 2014.

Next post in HappyPancake story: Getting started with Node.js, Lazojs

🤗 Check out my newsletter! It is about building products with ChatGPT and LLMs: latest news, technical insights and my journey. Check out it out