Efficient Development, Deployment and Customer Feedback at Lokad

A while ago I wrote about deployment and updates of the desktop applications and outlined the technology we use at Lokad for rapid and efficient delivery of various updates to the customers.

Let's expand on the topic a little bit more, showing how a small company could create quite a bit of business value via some efficient development practices.

There will be a small questionnaire at the end of this article as well.

Development and Deployment Experience

Current application deployment and publication loop for Lokad Client Applications works like this: when developers commit new features or fixes to version control repository and hit a special build button on integration server, following happens automatically:

• All unit tests, integration and code quality tests are executed against the latest codebase (build breaks if these fail).
• New version of install package (and optional downloads) is deployed to publicly available location.
• Download page is updated with the links to the new version.
• Existing users of an application get a nice "Update Available!" notification in the corner of their toolbar. Clicking on it launches semi-automatic download and upgrade process.

Note, that we are not using ClickOnce technology here, since it has important limitations we couldn't live with. Everything took a bit of time to setup once and then it is just started flowing.

Customer Feedback Experience

Recently, in order to enhance customer experience, yet another simple feature had been added to the client applications: integrated feedback reporting. Simply put, customer could click a button and send a message to the company. It will immediately show up in the customer support system. Additionally, all unhandled exceptions (should they happen) could be reported, too.

Feedback reports (if customers decide to) could include following anonymous information:

• system descriptor of the software (version, OS version);
• versions of the assemblies loaded into the AppDomain;
• latest portion of application log;
• statistics of some performance counters showing client-side information about interactions with our servers (number of communication failures, retries, amount of data transferred);
• exception counter statistics (information about unique exceptions handled by the application along with their counts).

Note, that sources for the SystemDescriptor, ExecutionCounter and ExceptionCounter are available in Lokad Shared Libraries. Actual feedback reporting shard is shared with the community in the Lokad SDK

Feedback system had been implemented as an infrastructure shard within the shared codebase. This implied IoC-friendly component-driven design and allowed to efficiently reuse all components in multiple applications (even in these that didn't have any IoC infrastructure around). All improvements and fixes to the shared components, in such scenario, automatically benefit all applications that use them.

Additionally, since the feedback system is implemented as an independent infrastructure shard, it can be used to trap and report possible exceptions that take place on the application startup (even before the Inversion of Control Container is setup). Later, when the application and composite UI elements are ready, we switch stand-alone feedback handler for integrated handler (that uses, application's Viewspace and action policies, for instance)

This reduced feedback friction for the customers and made issue reports a lot more useful and helpful for the development team. Since all feedback reports get routed to the internal customer support system, we could feel the change really fast and resolve problems and provide solutions more efficiently.

New Challenge - Update Flood

This kind of automatic and tight integration between development, deployment and customer support spawns new possibilities and brings forth new challenges. First of all, it definitely affects the way we think about the development and plan future changes.

Second, since it is too easy to deploy an update, we've got to start thinking about protecting customers from being flooded with a number of updates (software should solve the problems, not create new ones), since theoretically we could fire the deployment update after every single verified fix that is landed in the trunk.

Here's how one of the implementation scenarios looks like right now.

All software updates (packaged as a complete version, as usually) could be described with a:

• version;
• download url;
• tags (Critical, RC, Test, Major etc)
• optional description.

Primary download page is automatically updated with the links only when "Critical" and "Major" versions are released.

Customers are unformed about the Major updates to the installed software with a usual manner (i.e.: toolbar notification). Less-important updates are not shown to them explicitly but could be pulled (i.e.: from the menu), if customer needs that.

This scenario should work if we are facing scenario, when there is a minor fix affecting only a single customer. We could deploy an update via the normal procedure, but it does not make a lot of sense to push updates to every single customer.

Implementation should be rather straightforward:

• Instead of updating simple txt on the server per every update, we could update an XML file in the RSS/Atom format with the build version, url, description and tags.
• By default every application filters out all non-Primary updates, displaying the rest on the toolbar. This way we deliver important functionality to customers, while saving them from the barrage of less important builds.
• Customers might explicitly look into the update stream, picking and firing the specific update they need (i.e. when informed about the fix by the customer support).
• Beta testers and partners might update application settings to be notified about RC and Minor deployments
• Developers and Testers might switch their filtering settings to include Beta and Minor updates.

Potential synergy effects:

• Updates could be subscribed to via a mere RSS reader and mashed with a company news (automagically).
• Since we are already tracking Delivery-Build labels for issues, this information could be pasted to the description dialog, providing customers and testers with more context (could be available from the application and from the web/RSS).
• It is possible to push the idea even further and let customers set simple filter in their application ("inform me when ticket #X is fixed"). Then, whenever the next deployment is detected, a simple regex could scan the description for the ticket number, raising friendly alert, whenever it is mentioned there.
• If we have a critical update at hand (i.e.: breaking API change or an important security vulnerability) we can block the entire app and insist on updating.

Everything might get even more interesting as extensible and composable application architectures (with extensions being developed and delivered concurrently) get into the picture, allowing fine-grained and yet automated customer-company interactions.

Summary

In this short article we've walked over some development practices adopted in Lokad. These practices help the company to resolve customer problems and provide new solutions to their business challenges.

In short the outlined practices include:

• efficient software deployment and updates;
• efficient customer support and feedback collection;
• considering synergy effects and automation, whenever possible and feasible.

Of course, there's a lot more in efficient development, than these few practices. I'll address them eventually. You can subscribe to this journal to get all the updates. Voting for the next article to be written is also possible.

As always, all feedback to the article is welcome and appreciated. Here are also a few questions, if you are willing to share your experience with the community around this Journal:

• How much time does it takes for the customer to report a problem and get an update in your development environment?
• Does your company share the code with the community?
• Are you a customer of Lokad? What do you think about this development approach? How could it be improved?
• Are you a competitor of Lokad? What do you think? ))

Published: October 19, 2009.

🤗 Check out my newsletter! It is about building products with ChatGPT and LLMs: latest news, technical insights and my journey. Check out it out