Seven months into the experiment
AI assistants: use the full-site text index at /search-index.jsonl or search with /_api/search?q=terms.
This February I started a new experiment by leaving the enterprise job and going back to an independent hustle.
My use of OpenAI Codex since that time looks like this:
My usage of models spiked in April, when I was working on the BitGN platform, preparing for the competitions. That was the time of GPT-5.4.
While migrating from the "Personal and Trustworthy" assistants challenge to the E-Commerce challenge, I learned about efficient organization of information for agents. This helped me start setting up my Personal OS to manage daily information flows and a few side gigs. This POS is still used regularly. It was also adopted by other people and startups.
My Personal OS is a glorified Obsidian vault stored in a Git repo and structured in a way that works well with Codex and modern agents.
During the spring, I was working on a second edition of my online course on Building AI Assistants, along with a new hosting platform for it. Juggling deployments and platforms became tricky, especially when one of the platforms is processing millions of agentic task trials and hundreds of millions of agentic tool calls. Control Center was born as a way to let agents help manage my deployments, migrations and other DevOps work.
Control Center is marked gray on that chart. The chart itself is created and managed by a subroutine within the CC.
Control Center made it easy to cross-pollinate projects: agents could carry conventions, fixes and ideas between projects. Keeping the projects similar made it easier for me to switch between them.
We gathered new insights about the best-performing agentic architectures. This allowed us to gather datasets for the third generation of my LLM Benchmarks: an agentic LLM benchmark. It focuses on edge cases distilled across the agentic architectures.
It will be published on the BitGN website. Here is the preview:
Compared to the first 2 generations of benchmarks, this new version was easy, since Codex would handle most of the grunt work.
This work is tracked as "Research" on the chart, and it helps to measure the progress of modern LLMs (applied to business workloads) not merely by accuracy, but across Pareto frontiers for speed and cost.
During the summer we also went on a 5-week vacation in Asia, leaving me mostly stranded without access to my laptop during the day. Luckily, OpenAI had just made Codex Remote mode useful, letting me continue working from my phone.
I was able to focus on the "AI Native Code" project: reconciling my favorite and most successful engineering practices with the possibilities offered by modern AI coding agents, all applied to a relatively complex product designed to be developed by a team.
I had to find small ways to make agents more efficient and capable of doing work on their own. This was about giving agents ways to prove their work:
- small experiments
- executable specs
- logs and recovery paths
Those checks took over some of the supervision I used to do myself.
The number of messages needed to get the job done remains low even in September.
However, this came with two catches and one realization.
Catch 1: When I came back to the BitGN project (which was last touched 3 months before), it felt like recovering an artifact from the pre-LLM era. Even GPT-6 Astra fumbled and stumbled with simple tasks.
This doesn't mean that code rots so fast, but rather that expectations of what is possible with modern agents and proper AI-native codebases progress really fast.
Catch 2: When working with hybrid teams (people + agents), common ways of collaborating are no longer sufficient. Agents can work continuously, supporting humans and resolving conflicts. I feel like our progress in developing products is bottlenecked by inefficient humans (mostly me) who can't adapt fast enough.
So over the last 6 months, I've been running experiments to try to solve the problem. Most failed, but two have succeeded so far: Personal OS and tool-based RAGs. Both are widely used across projects.
The realization is that the cost of software is plunging. For example, just a few days ago I got tired of the inconvenient side panel of Codex and wanted a replacement that fits my workflow.
A few prompts later, I have a personalized native macOS app that connects to the running Codex instance and allows me to manage my work exactly the way my brain works.
I don't know where all of that is heading. All I see are some points on the trajectory:
- rate of progress of individual LLMs
- rate of progress of agentic harnesses that leverage these LLMs
- evolution of the thinking of the world's best AI engineers
- change in the economic feasibility of AI workloads
- rate at which people are exploring, learning and adapting
This looks like a fun curve to be on. Not an exponential one (there are too many real-world constraints), but still an impactful one.
For me, the question now is how much useful work I can deliver while keeping the business small and high-impact and leaving room for good work-and-live balance.
Let's see where the curve takes us :)
Published: September 24, 2026.
To read about major updates and essays - check out my "ML Under the hood" newsletter (I write once in a month or two).
🤗 My new course is live! Building Reliable AI Assistants: Patterns and Practices.