Enterprise RAG Challenge

Enterprise RAG Challenge is a friendly competition that compares different RAG architectures. The goal is to build an AI-driven system that will be able to answer questions about annual reports of the companies.

You can find more technical details in this github repository.

Round 1

Round 1 was organised by TimeToAct Austria (read more).

Solution using Checklist pattern with Structured Outputs took the first place. Second place used a classical vector database with LangChain.

AIR - teams leveraged my AI Research
TTA - teams were a part of TimeToAct community

Round 2

Round 2 was organised by TimeToAct Austria (read more) and sponsored by IBM WatsonX AI.

Here are the top leaderboards for teams and experiments (regardless of their prize nomination status)

Hours - time it took the team to produce the results
R - Retrieval Score. Max: 100
G - Generation Score. Max: 100
Score - Final score (R/3+G). Max: 133
AI - teams leveraged my AI Research (through communities or TimeToAct)
Lcl - Local model was used

erc-r2-team-large-v3.png.png.png

erc-r2-sota-large-v3.png.png

Published: March 13, 2025.

Next post in Ship with ChatGPT story: ChatGPT quickstart for developers

🤗 Check out my newsletter! It is about building products with ChatGPT and LLMs: latest news, technical insights and my journey. Check out it out