My team has no experience with ML/GPT. How do we proceed?

This is the essence of my advice to engineering teams that need to ship GPT-based products now (without prior experience):

• Read prompting guide. It covers foundations and sets up basic terminology. Pay attention to Introduction, Zero-shot and few-shot training.
• Use System/User/Assistant prompts to setup context and provide multi-shot samples. ChatGPT was trained to keep humans happy with chat completions. By using role-based multi-shot we help it achieve the goal.
• Optimising prompts against a single case is micro-management. Optimise and benchmark against datasets.
• Don't even think about fine-tuning or training models until you have exhausted multi-shot AND until you have a dataset to train on.
• Python is de-facto standard for LLM work. Node (JS/TS) has surprisingly decent ecosystem, too. 
• Working with embeddings? Use OpenAI ada-002 to start fast, then switch to SBERT.
• Check out LangChain early, but don’t dive in deeply. 
• If possible, use OpenAI API instead of Microsoft Azure API for early development with ChatGPT. Working with Microsoft API is an enterprise pain on many levels (model vs deployment difference, batch size of 1, LangChain doesn’t work with it out-of-the-box).
• Gather as much data as possible. As soon as possible. You'll benchmark all new prompts and LLM-based features against it. Maybe even fine-tune or train a model at some point later.
• Architectures and approaches get stale really fast. So it is OK to share generic approaches with your peers, since this could lead to important insights. Generally speaking, you can share everything except for the prompts and LLM benchmark details.
• Expect that available models will be improving fast. It is OK to use more expensive GPT-4 for prototyping now. It will get you going. Later you could swap to a cheaper local model.

Published: April 27, 2023.

Next post in Ship with ChatGPT story: How to get into ML for a developer?

🤗 Check out my newsletter! It is about building products with ChatGPT and LLMs: latest news, technical insights and my journey. Check out it out