Weekly Digest
System Prompts and Production Readiness
Feb 5 – Feb 11, 2024
This week focused on building reliable AI systems. Key insights on writing persistent system prompts and handling hallucinations in production environments.
Top Tips This Week
All Tips from This Week
How do I handle AI hallucinations in production?
Layer defenses: ask the model to cite sources, implement fact-checking against known databases, flag low-confidence responses for human review, and set clear expectations with users that AI can make mistakes. Never fully automate high-stakes decisions.
Should I use GPT-4 or a fine-tuned smaller model?
Use GPT-4 for prototyping and diverse tasks. Fine-tune smaller models when you have consistent, narrow use cases with thousands of examples. Fine-tuned models are cheaper to run but expensive to train and maintain. Start with GPT-4, only fine-tune if cost becomes prohibitive.
How do I write system prompts that stick?
Put the most important instructions at the start AND end of the system prompt (primacy and recency effects). Use clear formatting like numbered lists. Repeat critical constraints. Test with adversarial inputs to see if the model breaks character.