Lessons from systems in production.
Evergreen notes, not a feed. Each piece is drawn from a system I actually built and shipped, and links back to its case study as the worked example. The throughline: production AI needs the same care that safety-critical engineering takes for granted.
How I Evaluate AI Systems
A model's headline benchmark is not its value. What matters is whether its output survives contact with the rest of the system: structure, cost, and the cleanup a wrong answer forces later.
Where Not to Call the Model
On a real budget, the highest-leverage decision in an AI system is often not calling the model at all. Cost tracks the work you route to it, so route only what genuinely needs it.
Safety-Critical Rigor in Production AI
What keeps a regulated system safe: code makes the decisions, every boundary validates, the system stops when in doubt, and anything irreversible needs a human to approve it. That is exactly what production LLM systems are missing.
Keeping Agentic Workflows Debuggable
When an LLM can take actions, the hard part is no longer capability but observability and bounded autonomy. (Forthcoming.)