Distillation

Distillation is the process of training a smaller, faster model to mimic the outputs of a larger one — getting most of the capability at a fraction of the cost.

If you're paying frontier-model prices for a high-volume task, distilling a small model on the frontier's outputs often gets you 90% of the quality at 5% of the cost. Common in production: use Opus to generate training examples, fine-tune Haiku on them, deploy Haiku.

Example

Acme uses Claude Opus for ticket classification at $0.012/ticket. After distilling to a fine-tuned Haiku, classification drops to $0.0004/ticket with no measurable accuracy loss.

How OpenLabor uses it

OpenLabor uses distilled small models on high-volume routine steps and saves frontier models for reasoning.

Does distillation work for any task?

Best for narrow tasks with stable patterns (classification, extraction). Reasoning still benefits from frontier models.

Related: fine-tune, llm, cost-per-task.

AI Labor Glossary