Slancha blog
Guides on making LLM and agent evals faster and cheaper. Every figure comes from a cited public source, not from Slancha results. Subscribe with the RSS feed.
How to build agent evals from production traces
Keep the full trace, pick cases on purpose, write the expected behavior and check in code first.
How many examples does an LLM eval need?
How many independent examples a pass rate needs, with margin tables and paired comparisons.
Can a cheaper LLM judge do the job?
Check whether a cheaper judge agrees with your labels before you switch.
Why LLM evals cost what they do
The cost formula, a worked example and published benchmark costs.
How to speed up LLM evals
Run a curated subset, cache the rubric, check in code and send calls concurrently.
How to reduce LLM evaluation cost
Cut the number and the price of judge calls before you change platforms.