LLM Benchmarks Test Knowledge, Not Behavior
What today’s LLM benchmarks actually measure, what they miss, and why we need behavioral benchmarks instead of more knowledge tests.
What today’s LLM benchmarks actually measure, what they miss, and why we need behavioral benchmarks instead of more knowledge tests.
Two requests with near-identical input tokens, a 46x cost gap. How prompt caching really works, and why agents should be designed around structured state.
Coding agents re-read the same files and re-ask the same questions every day. What memory does for an agent, how it differs from docs, and when it pays off.
Bigger context windows didn’t give better results. Notes on the context-management problems that keep recurring in real coding agent sessions.