#engineering
9 papers
-
inspiration
The Tokenizer Decides What the Model Can See
A model never sees characters or words — it sees tokens. The tokenizer is a lossy, fixed decision made before training, and it quietly shapes what the model is good and bad at: arithmetic, rare words, code, and non-English text all live or die by tokenization.
-
inspiration
Fine-Tuning Is Usually Not the First Move
Reaching for fine-tuning to fix a model is often the expensive wrong turn. Most problems that look like they need fine-tuning are really retrieval or prompting problems. Fine-tuning changes behavior and style; it is a poor and costly way to inject knowledge.
-
whitepaper
The While Loop Is the Easy Part: Engineering Agents for Production
Every LLM agent converges on the same structure: call the model, execute tools, repeat. That loop is not where the engineering lives. The hard parts are termination conditions, context budget management, error classification, tool safety rails, and observability infrastructure — and most agents that fail in production fail there, not in the model.
-
inspiration
The Tokenizer Is the Bug
Every LLM failure starts with the same invisible step: tokenization. It runs before inference, produces no logs, and degrades outputs silently. Most debugging sessions end at the model. They should start at the tokenizer.
-
whitepaper
Evals Are Not Optional
Benchmark scores are not evaluations. Contamination is widespread, Goodhart's Law is in effect, and the gap between a leaderboard number and production behaviour is unbridged without a real eval pipeline. This paper defines what evals are, why the major benchmarks are unreliable in isolation, and how to build an evaluation practice that actually catches failures.
-
inspiration
Temperature Is Not Creativity
Temperature is a probability reshaper, not a creativity dial. Calling it a creativity parameter is a category error — one that leads to misconfigured systems and wasted inference budget.
-
inspiration
The Model Is Not the Agent
An LLM does not call tools. It requests them. The loop is the agent — and most broken agents are broken loops, not broken models.
-
inspiration
The Cost of Abstraction
Every layer you add is a layer someone else must debug.
-
inspiration
Context Is Not Memory
A large context window does not make an LLM remember. It makes it attend. The distinction changes how you build.