The Model Is Not the Agent

When a model uses a tool, here is what actually happens:

1. You send a message plus a schema of available functions.

2. The model outputs a structured JSON object — a request to call a function.

3. Your code executes the function.

4. You send the result back to the model.

5. Repeat until done.

The model never executes anything. It generates text that describes what it wants to happen. The loop — your code — does the work.

This distinction matters.

Where Agents Break

Most agent failures are loop failures, not model failures.

The model hallucinates a tool name. Fix: validate outputs against your schema before dispatching. The model does not know which tools you actually registered — it predicts based on training and context. If your schema is ambiguous, so is the output.

The model calls tools in sequence when parallel calls would work. Fix: parallel tool calling is a capability you must design for. Sending 5 weather lookups sequentially when they are independent is a loop design mistake, not a model limitation.

The model loops forever. Fix: your loop needs a termination condition. The model has no concept of done beyond what you encode. Add a step counter. Add a goal-completion check. The model will not stop itself.

The result gets lost. Fix: every tool result must return to the model with enough context to be useful. A bare status 200 tells the model nothing it can reason about.

The Right Mental Model

Think of the model as a reasoning core — stateless, context-bound, isolated. Think of the agent loop as the scaffolding that gives it continuity, action, and memory.

The loop maintains state, dispatches calls, handles errors, and decides when to stop. The model decides what to call next, interprets results, and synthesizes a final response.

Neither is sufficient alone.

Practical Consequence

Before debugging a flaky agent, ask: is this a model problem or a loop problem?

Nine times out of ten, it is the loop. The prompt is incomplete. The schema is ambiguous. The error handling is absent. The termination logic is missing.

Fix the loop. The model will follow.

References

• Schick, T., Dwivedi-Yu, J., Dessi, R., Raileanu, R., Lomeli, M., Zettlemoyer, L., Cancedda, N., & Scialom, T. (2023). "Toolformer: Language Models Can Teach Themselves to Use Tools." NeurIPS 2023. https://arxiv.org/abs/2302.04761
• Yao, S., Zhao, J., Yu, D., Du, N., Shafran, I., Narasimhan, K., & Cao, Y. (2023). "ReAct: Synergizing Reasoning and Acting in Language Models." ICLR 2023. https://arxiv.org/abs/2210.03629
• OpenAI. (2024). "Function Calling and Tool Use." OpenAI API Documentation. https://platform.openai.com/docs/guides/function-calling

devinfo.dev. (2026). "The Model Is Not the Agent." devinfo.dev:2026.0008. https://devinfo.dev/d/2026.0008