Skip to content
Road to Intelligence

Concept · Chapter 13: Agents

Tool Execution in Agents

7 minDifficulty

A model proposes a tool call, but separate software validates, authorizes and executes it.

The problem

Well-formed output can still request an irrelevant or unauthorized action.

The solution

Check syntax, scope and permission before dispatch, then inspect the resulting evidence.

The consequence

A schema is an interface contract rather than a guarantee of correctness.

You should understand first

  1. Text as Data
  2. Probability and Distributions
  3. Conditional Probability and Bayes' Theorem
  4. Probability of Sequences
  5. Language Modeling
  6. Vectors
  7. Dot Product
  8. Embeddings
  9. Attention
  10. Softmax
  11. Self-Attention
  12. Causal Masking
  13. Entropy
  14. Loss Functions
  15. Cross-Entropy Loss
  16. Autoregressive Next-Token Prediction
  17. Pretraining at Scale
  18. GPT-1 → GPT-2 → GPT-3
  19. In-Context Learning
  20. The Turing Test
  21. Symbolic AI
  22. Logic and Rules
  23. Expert Systems
  24. Knowledge Representation
  25. The Knowledge-Acquisition Bottleneck
  26. From Rules to Learning
  27. Supervised, Unsupervised and Self-Supervised Learning
  28. Expected Value and Variance
  29. Reinforcement Learning
  30. MDPs, Policies and Value
  31. Decoding: Greedy, Temperature, Top-k, Top-p
  32. One-Hot Encoding
  33. Tokenization
  34. Chat Templates
  35. System Prompts and Instructions
  36. Structured Outputs and Constrained Decoding
  37. Tool Calling
  38. LLM Agents
  39. Agent Runtime
  40. Tool Execution in Agents

Three checks

For a proposed read of src/total.ts, the schema can require a string path. The permission layer checks whether that file is in scope. The returned content determines whether the action actually helped.

Clear tool descriptions reduce ambiguity. Clear observations reduce uncertainty: return the affected version, relevant excerpt or explicit failure rather than a generic “OK.” Toolformer studies learning API use; SWE-agent studies an interface designed around an agent's needs.

Writes need stronger semantics

A tool that patches a file can require its expected current version to prevent applying an edit to stale content. A payment tool can use an idempotency key to avoid repeating a transaction after a timeout. These properties belong to the tool and runtime, even if the model knows how to describe them.

Try it · toy model

Hold the Boundary

A planted line in a bug report tells the agent to leak a secret. Choose the runtime's rules, approve or deny writes as the run pauses for you, and see what leaked, what got fixed and what changed that shouldn't have.

8 min

What to remember

  • A model proposes a tool call, but separate software validates, authorizes and executes it.
  • Check syntax, scope and permission before dispatch, then inspect the resulting evidence.
  • A schema is an interface contract rather than a guarantee of correctness.

Key papers

Essential

Toolformer: Language Models Can Teach Themselves to Use Tools

Timo Schick, Jane Dwivedi-Yu et al. · 2023

Showed a model can learn when to call a calculator, search engine or calendar and how to use the result, the core idea behind tool calling.

~40 min readarXiv:2302.04761✓ verified 2026-10-05