Back to docs

Recipe: Tools dispatcher + validator

Route tool calls through a central dispatcher that validates parameters, enforces rate limits, and returns structured errors.

Overview

The dispatcher sits between your LLM agent and raw tool implementations. It validates JSON schemas, checks per-tool rate limits, and normalizes error shapes so the agent can recover gracefully.

Architecture

Agent
  |
  v
Dispatcher
  |-- validate(schema, args)
  |-- rate_limit(tool_id, user_id)
  |-- dispatch(tool_id, args)
  |
  v
Tool implementations

Key decisions

  • Schema-first: every tool declares a JSON Schema. The dispatcher rejects invalid calls before they reach implementation code.
  • Token-bucket rate limiting: per-tool, per-user buckets with configurable refill rates prevent runaway loops.
  • Structured errors: all failures return a standard envelope so the agent can parse and retry intelligently.

Error envelope

{
  "error": true,
  "code": "RATE_LIMITED",
  "message": "Tool 'search' exceeded 10 req/s",
  "retry_after_ms": 1200
}

Next steps

Combine this recipe with the agent loop recipe for a complete runtime. Add observability hooks to log every dispatch for debugging.

Back to docs