Agent Tool Schema Versioning: Catch Silent Breaks

Agent tool schema versioning is not finished when old requests still parse. A tool can accept the same JSON while changing which records it searches, what a number means, or whether an operation takes effect immediately.

The takeaway: version the executable contract, not just the argument shape. Bind discovery, validation, execution, and verification to an identifiable release. Test what the operation means before letting a long-running agent cross a migration boundary.

This is a design guide. The examples and migration cases below are illustrative, not results from a completed experiment.

JSON Schema compatibility has a direction

Suppose a tool accepts a customer identifier and returns that customer’s open projects. The next release adds an optional include_archived argument. Old requests might still validate against the new input schema. That is only one compatibility question.

Can an old server accept requests generated from the new definition? Does omission preserve the old behavior? Can an old client read the new response? Does an agent that sees the new description select the tool under the same circumstances?

The JSON Schema object reference makes two relevant distinctions: listing a property does not make it required, and additional properties are allowed by default unless the schema restricts them. Consequently, an apparently additive field can fail against an older closed schema. Conversely, an open schema can accept a field that an implementation silently ignores.

Neither outcome is captured by saying that a change is “backward compatible” without naming the producer and consumer.

Change	Compatibility question	Required evidence
Add an optional input	Do old requests retain their meaning on the new server?	Omitted-field cases and preserved defaults
Add a required input	Can old callers still invoke the operation?	Explicit adaptation or a separate contract
Add an output field	Do old readers tolerate unknown fields?	Actual consumer and validator checks
Add an output enum value	Can existing decision logic handle it?	Branch coverage or a defined unknown-state path
Change a description	Does tool selection or argument interpretation change?	Agent-level task cases with the new definition
Keep the schema, change units	Will the same valid request cause the same intended effect?	Semantic fixtures and postcondition checks

These are review questions, not automatic verdicts. An additive change can be compatible for one client and incompatible for another. Record which client, adapter, validator, and schema dialect were tested.

Semantic drift survives valid tool arguments

Consider an illustrative schedule_reminder tool with a numeric delay field. Release A interprets the number as minutes; release B interprets it as seconds. Both accept an integer. Both may return a successful response. A type validator cannot establish that the reminder was scheduled at the intended time.

Less obvious changes deserve the same scrutiny:

• A search tool includes archived records unless explicitly excluded.
• A timestamp without an offset switches from a tenant timezone to UTC.
• A result changes from a complete set to the first page without a clear continuation signal.
• A write changes from synchronous completion to queued acceptance.
• A retry key stops sharing a deduplication namespace with the previous release.
• A read operation begins updating a “last viewed” field that triggers downstream automation.

For each tool, maintain a short semantic contract alongside its schema: units, defaults, ordering, pagination, authorization scope, side effects, error meanings, and what constitutes completion. Include the evidence a caller must obtain before announcing success.

This extends the argument in tool-protocol governance: a shared transport can standardize how a call is expressed without establishing that two implementations mean the same thing.

A description fix may be a patch in your release policy. A changed default may require a separate major contract. The important part is that a written compatibility policy determines the version, rather than a version number standing in for that policy.

Pin tool discovery to an executable release

The MCP tools specification dated 2025-06-18 defines tool discovery, input schemas, optional output schemas, and notifications when the tool list changes. It also distinguishes protocol errors from tool-execution errors. These are useful integration mechanisms, not a complete application migration policy.

A notification tells a client to refresh its view. It does not prove that a saved plan remains valid under the refreshed definitions. A protocol revision identifies the wire contract; it does not uniquely identify the business behavior behind a tool name.

A practical harness can keep an immutable release manifest containing:

1. The tool definitions actually shown to the model, including descriptions.
2. The input and output schema artifacts and validator configuration.
3. The executor or server release and any adapter release.
4. The semantic-contract revision and applicable authorization policy revision.
5. The verifier revision used to judge completion.

Retain the artifacts behind each identifier. A hash establishes identity only for the bytes it covers; hashing a schema does not pin a mutable remote implementation. If the provider cannot supply an immutable behavioral version, record that limitation and monitor it through controlled contract checks.

At dispatch, bind the selected tool contract to an executor that implements it. A gateway can route a versioned tool name or authenticated deployment identifier; merely logging the expected version after a call cannot prevent mismatch. Resolve the route once for the invocation so a rollout cannot swap the implementation between validation and execution.

For long-running tasks, decide explicitly whether to keep using the old contract, adapt the saved request, or pause for replanning. Do not silently replace the definition underneath an already generated call. Current authorization still applies: pinning old semantics must not preserve permissions that have since been revoked.

Treat adapters as migration code

Suppose the old reminder tool accepts delay_minutes, while the replacement accepts an absolute scheduled_at timestamp. An adapter might translate one representation into the other, but only after defining the reference clock, timezone, rounding, and retry behavior.

Recomputing the absolute timestamp on every retry would move the reminder. A faithful adapter resolves it once, persists the interpreted intent, and reuses that value. It also checks the replacement’s time-range constraints and maps the response back to the old completion meaning.

A queued receipt cannot honestly become an old-style “completed” response merely because the adapter needs a matching field. The adapter must wait for the evidence the old contract promised, or reject the incompatible operation through a documented failure path.

Keep adapters narrow and observable. Record source contract, destination contract, interpreted arguments, and response mapping, subject to normal data-retention limits. Give the adapter its own fixtures and retirement criteria. Its existence does not make unsupported old behavior disappear.

Build migration tests around outcomes

Use three layers of checks before switching traffic.

Structural checks exercise accepted and rejected inputs, missing fields, nulls, unknown fields, boundary values, and output variants. Run them through the actual client and server validators, not only a generic schema-diff tool. Provider-supported schema subsets and adapter normalization can change what reaches execution.

Semantic checks invoke the operation against isolated fixtures and inspect the resulting state. For a reminder, compare recipient, scheduled instant, message, and effect count. For a search, verify scope and whether the result is complete. For a queued write, distinguish accepted work from committed work.

Agent-level checks test selection, argument construction, error recovery, and interpretation of the result with the full proposed tool surface. A renamed tool may implement identical behavior yet change which operation the model chooses. Keep the model, task suite, permissions, and unrelated tools fixed when measuring that migration.

The following are proposed acceptance cases, not a report of executed model runs:

Fixture	Expected release behavior
Old request omits the new optional field	Preserve the documented old default
Old client receives a new enum value	Follow a tested fallback or reject explicitly
Task resumes with an old pending call	Route to its supported contract or migrate before dispatch
Tool list changes after planning	Detect a stale binding before executing incompatible semantics
Request times out after the write	Reconcile the existing effect without creating another
New server returns queued acceptance	Do not declare completion until the promised evidence exists
Permission is revoked during rollout	Reject the write even if the old contract remains available

Tie these cases to agent regression gates. Keep historical failures, changed-default controls, and incorrect-output fixtures. Compare verified outcomes, not just whether the agent produced a fluent final answer.

Shadow comparisons need an effect boundary. Replay reads only where their actual behavior and data access permit it. Compare writes in isolated copies or validated dry-run implementations; do not send a second production mutation merely to compare releases. A dry run itself needs evidence that it cannot commit effects.

Rollback changes routing, not history

A successful rollback redirects future calls to a supported release. It does not unsend messages, restore overwritten data, or resolve requests whose outcomes are unknown.

Before rollout, establish how old and new executors recognize the same logical operation. If deduplication is meant to span versions, preserve a stable operation identity and compatible ledger semantics. Otherwise, a retry routed to the previous release can look like a new instruction.

The idempotency contract for agents belongs in the migration plan, not in an incident follow-up.

Also check whether the old release can read state written by the new one. A schema expansion may be reversible at the API boundary while introducing a persisted enum or timestamp representation the old executor cannot understand. Keep the supported rollback window explicit, including database migrations, active tasks, and outstanding asynchronous jobs.

Retire a contract only after checking queued calls, resumable checkpoints, configured clients, and relevant retention windows. If an expired task later resumes, reject its unsupported contract with a recoverable status rather than guessing what its arguments used to mean.

A tool release needs a completion argument

A reviewable release should answer four questions: what changed, which caller combinations remain supported, what evidence establishes equivalent behavior, and what happens to work already in flight.

Store those answers with the release manifest and migration results. Monitor invalid requests, unknown output states, reconciliation failures, and verified task outcomes by contract version after rollout. A syntactically quiet release can still be semantically wrong.

Version numbers help people coordinate. Pinned executors, explicit meaning, outcome tests, and an honest rollback boundary make that coordination enforceable.