2026-10-15
Design agent tool authorization around each effect, not a cached tool list. Bound delegated scopes, enforce revocation, and recheck long-running sessions.
8 min read
Field notes · 2026
Defining well-performingagentic systems — one experiment at a time.
Practical essays on designing, evaluating and tuning meta harnesses for LLM agents.
ScaffoldsEvalsPromptsTool policyLoop architecture
Filter
2026-10-14
Design agent cancellation around durable intent, blocked dispatch, and effect reconciliation. Show what stopped, what completed, and what stays uncertain.
8 min read
2026-10-13
Measure agent evaluation uncertainty at the task level. Separate repeated trials from evidence, compare releases in pairs, and keep inconclusive verdicts.
8 min read
2026-10-12
Prevent agent memory poisoning with write admission, source provenance, correction chains, and quarantine. Keep retrieved claims from becoming authority.
8 min read
2026-10-08
Use agent tool schema versioning to catch silent changes in defaults, units, outputs, and authority. Test migrations and keep rollback tied to effects.
8 min read
2026-10-07
Build agent concurrency control with revision checks, leases, and fencing. Reject stale workers at the write boundary and test races before they hit users.
9 min read
2026-10-06
Audit agent evaluation graders before trusting a passing score. Separate state checks from model judgment, calibrate false passes, and version each rubric.
8 min read
2026-10-05
Design agent memory retrieval around provenance, freshness, and conflicts. Verify sources before acting, and abstain when recall cannot support a claim.
8 min read
2026-10-01
Build production agent architecture around queues, durable state, scoped executors, and verifiers that distinguish a finished attempt from a verified task.
8 min read
2026-09-30
Agent context window management preserves evidence across compaction. Set budgets, retain references, and rehydrate facts before consequential actions.
9 min read
2026-09-29
Build agent evaluation datasets around evidence and failure boundaries. Separate difficulty from noise, prevent leakage, and test shortcuts before release.
8 min read
2026-09-28
Build agent recovery checkpoints that preserve intent, reconcile uncertain effects, and revalidate authority before interrupted work resumes after a crash.
8 min read
2026-09-24
Test agent guardrails against forbidden effects, not reassuring replies. Build replayable attacks, verify state, and keep legitimate tasks in test suites.
7 min read
2026-09-23
AI agent cost optimization cuts repeated context and tool overhead while preserving verified outcomes. Measure savings at the task boundary, not per call.
8 min read
2026-09-22
Catch multi-agent failure modes at handoffs: reject stale results, resolve state conflicts, and give final writes one owner with a checked recovery path.
7 min read
2026-09-21
Handle agent tool call errors by separating invalid requests, execution failures, and unknown outcomes. Recover safely without repeating external effects.
7 min read
2026-09-14
Design multi-agent orchestration with independent tasks, evidence-backed handoffs, and one owner of final effects; weigh gains against coordination cost.
6 min read
2026-09-09
An agent framework comparison should evaluate the harness surface: state, tools, control flow, observability, and safe promotion—not feature counts.
8 min read
2026-09-08
Agent regression gates turn known failures into release blockers, so every repaired incident becomes a durable test before the next deployment.
8 min read
2026-09-07
Agent sandboxes limit permissions, isolate side effects, and make recovery explicit so a bad run stops locally instead of becoming an incident.
8 min read
2026-09-03
MCP is useful when it makes tool boundaries explicit. The real tool-protocol decision is how you preserve authority, schemas, observability, and change control.
8 min read
2026-09-02
Human-in-the-loop escalation design gives agents clear stop rules, useful handoffs, bounded authority, and a path to learn from review.
8 min read
2026-09-01
Cost per completed task turns agent spend into an operational metric: include retries, human review, failures, and the price of making work right.
8 min read
2026-08-31
Long-running agents need memory outside the context window. Design working, episodic, semantic, and procedural memory with retrieval and expiry.
8 min read
2026-08-29
Agents retry, resume, and get interrupted. Idempotency makes those reruns safe: same intent, one durable outcome, and an audit trail.
8 min read
2026-08-27
An agent retry with the same prompt and same context produces the same failure. Diversified retry policies turn repeated failures into forward progress.
7 min read
2026-08-24
Agent verification lives on a spectrum from cheap schema checks to expensive ground-truth comparison. The art is matching the verifier to the risk at each step.
8 min read
2026-08-22
A model swap breaks agents in predictable ways: tool-call formats drift, stop conditions change, costs spike. Here's the drill to run before you deploy.
8 min read
2026-08-21
Most agent logs are noise. The signal lives in tool calls, decision boundaries, and failure traces. Here is what to record, what to throw away, and why.
7 min read
2026-08-19
Prompt versioning turns agent regressions from archaeology into a diff. Version, review, and roll back prompts with the discipline you apply to code.
8 min read
2026-08-18
Agents fail sideways: runaway loops, blown budgets, tools misused at 3am. Guardrails and kill switches turn an agent from a liability into infrastructure.
8 min read
2026-08-17
An agent harness is the scaffolding around an LLM that makes it reliable — tools, context, loops, and guardrails. The anatomy of a well-performing agent.
9 min read
2026-08-16
Most agent failures are loop failures. Design stop conditions, verifiers, and retry budgets that make LLM agents finish reliably instead of wandering.
11 min read
2026-08-15
What your agent knows at inference time is decided by context engineering, not prompts. Compaction, retrieval, and window hygiene for LLM agents.
12 min read
2026-08-14
The tool surface is the agent's API to reality. Design LLM agent tools that are hard to misuse: small surfaces, strict schemas, scoped permissions.
8 min read
2026-08-13
Model benchmarks don't predict agent performance. How to build task-level evals: golden traces, scenario suites, regression gates, scoring harness design.
10 min read
One experiment, every week
Each post ships with the harness config, the eval suite and the traces. No newsletter theatre — just the run log.