Agent Infrastructure Is the Product Now

A solo developer just shipped Shikigami, a free desktop app that runs multiple AI coding agents in parallel, each isolated in its own git worktree so they never clobber each other's edits. Monaco editor. Diff viewer. Git history browser. MySQL and Redis inspector. Docker Compose management. Per-agent PTY terminals. Sessions that save and resume.

That is not a model. That is an operating system for agents.

It hit Hacker News the same week Anthropic published a case study on running its own coding agent across thousands of files in production. The pattern is identical in both: the model is no longer the product. The infrastructure around the model is the product. If you are still thinking about which model to use, you are thinking about the wrong layer.

An agent is not a model. A model generates tokens. An agent runs for hours, keeps state, calls tools, recovers from errors, and touches your filesystem, your database, and your network. The tooling that makes any of that safe, cheap, and observable did not exist a year ago. It is being built right now, in parallel, by frontier labs and solo developers. The convergence this week is unmistakable. Five separate releases point at the same missing layer: the operating system for agents.

What an Agent Needs That a Model Does Not

A model answers a prompt. An agent pursues a goal across many steps. That gap requires five things a bare model API does not provide.

1. Isolation. Two agents editing the same working tree destroy each other. Shikigami's answer: one git worktree per agent.
2. Cost control. Calling the frontier model for every step bankrupts you. FlexInference's answer: a router that drops costs by an average of 47 percent.
3. State and resumability. Agents crash. Networks drop. Sessions must survive. Shikigami saves and resumes sessions exactly where they left off.
4. Verification. Agents produce wrong output. Anthropic's answer: review gates that catch the 5 to 10 percent of migrations needing human correction.
5. Resource bounds. Agents with unbounded memory or storage create hidden costs and failure modes. The arXiv paper "The Hidden Footprint" argues storage should be a first-class evaluation metric.

None of these problems are model problems. They are infrastructure problems. The model layer is mostly solved. The infrastructure layer is not.

Isolation: One Worktree per Agent

Running multiple coding agents in terminal tabs means they fight over the same working tree. One agent's edit clobbers another's. No clean rollback.

Shikigami's design is brutally simple. Each agent runs in its own git worktree. Worktrees are cheap, branchable, and disposable. If an agent goes off the rails, you throw away the worktree, not the repo.

For readers who have not used git worktrees: a worktree is a separate working directory pointing at the same repository. Multiple agents can edit different branches simultaneously without interfering.

The worktree pattern is the agent equivalent of process isolation in operating systems. You would not run two processes in the same memory space. You should not run two agents in the same working tree.

This matters beyond coding agents. Any agent that touches a filesystem needs the same isolation. Scraping agents. Data-processing agents. Research agents. The worktree is the universal primitive.

Cost Control: Two Sides of the Same Coin

For teams using API-based models, FlexInference is an LLM routing service that drops inference costs by an average of 47 percent by routing requests to cheaper service tiers within a user-defined time budget.

The "flex race" pattern: try a cheaper tier first. If the request cannot start within the SLA window, escalate to the standard tier before writing a single token. The miss costs nothing.

It is built on Cloudflare Workers across 300-plus cities. Routing adds 1 to 5 milliseconds on a cold start. The company claims prompts are never stored or read. It now supports Bedrock and Vertex as providers, with automatic fallback on 429 and 5xx errors.

You do not have to pick one model. You pick a budget and a deadline, and the router picks the cheapest model that can meet both. This is the same pattern as spot instances in cloud computing, applied to inference.

The 47 percent figure is an average across mixed workloads. Your savings depend on your latency tolerance. Batch and background work wins most. Real-time, user-facing work wins least. The router makes the tradeoff explicit instead of hiding it in a fixed model choice.

But API routing is only half the picture. For developers running self-hosted infrastructure and open-weights models, the cost metric shifts away from API billing entirely. It becomes about local VRAM management, compute load, and hardware thermal limits. Running a 70B parameter model locally means allocating 40-plus GB of VRAM per instance. Run three agents in parallel and you are either pooling GPU memory or swapping to system RAM, which destroys inference speed. The hidden cost is not tokens. It is hardware saturation. The same agent that costs pennies on an API might cost nothing in direct billing but peg your GPU at 100 percent for hours, throttling every other process on the machine.

This is where cost control meets resource bounds. The FlexInference router solves API spend. The self-hosted equivalent is a scheduler that queues agent runs against available GPU memory, preempts low-priority tasks when VRAM runs out, and measures cost in watt-hours and thermal headroom instead of token count. Neither approach is complete without the other. The article's resource bounds section covers storage. The same logic applies to compute.

The token budget is no longer hypothetical. The tools that enforce it are shipping.

Verification Gates: Anthropic's Production Pattern

Anthropic published a case study on using Claude Code for large-scale code migrations. Thousands of files. API renames. Import path updates. Type migrations.

Claude Code operates on the actual codebase with full context, not in isolation. Anthropic built review gates and rollback mechanisms to catch the 5 to 10 percent of migrations that need human correction.

Engineers review diffs rather than writing migration scripts themselves. This is standard practice for internal refactors at Anthropic.

The architectural lesson: the agent does not commit directly. The agent produces a diff. The diff passes automated checks. Only failures or risk-threshold crossings reach a human.

The review gate is not a slowdown. It is what lets you trust the agent in the first place. Without it, you cannot deploy agent output to production. With it, a 5 to 10 percent error rate is acceptable because the gate catches it.

State, Resumability, and Resource Bounds

Shikigami saves sessions and resumes them exactly where they left off. Quit and reopen, and the agent's state is intact. This sounds trivial. It is not. Most agent frameworks still treat a session as a single unbroken process. When it dies, the work dies with it.

The arXiv paper "The Hidden Footprint: Making Storage a First-Class Metric for LLM Agent Evaluation" argues that storage footprint should be a first-class evaluation metric for LLM agents, alongside accuracy and latency. Agents with unbounded memory consumption create hidden costs and failure modes that standard benchmarks miss. The authors found that configurations with identical models, tools, and tasks, and 100 percent accuracy, differ by up to 15.7 times in retained bytes. A content-addressed store can reduce retention by 4.8 to 32.7 times while preserving every reconstructability score.

The arXiv paper "Compile, Then Page: Executable SOP Programs and a Capability-Gated Runtime for Procedural LLM Agents" describes a runtime that separates what an agent can do from how it does it, enabling safer delegation. Capability-gated runtimes limit agent permissions to exactly what the current step requires. The authors show that compiled SOP guidance never significantly hurts performance and gains up to 16.0 points where official prose underperforms.

Three papers, one conclusion. Resumable state keeps work alive. Bounded storage keeps costs predictable. Capability-gated runtimes keep actions safe. That is an operating system.

The Multi-Agent Workforce Pattern

The unit of work is shifting from "one model, one prompt" to "a team of agents, each with a role, handing off to each other and to humans."

Prodigy provides on-demand subagents working as autonomous teammates alongside human teams. Agents assigned to specific roles and workflows, with handoffs between agents and humans.

The mathematician's guide "AI Agents for the Working Mathematician" describes running an agent for hours on a conjecture, keeping track of its own progress, learning from mistakes, avoiding retried approaches. The agent uses Python, LaTeX, SAGE math, and browser access, the same tools a mathematician would use.

The MAG benchmark from arXiv evaluates web agents on both performing tasks and explaining what they are doing. Even the strongest model completes fewer than 40 percent of the tasks.

You cannot run a workforce on a bare model API. That shift requires the infrastructure in the prior sections.

The Framework: What Your Agent Stack Needs

Here is a checklist you can use to evaluate any agent setup:

• Does each agent run in its own worktree or sandbox?
• Is there a router or budget enforcing spend per run?
• Can a session survive a crash and resume?
• Are there automated gates between agent output and production?
• Are memory and storage capped and measured?
• Are agent permissions scoped to the current step?

If any answer is no, you do not have an agent stack. You have a model with a script around it. That will work until it does not, and it will not at the worst moment.

This checklist is the answer to "where do I start with agents." You do not start by picking a model. You start by building or adopting the layer that makes any model safe to run.

Key Takeaways

The model generates tokens. The agent runs for hours, keeps state, calls tools, and touches your systems. The infrastructure around the model is the part that ships.

Five primitives define the agent infrastructure layer: isolation, cost control, state, verification, and resource bounds. Shikigami, FlexInference, and Anthropic's migration workflow each cover parts of the set. None covers all of it yet. That is the opportunity.

Git worktrees are the universal isolation primitive. One worktree per agent. Throw the worktree away when the agent goes wrong, not the repo.

A router that races cheap tiers against expensive ones within your SLA drops inference costs by an average of 47 percent. The token budget is no longer hypothetical. The tools that enforce it are shipping.

Anthropic's own production pattern is agent produces diff, automated gates check it, humans review only the failures. That is what lets you trust a 5 to 10 percent error rate.

If your agent setup cannot answer yes to all six items on the checklist, you do not have an agent stack. You have a model with a script around it.

The model layer is mostly solved. The infrastructure layer is not. That is where the value is. That is where the next wave of companies will be built.