LoRA Can't Teach Your Agent Procedures

Full fine-tuning scores 4.11 on a procedural task with 14 decision nodes. Every LoRA configuration, from rank 16 to rank 128, scores 2.54 or lower. Not one LoRA variant matched full fine-tuning. Higher ranks did not help. Scores went down.

You fine-tuned a model on your company's workflows. It chats perfectly. It answers questions about your domain flawlessly. Then you hand it a multi-step task with conditional branching and it falls apart. You blamed the data. You blamed the prompt. You blamed the model. The paper says blame the math.

LoRA, the default parameter-efficient fine-tuning method for the entire AI industry, fundamentally cannot capture procedural knowledge. The weight changes required to learn multi-step procedures are not low-rank. LoRA is low-rank by definition. This is not a tuning problem. It is a mathematical wall.

Why Your Fine-Tuned Agent Fails at Procedures

Procedural knowledge is not knowing facts about a domain. It is the ability to follow a sequence of steps with conditional branching: if the user has a premium account, do step A; if not, do step B; then branch again based on the result. It is a tree, not a list. And trees live in high-dimensional spaces.

The paper uses a travel booking task as its concrete example. Fourteen nodes. Each node is a decision point. The agent must navigate the tree, not just classify the leaves. Full fine-tuning learns the tree. LoRA learns fragments of the tree and forgets the connections between them.

This is the difference between knowing that Paris is a city and knowing how to book a multi-leg itinerary with a layover, a visa check, and a baggage transfer. Facts are flat. Procedures are graphs. You cannot compress a graph into a flat vector without losing the edges. LoRA compresses. That is what it does. And compression destroys graph structure.

The SVD analysis is the proof. Full fine-tuning produces weight changes with an effective rank of 761 to 1,026 across three domains. That means the procedural knowledge lives in a high-dimensional space. LoRA at rank 128 can only represent a 128-dimensional subspace of that. It captures 43 to 51 percent of the signal. The other half of the procedure is gone.

Think about what that means. Your agent knows step three exists. It knows step seven exists. It does not know that step three leads to step seven only when the customer has a connecting flight and a checked bag. The connections are the procedure. LoRA cannot represent them.

The Rank Paradox

Here is the counterintuitive finding that breaks the default assumption. Higher LoRA ranks did not help. Scores actually decreased at rank 128 compared to lower ranks in some configurations.

Most engineers think: if rank 16 is bad, rank 128 must be better. More capacity, more signal. The paper shows the opposite. The issue is not capacity alone. It is that the low-rank constraint distorts the weight updates in ways that interfere with procedural learning. More rank does not fix the distortion. It amplifies it.

This is why cranking up the LoRA rank is not a solution. You cannot solve a structural limitation by making it slightly less structural. The constraint is in the method itself. Rank 128 is still low-rank. The actual weight changes needed are rank 761 to 1,026. You are not close. You are not even in the same ballpark.

The Cross-Domain Check

The paper replicated the failure on two additional domains: Zoom customer support workflows and insurance claims processing, both at 8B parameters. LoRA underperformed full fine-tuning by 0.8 to 2.2 points on these tasks.

This is not a travel-booking quirk. Any domain with multi-step procedures and conditional logic hits the same wall. If your agent needs to follow a process, not just answer questions, LoRA is fighting you. Your customer support agent, your claims processing agent, your onboarding agent, your compliance agent. They are all procedural. They are all failing for the same reason.

Why the Industry Default Is Broken for Agents

LoRA became the default because it is cheap. Full fine-tuning a 70B model requires serious compute. LoRA at rank 16 trains on a single GPU. The efficiency argument was correct for classification, summarization, and style transfer. It is wrong for procedures.

The entire industry is moving toward agentic systems. Agents are procedural by definition. They execute workflows with branching logic. If your fine-tuning method cannot capture branching logic, your fine-tuning method is incompatible with your use case.

I wrote about this in "Your Agent's Harness Is Your Real Model." The harness matters more than the weights. But if your weights cannot represent the procedure, no harness compensates. A harness cannot execute a tree the model never learned.

I also wrote about it in "The 1,400-Line While Loop." Production agents are multi-step procedural systems. The gap between tutorial agents and production agents is exactly the gap this paper exposes. Tutorials show you a single API call. Production is a 1,400-line while loop with seventeen branches. LoRA learns the API call. It cannot learn the while loop.

The Alternative: Learn at Inference, Not at Weights

FlowEvo (arXiv, July 27, 2026) takes a different path. Instead of baking procedures into weights, it compiles successful execution traces into reusable skill records. The agent learns from experience at inference time, not through weight updates.

The numbers: 82.8 percent success rate on ALFWorld, 23.6 percentage points above the strongest baseline, using less than half the tokens per episode. No weight updates. No fine-tuning. No rank constraints.

The architectural insight is simple and devastating. Procedural knowledge does not need to live in the weights if you can store and retrieve it as executable skills. This is the harness-over-weights thesis taken to its logical conclusion. The agent remembers what worked. It reuses what worked. It gets better without touching a single parameter.

HierFlow (arXiv, July 27, 2026) adds another layer: automated workflow synthesis at test time, using hierarchical search over topology and execution. The agent does not just remember procedures. It discovers better ones. It searches the space of possible workflows and finds combinations no human designed.

The pattern across all three papers is the same. The next leap in agent capability is not bigger models or better fine-tuning. It is better architectures for retaining and reusing experience without touching weights.

The Decision Framework

Three paths, grounded in the data.

Path 1: Full fine-tuning. If your task is procedural and your model is small enough (under 13B), full fine-tuning works. The effective rank is 761 to 1,026. Full fine-tuning captures it. The cost is compute. Pay it. If you are building a production agent that must follow branching workflows, this is the only weight-based approach that works.

Path 2: Inference-time skill compilation (FlowEvo pattern). If full fine-tuning is too expensive or your procedures change frequently, do not fine-tune at all. Let the agent compile its own skills from successful traces. No rank constraints. No weight updates. The procedure lives in the harness, not the model. This is where the industry is heading. The 82.8 percent success rate with half the tokens means it is cheaper to run than your current LoRA pipeline.

Path 3: LoRA for non-procedural tasks only. If your use case is classification, summarization, style transfer, or single-step Q and A, LoRA is fine. It was designed for these. The paper does not contradict that. The failure is specific to multi-step procedures with conditional branching. Use LoRA where it works. Do not use it where it mathematically cannot.

The wrong path: LoRA at rank 128 for a procedural agent, hoping more capacity fixes a structural problem. The paper proves it does not. Scores decrease at higher ranks. You are making it worse and paying more for the privilege.

What to Do Today

One: Audit your fine-tuned agents. Identify which ones execute multi-step procedures with conditional branching. Those are the ones at risk. The paper's failure applies to your use case, not just travel booking. Your agent that processes insurance claims, your agent that handles customer support escalations, your agent that manages inventory workflows. If it branches, it is broken.

Two: If you are using LoRA for a procedural agent, run the comparison. Fine-tune full on a small model (7B or 8B) and compare task success against your LoRA variant. The paper's gap of 1.57 to 2.2 points is large enough to show up in your evals. You will see it. The question is whether you will admit what you are seeing.

Three: If full fine-tuning is too expensive, evaluate inference-time approaches. FlowEvo is training-free and open. The 82.8 percent success rate with half the tokens means it is cheaper to run than your current LoRA pipeline. You are not sacrificing performance for cost. You are gaining performance and saving cost.

Four: Stop increasing LoRA rank to compensate. The paper shows scores decrease at higher ranks for procedural tasks. More rank is not better. It is worse. You are throwing compute at a mathematical impossibility.

Five: Separate your fine-tuning strategy by task type. Use LoRA for style and classification. Use full fine-tuning or inference-time skills for procedures. One method for everything is the mistake the industry is making right now. It is the mistake you are probably making.

The Uncomfortable Question

You are shipping a fine-tuned agent that cannot follow a 14-step procedure. The paper proved it six ways across three domains. How many of your production failures are LoRA limitations that you have been blaming on your data?