The Model Decides, the Code Controls

I am writing this from Dreamforce, a few hours after walking off the stage. This afternoon I sat up there and answered three questions about what it actually takes to run agentic AI in a regulated industry, in front of a room that had every reason to be skeptical. I had come in with a set of boundaries drawn on paper, written down before I flew out in Draw the Boundary Before You Buy Deeper, because I did not want a vendor drawing them for me on the floor. The session was the other side of those lines: not the questions I wanted to ask, but the three I had to answer myself, live. Bobby, who ran it, had announced the session here a few days earlier. This is the long version of what I said, while it is still ringing in my ears.

The three I was handed are the ones every enterprise is circling right now. Why do so few agents take real action. Why do so few of the ones that work ever reach production. And why does the bill arrive before the return does. McKinsey’s State of AI in 2025 put a number on the last one that stuck with the room: the large majority of enterprises are not seeing strong ROI from AI yet, despite record spending. Adoption is nearly universal. Impact is not.

I want to make one claim up front, because everything else hangs off it. The governance work is what gets agents into production, not what holds them back. That is the opposite of what most people expect a bank to say, and it is the honest version of what I have watched happen. Underneath it sits a design principle I kept coming back to on stage: the model handles judgment, the code handles control flow, and projects die where those two get mixed up.

From one agent to a governed chain

The first question was why so few agents take meaningful action, and I answered it by refusing the premise buried inside it: that an agent is one clever assistant, one prompt, one impressive demo. Picture instead the work a regulated lender actually needs done. To draft a credit memo you do not want one agent, you want a chain. Eight specialists read the inputs, one each for the balance sheet, the cash flow, the collateral, the income statement, the personal returns, the industry, the asset-based lending view, and the deal itself. A second tier takes those outputs and produces the risk and mitigants, the covenant testing, and the executive summary. A final step assembles the memo. A relationship manager and a risk director still hold credit authority at the end of it. The chain drafts. People decide.

That shape is the answer. Meaningful action in a bank is not a single model call. It is a sequence with authority preserved at the end, and building it is a systems problem before it is a prompt problem.

Here is the design question that decides whether the whole thing works: where does the model get to decide, and where does the code decide. Get that line wrong in either direction and you lose. Push everything into the model and you get a system you cannot predict or audit. Push everything into code and you have rebuilt a rigid script that did not need a model at all. In the credit chain the split is clean. The model reads the financial statements and writes the narrative, because that is judgment. The code decides what runs when, what feeds what, and where a human signs, because that is control flow. The sequence is a pipeline, not a prompt.

So why a dozen small agents and not one big one. The same reason you break up a monolith. Each stage has defined inputs and outputs, so you can test one, change one, or replace one without touching the rest. When a memo comes back wrong you can point to the stage that produced the bad section instead of shrugging at a black box. One large agent gives you something you cannot debug, and something you cannot debug is something you cannot ship in a regulated shop.

None of this means you skip the prototype. Prototyping fast and loose is worth doing, and the curiosity that produces it is exactly what you want from your engineers and from the technology. What you cannot do is confuse the prototype with the product. You cannot vibe code an agent into production. Every practice we already trust still applies once it is real: interfaces, tests, version control, error handling, rollback. The discipline is knowing where to let the curiosity run and where to pin the behavior down in code.

And once you have a dozen agents with dependencies between them, be honest about what you have built. It is a distributed system, and it fails like one. Retries, partial failure, ordering, state that has to survive a restart. Systems thinking is the whole game here, and the good news buried in that sentence is that none of it is new. The industry has been running distributed systems for years. The orchestration and tracing layer that routes the chain deterministically and lets you follow a request end to end is the next thing to build, not the first thing to panic about.

If you take one thing from this section, take the boundary. The model owns judgment. The code owns control flow. Projects die where those get mixed up.

Why governance ships agents

The second question is where I expected the most pushback and got the least. Building the agent is the easy part. Shipping it is the work.

So when the thing you are shipping is an agent, what actually changes in your software lifecycle? Four things. Prompts, models, context, and tools. That is the whole delta. Everything else about the lifecycle is what it always was. The sharpest way I can put it: an agent is an application with four new moving parts. Once you see it that way, the production question stops being scary and starts being familiar.

Which leads to the mistake I watch teams make. They decide an agent is so novel that it needs its own separate governance track, parallel to the one they already run for software. Do not do that. Augment the standard you have. Two sets of rules means people follow neither, and you spend a year reconciling the tracks instead of shipping anything down either. You already have a solid delivery standard. Extend it to carry the four new parts, and you are done.

What that looks like is concrete, which is the point. Prompts go in source control and get reviewed like any other artifact. Models get version pinned and change controlled, so nobody swaps the brain of a production system by editing a config nobody diffed. Context gets provenance, so you can say where a fact the agent used came from. Tools get scoped permissions, so an agent can reach exactly what it needs and nothing else. That is not exotic. It is your existing discipline pointed at four new surfaces.

Now the part that sounds like friction. The pattern I have come to trust is a single governed intake for every enterprise agent, with the reviewing functions sitting in one room instead of passing the thing down a chain, and a sign-off before it reaches production. When I describe that, people hear a bottleneck. It is the opposite. It is the reason anything reaches production at all. Without one agreed path, every agent stalls, because nobody will put their name on the risk alone. A shared intake gives them a way to say yes. Four approvals in a room is faster than four approvals in a relay.

The trick is that the controls live in the pipeline, not at the end. Data boundaries and escalation limits get set when you stand up the environment. Least privilege and secret management happen at build. Security testing runs alongside functional testing, not after it. By the time the thing reaches a review board, there is nothing left to discover, which is precisely why it clears. A gate is fast when everything upstream already satisfied it.

Two more pieces belong here. Scope the whole regime deliberately: this is for enterprise agents with real autonomy, not for the personal-productivity assistant someone runs to summarize their inbox. Try to push everything through one gate and you get a gate nobody uses. And you need a live registry underneath it all: every agent, who owns it, what it can call, what data it can reach. Live, queryable, current, not a spreadsheet someone last touched in March.

The conversation gets easier when the platform that already governs your APIs grows into governing agents on the same control point: an LLM proxy, agents, and MCP servers as one more class of governed traffic. That is the capability class Salesforce is aiming Agent Fabric at, and it is the same instinct I argued from the other direction before the conference. Whether any specific product delivers it is a floor question, and I am keeping my boundaries from the last post. What matters is the shape: one more class of traffic on a control point you already run beats standing up a second governance plane next to the first.

The landing point is short. An agent is an application with four new moving parts. Extend the standard you have, and governance stops being the thing that blocks the launch and becomes the thing that permits it.

Runtime enforcement, not prompt-based limits

The third question is the one a CFO cares about, and it is where the abstract governance argument turns into money.

A deterministic service has a known unit cost. You can forecast it, price it, and build a return on it. An agent that picks its own path costs something different on every run. That is the real ROI problem, and it is not a rounding error. You cannot build a business case on a number that moves per transaction, and the reason so much AI spend cannot show a return is that nobody can attribute it. Right now tokens tend to show up as one bill, not as a metric broken out per agent and per business line. You cannot forecast spend you cannot attribute.

The fix is to make the economics knowable by bounding them. Token ceilings, tool-call limits, capped retries, attribution per agent. Those are not only safety features. They are the inputs to a business case, which makes bounding an economics problem as much as a control problem.

And the bounds have to live in the infrastructure, not in the instructions. This is the line I care about most, and it is the one that got quoted back to me in the room: a limit that lives in a system prompt is a suggestion. If oversight, rate limits, data access, and the list of things an agent is allowed to call are enforced in the architecture, they hold. If they are written politely into a prompt, they hold until the model decides otherwise. Human oversight enforced in the system is a control. Human oversight requested in a prompt is a hope.

Here is the part most people miss, and it is what makes the argument legible to a CFO and not only to a risk officer. Safety and spend resolve at the same control point. An agent you can talk out of its guardrails is an agent you can talk into burning tokens. One gateway that enforces both fixes both. You do not buy a safety tool and a cost tool. You put one enforcement point in the path and it does double duty.

Identity is part of the same answer. Agents need real credentials from the same directories you use for staff, issued, scoped, and revocable, so that “who did this” has an answer when an agent did it. Machine identity is the next serious piece of work, and I made the longer case for it in The Agent Identity Federation Gap. Measurement closes the loop: usage and incident data feeding continuous monitoring instead of a review every six months, with spend and exposure visible by business line so the number stays honest between audits.

The landing point is the one to bring to a finance conversation. Be deterministic where you can be, bounded where you cannot, and you get something an agent alone never gives you: a number a CFO can underwrite. That is the same metering problem I flagged as the industry’s open wound in the pre-conference post, viewed from the side of the person who has to sign the invoice, and I have argued the broader version of it in The Compliance Tax.

What I’m still figuring out

Every line I just drew is a judgment call, not a settled rule. The orchestration and tracing layer that makes a multi-agent chain observable end to end is a thing to build, not a thing I can point at as finished. Metering and attribution are still the industry’s weak spot, the same open problem I named before I left, and until it closes every one of these architectures ships with a financial unknown baked in. The model-versus-code boundary is clean in a slide and messier in a real system, where a task that looked like control flow turns out to need judgment halfway through.

What I am confident about is the direction. The teams getting agents into production are not the ones with the cleverest single agent. They are the ones who treated the agent as an application with a few new parts, extended the discipline they already had, and put their limits in the infrastructure instead of the prompt. The governance is not the tax you pay to ship. It is the thing that lets you ship at all.

---

If this resonated, read Draw the Boundary Before You Buy Deeper, the pre-conference companion where I set out the questions I refused to leave the floor without answering, and The Agent Identity Federation Gap, on why machine identity is the next piece of the governance stack.

Find me on X @orestesgarcia or LinkedIn /in/setsero.