One Gateway, Three Kinds of Traffic: What the POC Has to Prove

The one-gateway demo is easy. Any vendor can show you an API call, an LLM prompt, and an agent handoff flowing through a single console in a sandbox. The proof-of-concept is where that claim either survives contact with your controls or quietly dies.

A newer class of gateway makes a consolidation promise worth testing. MuleSoft’s Omni Gateway, the evolution of Flex Gateway, claims to govern three traffic types from one control plane: classic north-south and east-west APIs, LLM calls through a model proxy, and agent-to-agent calls through agent policies. I made the conceptual case for what that changes in Beyond the Rebrand: What Anypoint Omni Gateway Actually Changes. This is the other half: if you are going to put that claim into production, what does the POC have to establish.

The short answer is that a demo proves the path works once, and a POC proves the control plane enforces under load. Four principles separate the two.

Write the objectives as pass/fail tests

The output of the POC is not “it worked.” It is a pass/fail result on a measurable assertion. That reframes every objective. You are not testing whether the gateway can enforce a token on an endpoint. You are proving that one hundred percent of calls without a valid token, client ID, or certificate were rejected, with no bypass path found under a penetration test.

Write each objective that way and the list writes itself: zero-trust access on every API, LLM, and agent endpoint; identity from your own provider validated per operation; sensitive data kept out of logs, prompts, and agent messages; every request, prompt, and agent task traceable end to end by a correlation ID in your SIEM; policy changes attributable to a named actor. Those map cleanly to the controls you already answer to, so there is no reason to design the POC any other way. Build it so its output is a trace you can read directly, not something you reconstruct after the fact.

Gate connectivity before you test a single policy

This is the most important step and the one most POCs skip. Before you evaluate any policy on the LLM lane, prove the proxy speaks natively to your model host.

Many enterprises land on Azure AI Foundry as the place their models are deployed. If the gateway only reaches that host through a stopgap, for instance by configuring it as though it were a different provider’s endpoint, then what you are testing is a workaround, not the product. The proxy’s native connectivity is still unproven, and that is the result to record.

The connectivity gate is a checklist you pass before policy testing begins. The proxy authenticates to the host with the host’s own identity model, not a borrowed one. The network path stays private, with public access off. The API format is confirmed and each in-scope model returns correct responses. Streamed responses pass through intact. Errors propagate so failover can be tested later. Token counts the proxy reports match the host’s own numbers, so your budgets and cost reports can be trusted. And the audit log names the real provider, model, and deployment, not the label of whatever compatibility shim got you connected, because a model inventory that lists the wrong provider is worse than no inventory. Only when that gate passes do the policy tests mean anything. Until then, the API lane and the agent lane can be proven in parallel.

In config, native connectivity is the difference between naming the host as itself and dressing it up as something else. The shape below is illustrative, but the load-bearing lines are the provider, the endpoint, and the auth: the proxy talks to Azure AI Foundry as Azure AI Foundry, over a private endpoint, with the host’s own identity rather than a static key copied out of another provider’s setup.

# LLM proxy route: native Azure AI Foundry connectivity, not a shim
llmProxy:
  name: foundry-gpt
  format: openai                    # the proxy speaks the OpenAI API format
  basePath: /llm/v1
  route:
    provider: azure-openai          # the real provider, named as itself
    endpoint: https://my-foundry.openai.azure.com
    deployment: gpt-4o-2024-11-20
    apiVersion: 2024-10-21
    auth:
      type: entra-managed-identity  # the host's own identity model
      # no static key borrowed from a different provider's config
    network:
      privateEndpoint: true
      publicAccess: false           # public path off

The proof that it worked is a single request you can follow from edge to model to agent on one correlation ID, with the token counts the proxy reports matching the host’s own numbers:

# one correlation id, three lanes, reconciled token counts
2026-10-02T14:03:11Z cid=7f3a9e api   POST /orders        200  authz=mtls
2026-10-02T14:03:11Z cid=7f3a9e llm   POST /llm/v1/chat   200  prompt_tok=512 completion_tok=288
2026-10-02T14:03:12Z cid=7f3a9e agent task=reconcile      200  policy=pii-redact:enforced
# proxy usage (512/288) == Azure AI Foundry usage (512/288)

Enforce, do not observe

I made this argument about spend in The Meter Is the Feature, and it is the same test here across all three lanes. A control that records what happened after it happened is a transcript. A control that stops the run at the limit is a leash. The POC has to prove the leash.

Ask the precise question on each lane. When a token budget is exceeded, does the next call get blocked, or does a number turn red on a dashboard while the call completes. When a prompt or response carries sensitive data, is it redacted before it leaves, or logged after it left. When one agent passes an injected instruction to another, is it blocked before it reaches the target, or noted once the target has already acted. The right answer in each case is the halt, and the way you prove it is to drive the system past the threshold under load and watch the backend, the model, or the target agent stay protected. A policy that only narrates is a log line, not a control. The test is whether the run stops.

Keep the three lanes separable, and prove you own the plane

Consolidation is the selling point, but each traffic type has to stand on its own evidence. The API lane, the LLM lane, and the agent lane should each produce an independent trail, so that a weakness in one does not borrow credibility from another. Test them together for the operational path, and apart so each lane’s trace stands on its own.

There is a licensing question underneath this, and it is an architecture-independence question in disguise. The vendor’s heavier control-plane suite, its agent-fabric offering, is a distinct product from the gateway that enforces the policies. Can you adopt the enforcement layer you need without buying the whole suite? Confirm it in writing, because the answer determines whether you are picking a policy engine or committing to a platform. And the deeper test, the one I framed in Governance or Containment, is whether the governance attaches to a runtime you own. Run it self-managed, prove no customer payloads leave your network for the vendor’s control plane, and you are holding the plane. Skip that and you are renting one, sight unseen.

Both sides of the console

The operational case for consolidation is real and strong. One control plane for three traffic types means one console, one policy surface, and one set of identity and logging integrations instead of three. Fewer audit trails to reconcile, fewer seams for something to slip through, and a single place to reason about limits and access. For a team already drowning in point solutions, that reduction is not a convenience, it is a control improvement in its own right, the argument I made about a single inspection point in We Inspect the Traffic, So We’re Covered.

The control case is just as real. A vendor control plane is one you have to prove you run, not assume. Where the plane executes, and whether your payloads ever leave your network to reach it, is a deployment property you can test, not a line in a datasheet. Enforcement has to be deterministic: the same policy, the same halt, every time, not a probabilistic best effort. And consolidation concentrates blast radius, since one plane fronting three lanes becomes a single failure correlated across all of them. These reconcile the way they always do on this blog: the consolidated control plane earns a yes only when it enforces rather than observes, runs where you can see it, and halts the same way under load every time. That determinism is what lets you safely say yes to putting LLM and agent traffic into production, but only once the POC shows it is a plane you run.

The honest limit

A POC that passes on a workaround has proven the concept, not production readiness, and the writeup should say exactly that. Some of the newest capabilities, native model-host connectivity and the maturity of agent-to-agent policy, may still be preview today, and preview is a fine thing to test and a dangerous thing to deploy. Consolidation concentrates blast radius, so the same gateway that simplifies your tracing also raises the stakes on getting it right.

None of that argues against the proof. It argues for doing it seriously. The demo is the easy part. Read the POC as the place where “one gateway governs three kinds of traffic” stops being a slide and becomes a control plane you have driven past its limits and watched hold. Objectives written as pass/fail tests, connectivity gated before policy, enforcement that halts rather than logs, and a control plane you can prove you run. Prove those four, and the consolidation is worth having. Skip any one, and you have a very impressive sandbox.

---

If this resonated, read Beyond the Rebrand: What Anypoint Omni Gateway Actually Changes, on what the consolidation shifts, and We Inspect the Traffic, So We’re Covered, on the quiet gap in a single inspection point.

Find me on X @orestesgarcia or LinkedIn /in/setsero.