Someone on the team says the words “let’s add an AI agent,” and watch what happens next. In a shop that has run on Java for fifteen years, where the ledger, the entitlements, the audit trail, and the transaction boundaries all live in Spring Boot, the reflex is still to open a fresh repo and reach for Python. That reflex costs more than anyone admits, and it is worth interrogating before you commit a roadmap to it.

The case for staying is not nostalgia. It is that the hard part of an AI application was never the model, and the place where the hard part lives is the JVM.

The 95% and the 5%

Start with the number everyone quotes and almost nobody reads. MIT’s Project NANDA published “The GenAI Divide: State of AI in Business 2025”, and the headline that traveled was that 95% of enterprise generative-AI pilots fail. The report’s own framing is the inverse, and more useful: just 5% of integrated pilots are extracting real value, while the rest show no measurable impact on the P&L. Fortune covered it under the failure-rate banner, and the finance world winced.

Josh Long, the Spring developer advocate, flips the framing in a way that reframes the whole strategy question. The interesting group is the 5%. Those winners are not demos. They are the pilots wired into a genuinely valuable workflow, and valuable workflows are enterprise business logic. Guess where enterprise business logic runs. It runs on the JVM. Long makes the point in his conversation on The Marco Show: if you want to join the 5%, you build where the value already lives, and bolting a separate Python tier onto the side of your business is a good way to join the 95%.

An AI app is mostly integration

Here is the claim that should change how you budget the work. In his first Spring AI post, Long writes that “90% of what people talk about when they talk about AI engineering is just integration with models, most of which have HTTP APIs.” And most of what those models take is “just human-language Strings.” His question lands hard: what better place for that integration to live than hanging off the side of your Spring-based workloads?

Strip the mystique and an agentic application is a familiar shape. It is an HTTP client calling a model endpoint, with retries and timeouts and circuit breakers. It is authentication and secret management. It is rate limiting and backpressure. It is reading from your database inside a transaction, enforcing entitlements before a tool runs, emitting metrics and traces, and writing an audit record a regulator can later read. Every one of those is a solved problem on the JVM, hardened over two decades of people running money through it. This is the same argument I made about financial-crime systems: the model was never the problem, the data and the integration around it were. Generative AI does not repeal that. It underlines it.

The bottleneck was never how much code you can produce. It was always integrating that code into a running system that cannot be allowed to break. More generated output, in a fresh language, on a fresh runtime, is more to integrate, not less.

The rewrite is the trap

The instinct to start clean in Python feels like momentum. It is usually the opposite.

Joel Spolsky named this a quarter-century ago in “Things You Should Never Do, Part I,” where he calls a from-scratch rewrite “the single worst strategic mistake that any software company can make.” The old code you want to throw away is not ugly by accident. It encodes a thousand bug fixes, edge cases, and hard-won corrections that a rewrite silently discards and then rediscovers in production. Choosing a new language to get closer to some model SDK means paying that tax on purpose.

There is a physics to it too. Data has gravity, and compute wants to sit next to the data. When your AI logic lives in the same process as your domain model, a tool call is a method call against code you already tested. Move it to a separate Python service and that same call becomes a network hop, a serialization boundary, a second deployment, a second on-call rotation, a second attack surface, and a second place your entitlements can drift out of sync with the first. You did not remove complexity. You added a tier and named it progress. If you are going to own an AI capability that matters, own the harness it runs in rather than renting a bolt-on you have to babysit.

One more point from Long, by way of DHH and a very old IBM slide. A bad spec produces bad output whether a human or a model writes the code, so understanding the domain is still the job. And you cannot fire the model when it ships something wrong. A human still has to sign their name to the result. That accountability lives with the team that owns the system, which is the team that already knows the Java.

What Python actually has

The honest part. Python did not win the AI conversation by accident, and pretending otherwise makes the rest of this weaker.

The research gravity is real. Hugging Face hosts millions of models and datasets, and the overwhelming majority of them ship as PyTorch reference code first. New architectures land in Python before anything else, vendor SDKs from OpenAI and Anthropic are Python and TypeScript on day one and everything else later, and the tutorial economy assumes you are in a notebook. LangChain carries well over a hundred thousand GitHub stars for a reason. On the TypeScript side, the Vercel AI SDK makes streaming UIs and full-stack single-language cohesion genuinely pleasant. If you are training models or living at the research frontier, Python is not a bad default. It is the correct one.

The popularity gap is real too. Stack Overflow’s 2025 developer survey puts Python near 58% of respondents and Java near 29%. So this is not a post claiming Java is more popular than Python. It is a post about what you should do when your investment, your talent, and your revenue-bearing logic are already in Java, and someone hands you a model with an HTTP endpoint.

Spring AI: the integration layer you already know

The tooling caught up while people were arguing. Spring AI reached 1.0 GA in May 2025, followed by 1.1 that November and a 2.0 GA in mid-2026. It gives you a portable ChatClient across roughly twenty model providers, an advisors chain for retrieval and memory, RAG helpers, vector-store abstractions across around twenty databases, structured output, and tool calling that turns an annotated Java method into something the model can invoke. Observability is Micrometer, so token usage and latency show up on the same dashboards as the rest of your service.

The tool-calling piece is the whole thesis in miniature. Your business logic is already written and tested. You expose it:

@Component
class AccountTools {

    private final LedgerService ledger;

    AccountTools(LedgerService ledger) {
        this.ledger = ledger;
    }

    @Tool(description = "Return the current balance for a customer account")
    BigDecimal currentBalance(String accountId) {
        return ledger.balanceOf(accountId); // existing, tested, audited code
    }
}

String answer = chatClient.prompt()
        .user("Is account 4021 overdrawn, and by how much?")
        .tools(new AccountTools(ledger))
        .call()
        .content();

There is no second service. The model reaches into the same LedgerService your web tier already calls, inside the same transaction and the same security context. That is the low-friction path Python cannot offer a Java shop, because in Python that LedgerService does not exist yet.

Embabel: planning that isn’t another model call

Tool calling is table stakes. Real agentic work needs planning, and this is where the JVM story gets genuinely interesting, because the person building the most compelling answer is the person who built Spring in the first place.

Rod Johnson’s Embabel is an agent framework for the JVM, written in Kotlin with first-class Java support and built on top of Spring AI. Its planner uses Goal-Oriented Action Planning, an algorithm borrowed from game AI. You declare actions with their preconditions and effects, you declare a goal, and a deterministic planner figures out which actions to chain to reach it. The critical detail: that planner is ordinary code, not another LLM call. So the plan is explainable and repeatable, which matters enormously the moment an auditor asks why the system did what it did.

@Agent(description = "Resolve a disputed card transaction")
class DisputeAgent {

    @Action
    Evidence gatherEvidence(Dispute dispute, LedgerService ledger) {
        return ledger.evidenceFor(dispute.transactionId());
    }

    @AchievesGoal(description = "A resolution the bank can sign its name to")
    @Action
    Resolution recommend(Dispute dispute, Evidence evidence, Ai ai) {
        return ai.withDefaultLlm().createObject(
                "Recommend approve or deny for this dispute given the evidence: " + evidence,
                Resolution.class);
    }
}

The actions receive real domain objects. Dispute, Evidence, and Resolution are Java records, so the prompts are typesafe, the tools are discoverable, and the whole thing survives a refactor instead of rotting into stringly-typed glue. Johnson’s rationale is blunt, and he has earned the right to be. “Much of the critical business logic in the world is running on the JVM, and for good reason. Gen AI enabling it is of critical importance.” He is not aiming small either: “We want not just to build the best agent platform on the JVM, but to build the best agent platform, period.”

The performance story is quietly solved

The old objection is that Java is too slow or too heavy for the AI layer. That objection is a decade out of date.

The Model Context Protocol, the standard way agents talk to tools and data, has an official Java SDK maintained in collaboration with the Spring AI team, so a Spring bean becomes an MCP tool with a single provider registration and no bespoke plumbing. Native inference is handled too. llama3.java runs Llama 3 in a single file of dependency-free Java, compiles to a GraalVM native image with instant startup, and uses the incubating Vector API for the matrix math. Underneath it, Project Panama’s Foreign Function and Memory API went GA in Java 22 and lets the JVM bind to native math libraries without the old JNI misery. Project Valhalla’s value types, when they land, close the last of the numeric-density gap.

Long’s summary is the honest one: technically, the fight is already won. Building generative-AI applications is mostly REST calls and integration, and that has been the JVM’s bread and butter since before most of these frameworks existed.

The gap that’s left is marketing

So if the platform is ready, why does everyone still assume Python is the only door? Inertia, mostly, and the fact that the war of ideas is fought with more than code. Spring itself did not win purely on merit. It won because its advocates prosecuted the argument in public for years, relentlessly, until the industry turned. The JVM AI story has the engineering and is still missing that decade of drumbeat.

Long names the real fear, and it is the right one to hold. The risk is not that Java arrives unready. It is that Java arrives ready but ten years too late, the way Java could have been the language of container orchestration and Go got to Kubernetes first. Being technically correct and culturally invisible is its own kind of failure.

And you should be honest about what the JVM still does not have. There is no model hub to rival Hugging Face. New research ships as Python you have to wait for someone to port. The vendor SDKs treat Java as a second release. None of that is nothing. But notice what none of it touches: the integration, the data access, the security, the transactions, the observability, the accountability. The part that decides whether a pilot joins the 5% is exactly the part Java has always been best at.

The bottom line

If you are starting from a blank page and your goal is research, use Python. That is not the situation most enterprises are in. Most are sitting on millions of dollars of Spring Boot, a team fluent in it, and business logic that already runs the company. For them the burden of proof runs the other way. The question is not “can we justify staying in Java,” it is “can anyone justify the rewrite,” and the honest answer is usually no.

The 5% that win build their AI where the value already lives. For a very large number of organizations, that place has a coffee cup on the logo. The technology caught up. The only thing left to change is the assumption.

---

The companion read is The Model Was Never the Problem, which makes the same case one layer down: the classifier is a commodity, and the data and integration around it decide whether the system works.

I write about AI-assisted development, enterprise architecture, and the infrastructure layer between them. Find me on X @orestesgarcia or LinkedIn /in/setsero.