Model-Agnostic AI Infrastructure

What model-agnostic means

Model-agnostic infrastructure lets you change the model behind an agent without rewriting the agent.

In practice that means the model is configuration, not architecture. You can move a workflow from one provider to another, run different models for different steps, or fall back to a second provider when the first one is down, and the agent logic does not change.

It is easy to claim and harder to build. A platform is not model-agnostic because it supports several providers. It is model-agnostic when swapping between them is a configuration change rather than a migration project.

Why it matters

The frontier moves

Model leadership has changed hands repeatedly, and the gaps between providers open and close within months. Any architecture that assumes today’s ranking is permanent is betting against the one thing the last three years have been consistent about.

Price moves faster

Cost per token has fallen sharply and unevenly. A workflow that was uneconomic at one provider’s pricing becomes viable at another’s. If switching costs you a rewrite, you do not capture that, and you keep paying the old price.

Not every step needs the same model

Classifying an email and drafting a legal summary are different problems. Sending both to your most expensive model is a design choice most teams make by default rather than on purpose. Routing by step is where the real cost reduction lives, and it requires the model to be swappable.

Availability

Providers have outages and rate limits. If your production workflow has exactly one way to reach a model, their incident is your incident.

Sovereignty and regulation

Some buyers cannot send data to a US provider at all. European public sector bodies, defence, some banks. For them the requirement is not a preference for an open model, it is the ability to run inference somewhere specific, sometimes with no internet connection at all. That is only possible if the model layer is replaceable. More on sovereign deployments.

Where lock-in actually comes from

Teams underestimate this because they think of lock-in as the API call, which is the easy part. The real attachments are elsewhere.

• Prompts tuned to one model. Prompts that were iterated against a single model degrade against another. This is real work to redo, and it is the main reason “we could switch” turns out to be false.
• Provider-specific features. Structured output formats, caching behaviour, tool calling conventions, context windows. Build on them and you have built a dependency.
• The platform above the model. If your agents live inside a vendor’s proprietary abstraction, the model is portable and your agents are not. This is the lock-in that matters and the one least often discussed.
• Evaluation debt. Without a test suite you trust, switching models is an act of faith. Most teams have no such suite, so they never switch.

How to build for it

1. 1. Put an abstraction between your agent and the provider. Never call a provider SDK directly from agent logic.
2. 2. Keep prompts as versioned artefacts. Separate from code, versioned, with a note on which model each was tuned against.
3. 3. Build an evaluation set early. Thirty real cases with known good answers is enough to make a switch a measurement instead of a gamble.
4. 4. Route by step. Assume from the start that different steps use different models, even if today they all use one.
5. 5. Own your keys. If your platform resells model access, you cannot move, and you cannot see what you are paying per call.
6. 6. Check the platform layer, not just the model layer. Ask what happens to your agents if you leave. If the answer involves a rewrite, the model portability was beside the point.

The honest trade-off

Model-agnostic is not free. You give up some provider-specific features, you carry an abstraction layer, and you maintain evaluations you would otherwise skip.

The trade is worth making when your agents touch anything that matters commercially, when you have customers with sovereignty requirements, or when model spend is large enough that a routing change is material. For a single internal experiment, it is over-engineering. Be honest about which one you are.

Common questions

Does model-agnostic mean worse performance?

It means you do not use provider-specific features that have no equivalent elsewhere. For most agent workloads the cost is small and the routing savings are larger.

Is a gateway enough?

A gateway makes the API call portable. It does nothing about prompts tuned to one model, or about a platform that owns your agent definitions. Useful, not sufficient.

Can I run open-weights models?

With model-agnostic infrastructure, yes, including models you host yourself. That is the requirement behind most air-gapped and sovereign deployments.

Stackbone

Stackbone is model-agnostic and cloud-agnostic by design. You bring your own provider keys, we never proxy model traffic, and agents execute in your own cloud account. Changing model is configuration.