The Case for Open-Weight Defaults in Agent Frameworks

When you install an agent SDK and import it for the first time, the SDK has to make a choice for you. Which model does it call by default? The answer to that question is a statement about whose problem the SDK is solving.

Most agent SDKs default to their parent company’s hosted proprietary model. Claude SDK defaults to Claude. OpenAI SDK defaults to GPT. Google ADK defaults to Gemini. This is reasonable from each vendor’s perspective. It is unreasonable from yours.

What “default” actually does

The default model in an SDK shapes three things:

1. Onboarding. A new user runs the quickstart. Did it work? If the default needed a paid API key, the answer is “not yet.”
2. Lock-in. People build for the default. Switching providers later means changing prompts, retesting tools, and re-evaluating quality. Most teams never do it.
3. Deployment options. If the default can’t run on-prem, the SDK’s on-prem story is theoretical. Production teams notice this on day one.

Defaulting to a proprietary hosted model optimizes for the vendor on all three. It minimizes the friction of trying the SDK, but only if you already have the vendor’s API key. It locks people in to that model. And it makes on-prem an afterthought.

What an open-weight default does

Defaulting to an open-weight model — running either locally or on a free cloud tier — flips the optimization toward the user:

1. Onboarding works without an account. ONTO defaults to Ollama Cloud’s free open-weight tier. pip install onto-core, set one env var, you’re running. No credit card, no vendor relationship.
2. No lock-in to start. When you build on an open-weight default, your prompts and tools are tested against a model whose weights are public. Swapping to a different open-weight model is a config change.
3. On-prem is a one-line switch. Point at a local Ollama. Same model class, same prompts. Done.

The proprietary models are still there — Claude and GPT are one env var away. They’re upgrades, not requirements.

”Are open-weight models even good enough”

Three years ago this question had a clear answer: no. Today the answer is “for what.”

For frontier reasoning — multi-step math, long-context comprehension on hard documents, very subtle instruction following — proprietary frontier models are still the strongest. There’s a real gap.

For the bulk of agent work — extracting typed facts from user messages, generating a plan of tool calls, summarizing a session, structured output to a JSON schema, calling a single tool with the right arguments — modern open-weight models like gpt-oss:20b, Llama 3.1, and Qwen 2.5 are good enough. We’ve measured. They handle ONTO’s extraction loop, plan mode, and tool-calling guardrails without trouble.

The right pattern is to default to open-weight, upgrade where you measure a lift. Build, ship, measure. If a specific workload measurably benefits from Claude or GPT, route that workload — and only that workload — to the upgraded provider. The SDK should let you make this routing decision at the call site, not the install site.

The on-prem dimension

For regulated industries this isn’t a preference, it’s a requirement. Healthcare data can’t leave the VPC. Financial data has retention rules that hosted vendors won’t sign. Legal work has confidentiality requirements.

If your SDK can’t run with everything on-prem, you’re not building for these users. You’re building a hosted-only product and asking these users to make an exception. Most of them won’t.

ONTO’s on-prem path is straightforward:

• Run Ollama or vLLM locally with an open-weight model
• Store assertions in Postgres + pgvector inside your VPC
• Embed with fastembed locally
• Deploy with the included Helm chart on your own Kubernetes

No third-party API hits. The cost is the model serving infrastructure you’d run anyway.

The cost dimension

Open-weight models on a free or low-cost tier change the unit economics of agents that do a lot of small tasks. Memory summarization, conflict detection, extraction — these can run hundreds of times per session in a busy agent.

If each of those goes to a $0.005 GPT-4 call, your bill is a problem. If they go to a free Ollama Cloud open-weight call (or a self-hosted one), your bill is the foreground turns only. ONTO’s background memory worker pattern leans on this: route background summarization to a cheaper model by default.

The “ecosystem” objection

The pushback against open-weight defaults is usually “but the ecosystem and quality and tools are better on the proprietary side.” This is sometimes true. It’s also irrelevant to the default choice.

The default is what runs when you don’t pick. Picking should be an active decision, not the by-product of an install. An SDK that asks you “by the way, please configure a paid API key before this works” is asking for a commitment before you’ve evaluated. An SDK that runs out of the box on free open-weight, then lets you upgrade to Claude or GPT with one env var, is letting you evaluate first and commit second.

What this looks like in practice

# Default — Ollama Cloud free open-weight. No key needed.
onto = Onto(model=Models.GPT_OSS_20B)

# Same code, Claude. Set ANTHROPIC_API_KEY.
onto = Onto(model=Models.CLAUDE_SONNET_46)

# Same code, OpenAI. Set OPENAI_API_KEY.
onto = Onto(model=Models.GPT_5)

# Same code, local Ollama. Set OLLAMA_BASE_URL.
onto = Onto(model=Models.GPT_OSS_20B)  # routes to local

The library doesn’t care which one you pick. The library cares that the default lets you start without asking permission.

A final argument

There’s a structural reason this matters beyond ergonomics. When an SDK’s default is its parent company’s proprietary model, every tutorial, every example, every blog post is implicitly an advertisement for that model. The ecosystem the SDK builds is an ecosystem of users who tested only against the proprietary model.

That ecosystem then has trouble running anywhere else. The SDK is technically provider-agnostic; in practice it’s lock-in by inertia.

An open-weight default breaks that. Tutorials work on the free tier. Examples ship with hashes that match what the open-weight model produces. New users build prompts that aren’t covertly tuned to a particular proprietary quirk. The ecosystem becomes portable by construction.

If you’re building agent infrastructure that you want to outlive the current generation of frontier models, defaulting to open-weight is the only choice that doesn’t quietly bet on one vendor.

---

ONTO defaults to Ollama Cloud’s free open-weight tier. Swap to Claude or GPT with one env var — details on the docs page and the features page.

Frequently asked questions

Are open-weight models good enough for production agents?

For many agent workloads, yes. gpt-oss:20b, Llama 3, and Qwen 2.5 all handle tool calling and structured output well enough for typed memory extraction, plan generation, and most chat. For frontier reasoning tasks, you may still want Claude or GPT — and ONTO lets you swap with one env var.

Doesn't open-weight just mean local-only?

No. Ollama Cloud runs open-weight models hosted, with a generous free tier. The point of 'open-weight' is the model weights themselves are downloadable and runnable on hardware you control — independent of whether you run them locally or hosted.

What about quality? Aren't proprietary models still ahead?

For the hardest reasoning tasks, yes. For the bulk of agent work — extraction, tool calling, summarization, structured output — modern open-weight models are within striking distance. Default to open-weight; upgrade where you measure the lift.