# ONTO — full text Source: https://onto.neullabs.com License: MIT or Apache-2.0 This file is a concatenated dump of the ONTO marketing site for LLM ingestion. Sections are ordered: overview, architecture, quickstart, runtime primitives, comparison, industries, articles. --- ## Overview ONTO is the open-source agent SDK where humans drive the state. The agent reads state and proposes actions; the human writes state and approves. Three runtime primitives enforce this split: 1. UPO writes are explicit — every memory write is an add_assertion call with consent scopes and audit metadata. 2. Policy Guard gates every tool call against per-call consent scopes. Tools refuse with a structured error if scopes do not cover their requirements. 3. Plan Mode halts execution after task.create calls. Side-effect tools never fire until a subsequent run with execute scopes. ONTO ships in three native SDKs (Python via PyO3, TypeScript via napi-rs, Rust native) over one Rust core. Plus an HTTP server with auto-generated OpenAPI for any language. Defaults to Ollama Cloud (free open-weight gpt-oss:20b-cloud). Swap to Anthropic or OpenAI with one env var. On-prem path: Sled or Postgres + pgvector, fastembed local embeddings, Helm + Kubernetes recipes. Pre-1.0, currently 0.1.0. 238 tests across the stack. Phases 0–20 shipped. --- ## Architecture URL: https://onto.neullabs.com/architecture A single run passes five auditable checkpoints: 1. Read state — the runtime loads relevant UPO + SMO facts and gives the model a small, typed context (not a replayed chat log). 2. Propose — the model proposes tasks and tool calls. In Plan Mode the run halts here; nothing has executed or been written. 3. Policy Guard — each proposed tool call is matched against the granted consent scopes; unauthorized calls are refused before firing. 4. Human approves — the reviewer reads the plan, edits or rejects tasks, and grants execute scopes. The model cannot self-approve. 5. Execute + write UPO — a second run with execute scopes performs the approved work; durable facts commit to UPO with provenance, and the audit log records the whole path. Four layers over one Rust core: (a) the SDK bindings (PyO3, napi-rs, native Rust); (b) the typed memory store (Sled or Postgres + pgvector, fastembed local embeddings); (c) Policy Guard in front of every tool call; (d) the provider adapter (Ollama Cloud default, one env var to swap to Anthropic/OpenAI or a local Ollama server). The same fact written from Python reads back identically in TypeScript or Rust. --- ## Quickstart URL: https://onto.neullabs.com/quickstart 1. Install: pip install onto-core (Python), npm install @onto/core (TypeScript), or cargo add onto-runtime onto-storage onto-llm (Rust). 2. Point at a model: export OLLAMA_API_KEY for the free open-weight default, or ANTHROPIC_API_KEY / OPENAI_API_KEY to swap providers with no code change. 3. Run with consent scopes: onto.run_sync(prompt, user_id=..., consent_scopes=["memory:write"], extract=True). The agent proposes and extracts typed facts; nothing outside the granted scope can fire. 4. Plan then approve: run once with mode="plan_only" to get a reviewable task list, then run again with execute scopes after a human approves. --- ## Runtime primitives ### Plan Mode A first-class runtime mode. Set mode="plan_only" on a run and ONTO halts after task.create returns. No tool side effects, no memory writes, no external API hits. You get a reviewable task list. Approve task-by-task; only then does a second run, with execute consent scopes, actually do work. ### Typed memory (UPO + SMO) User Personalization Ontology (UPO) stores durable typed facts with provenance, confidence, decay curves, regulation tags ([pii], [phi], [mnpi]), and consent scopes. Session Memory Ontology (SMO) is transient context that auto-summarizes and promotes to UPO. Survives across sessions without replaying chat logs — facts, not embeddings. ### Four-level ontology extraction Set extract=True. The LLM proposes typed assertions and labels each one with a level: session (transient context), profile (durable identity), domain (app-scoped, multi-tenant), or tool (tool-local). The runtime routes, audits, and writes through Policy Guard. One prose paragraph becomes a typed graph with audit trail. ### Policy Guard Every tool declares required_consent (e.g., ["memory:write"], ["patient:order"]). Every run carries consent_scopes. If the union does not cover the requirement, the runtime refuses with a structured error you can audit. ### Async writes and background memory worker extract_async=True spawns memory writes to a background task; the next LLM turn starts ~one RTT sooner. background_memory=True routes session summarization and conflict resolution to a cheaper model on a background worker. Foreground latency drops without sacrificing durability. Foreground vs background cost is split. ### Cost meter onto.cost(user_id) returns per-user, per-session cost breakdown. Multi-tenant rollup is built in. Foreground vs background cost is exposed separately (matters for SaaS billing). ### Eval framework + guardrails onto eval reads TOML/JSON test cases, runs the agent, exits non-zero on failure. Built-in guardrails: max_input_length, max_output_length, regex_block, regex_redact, required_keywords. Gated at input (before first LLM call) and output (before return). ### Polyglot SDKs One Rust core. Python via PyO3, TypeScript via napi-rs, Rust native. Plus HTTP server (onto-cli serve) with OpenAPI for any language. Same semantics across all four. ### MCP + Claude Skills MCP transports: stdio, HTTP, SSE. Tools register as mcp:: with tools:mcp scope. Claude Skills register as skill: with skills:read scope. Same config shape as Claude Desktop, Cursor, VS Code. --- ## Comparison | Capability | Claude SDK | OpenAI SDK | Google ADK | ONTO | | --- | --- | --- | --- | --- | | Default model | Claude 4.5/4.6 | OpenAI Responses | Gemini | Ollama Cloud (free open-weight) | | Memory model | Conversation | Conversation | Conversation | Typed UPO + SMO | | Plan mode | Feature in Claude Code | Pattern via handoffs | Workflow-as-plan | First-class runtime mode | | Languages | Python, TS | Python | Python, TS, Go, Java | Python, TS, Rust | | Policy/consent | Tool allowlist | Guardrails | Tool allowlist | Per-call consent scopes | | Ontology extraction | DIY | DIY | DIY | Native, 4-level | | On-premise | Hosted only | Hosted only | GCP only | Yes — Helm + Postgres + Sled | | License | Anthropic Commercial | MIT | Apache-2.0 | MIT or Apache-2.0 | Pick Claude Agent SDK if your priority is code-editing agents and the deepest MCP ecosystem. Pick OpenAI Agents SDK for voice/realtime. Pick Google ADK for GCP-native workflow graphs. Pick ONTO for durable typed memory, on-prem deployment, polyglot SDKs, and Plan Mode as a first-class primitive. --- ## Industries ### Healthcare HIPAA-aligned. Role-based consent scopes (intake nurse vs clinician). [phi] tags on memory writes. Plan Mode before any clinical order. On-prem deployment with Sled or Postgres + local Ollama. Healthcare-triage example in the repo. ### Customer support Durable per-account memory partitioned by tenant_id + user_id. Plan Mode for refunds and escalations with policy-driven auto-approval. Tenant-scoped consent prevents cross-tenant leak. Per-tenant cost meter for SaaS billing. ### Personal AI UPO + SMO for durable cross-session memory. Async writes (extract_async=True) for snappy turns. Background summarization on a cheaper model. Open-weight defaults; on-device path via local Ollama + Sled. ### Research Domain-scoped extraction partitions claims by field. Conflict detection between contradictory assertions surfaces as Plan Mode tasks. Background consolidation for large corpora. Per-researcher cost rollup. --- ## Articles ### Why Agent Memory Is Broken (And the Path to Fix It) Published: 2026-05-25 · 14 min · thesis URL: https://onto.neullabs.com/blog/why-agent-memory-is-broken Tags: agent memory, architecture, thesis import Callout from '@components/article/Callout.astro'; import Pullquote from '@components/article/Pullquote.astro'; Three months into running a personal AI assistant, you notice the same thing every week: it doesn't know your timezone. You told it. Twice. It's gone. This is not a small bug. It's an architectural choice. The standard agent loop has two memory primitives, and both are broken for anything that has to last past a single session. ## The two flavors of broken **Conversation buffer memory** holds the current turn plus a sliding window of recent turns. When the window rolls, the agent forgets. A 200k context buys you longer recall — it does not buy you memory. The instant the user closes the tab, what they said is gone. **Vector memory** stores embeddings of past content and retrieves the top-k similar chunks at the start of each turn. This is fine for "find a document like this." It is not fine for "is this true." A user's timezone is a fact, not a similarity. Asking "is this user in UTC?" by computing the cosine distance between sentences is a category error. "What looks similar to this question" is a different question than "what is true about this user." Vector stores answer the first. They do not answer the second. Most production agents stack both: a buffer for the current session plus a vector store for retrieval-augmented generation. The result is an agent that knows what you said five minutes ago and can find documents you wrote last year, but cannot tell you what your job title is. You told it. It re-derives. ## What memory actually has to do Step back from how agent SDKs implement memory and ask what memory has to *do* in a long-running assistant. Four things: 1. **Hold facts that survive sessions.** "Alice is in UTC." "Alice prefers mornings." Not chat history. Facts. 2. **Carry provenance.** Where did this fact come from? Which session? Which message? Which extraction prompt? 3. **Carry confidence and decay.** "Alice said she's in Lyon" decays faster than "Alice's home country is the UK." The system has to know which. 4. **Carry scopes.** Some facts are PHI. Some are MNPI. Some are scoped to a tenant. The fact has to know what consent reads and writes need. A conversation buffer does none of these. A vector store does one (it carries the source chunk, sort of). Neither is enough. ## Typed memory: facts as first-class objects The fix is older than agents: store assertions, not chunks. A typed memory layer holds rows like: ```python Assertion( subject="alice", predicate="timezone", object="UTC", confidence=0.95, provenance=Source( session_id="s_2026_05_01", message_id="m_42", extraction_prompt="user-message-extraction-v3", ), decay=Decay.SLOW, # timezones don't change daily consent_scopes=["memory:write"], tags=["pii"], # routing for compliance created_at=..., superseded_by=None, ) ``` This row answers the question the buffer and the vector store can't: "is this true now, and how confident are we." It also has the provenance an auditor needs and the tags a compliance system needs. You can store a million of these. You can sort by recency, filter by tag, decay by curve, and supersede by a newer assertion. You can encrypt the `[pii]`-tagged rows at rest. You can ship the whole thing to Postgres and bound it by tenant_id. This is what we mean by typed memory. It is not a clever embedding trick. It is just *storing the actual fact* instead of storing the words around the fact. Vector memory tells you what's similar. Conversation memory tells you what's recent. Neither tells you what's true. That's the gap typed memory closes. ## Two ontologies, two lifecycles ONTO splits typed memory into two layers because facts have two lifecycles. **UPO — User Personalization Ontology.** Durable. Survives sessions. Has decay but tends to live. Examples: timezone, name, employer, account tier, preferred language. The agent loads UPO at the start of every interaction. **SMO — Session Memory Ontology.** Transient. Lives inside the session. Built up turn by turn. Examples: "current trip city is Lyon," "user is debugging the auth flow," "intent is rebooking." Most SMO assertions evaporate when the session ends. Some — the ones that *earn* durability through repetition or explicit signal — promote to UPO. The split matters because durability is expensive. Every UPO row is something an auditor can ask about. You do not want every passing comment to land there. SMO is the scratchpad; UPO is the ledger. ## Four levels of extraction The other half of the fix is how facts get *into* memory. The standard agent stack is "user said something → put it in the buffer → maybe later RAG retrieves it." That's not extraction. That's storage. ONTO routes natural language to one of four extraction levels: - **session**: transient context (current city, current intent, current document) - **profile**: durable identity (timezone, name, preferences) - **domain**: app-scoped, multi-tenant facts (this customer's tier, this clinic's protocols) - **tool**: tool-local state (a saved query, a search filter) The LLM proposes assertions and labels each with a level. The runtime routes by the label, writes through Policy Guard, and audits the decision. The user sees their typed graph grow. The auditor sees the source prompt and the routing decision. This is the difference between "the model put something in memory" and "the system stored a typed fact, at a known level, under known consent, with a known source." The first is opaque. The second is a system. ## Why this matters more in 2026 Two trends make this urgent. First, agents are running longer. Personal AI, customer-facing copilots, regulated workflows — every one of these has session lengths measured in months, not turns. Conversation buffers were never built for that. Second, the regulators arrived. HIPAA, GDPR, the EU AI Act, the FINOS AI Governance Framework. Every one of these asks "show me the data the model used and the decision it made." A model that re-derives facts from a buffer cannot answer that. A typed memory layer answers it by construction. If you are running an agent today, audit your memory layer. If your answer to "where do we store the user's timezone" is "the buffer" or "RAG retrieves it," you have a typed memory hole. Plugging it does not require a rewrite — it requires writing the facts you already know down, in a typed schema, with provenance. ## The "but vectors are useful" caveat To be clear: vectors are useful. ONTO ships vector search. The right model is *both*: - Typed memory for facts you can name (timezone, tier, preferences, schedules). - Vector memory for content you can't pre-typify (notes, emails, documents you index for retrieval). The mistake is using vectors for the facts. Storing "Alice's timezone is UTC" as an embedding and retrieving it later by similarity is what's broken. Storing it as an Assertion with a decay curve and a consent scope is what's right. ## What you give up Typed memory has costs. You need to define a schema. You need to write extraction prompts that target the schema. You need to handle conflicts when the user changes their mind. You need to deal with decay. None of these are free. What you get: an agent that knows who you are. An audit trail. A migration path when the schema evolves. A compliance story when the regulator calls. A foundation under whatever model you swap in next. That's the trade. Sometimes the right thing is also the harder thing. ## Where this leaves us The standard agent loop is good at one-shot tasks. It is bad at long-running ones because its memory primitives are mismatched to the problem. Conversation buffer memory is a side effect of how transformers see input. Vector memory is a side effect of how search engines compare documents. Neither was designed to answer "what is true about this user." Typed memory was designed for that. UPO + SMO are one way to do it; you could roll your own. What matters is that you stop treating the buffer as memory and the vector store as a fact ledger. They are not. If you build an agent that is going to live longer than a session, you are going to invent typed memory eventually. The choice is whether you invent it in a hurry, the day the auditor calls, or whether you start with it. Start with it. --- *Want to see typed memory in action? [Install ONTO](/docs) and run the extraction example — typed facts land in UPO with one flag, and you can query them with one line. The [features page](/features#typed-memory) has the deeper architecture and the [comparison page](/comparison) shows how the other SDKs handle memory differently.* --- ### Plan Mode: Approving What Your AI Agent Does Before It Does It Published: 2026-05-22 · 12 min · thesis URL: https://onto.neullabs.com/blog/plan-mode-explained Tags: Plan Mode, human-in-the-loop, agent safety import Callout from '@components/article/Callout.astro'; Most "human in the loop" implementations look something like this. The agent runs. A tool is tagged "needs approval." When it fires, the SDK pauses, shows a confirmation dialog, waits for the user to click yes, then resumes. This works. It's also not very useful, because the agent has *already started* by the time the dialog appears. Side effects earlier in the chain have already fired. Memory has been written. State has been mutated. The dialog is asking "okay to do one more thing?" — and the answer is meaningless because the world has already moved. Plan Mode is what you get when you take human-in-the-loop seriously enough to make it a runtime mode. ## The mode itself A run in Plan Mode produces no side effects. It calls `task.create` as many times as it wants — that's a memory write into a typed task list, which is allowed under `tasks:write` scope — and then it stops. Anything that would write to the world, hit an external API, or mutate user state refuses with a structured error. ```python # Plan run — no side effects. plan = onto.run_sync( "Plan a 3-day trip to Lyon.", user_id="alice", mode="plan_only", consent_scopes=["tasks:write", "tasks:read"], ) # plan.tasks is a list of typed Task objects. # The agent proposed: [book_hotel, book_train, reserve_restaurant, ...] # None of them have fired. ``` The human now has something to look at. A typed list of intended actions. Each task carries the tool name, the arguments, the cost estimate, and a model-generated rationale. They can approve all, approve some, edit arguments, ask the agent to re-plan, or just kill it. ```python # Human approves task by task. Then a second run, with execute scopes. for task in plan.tasks: if human_approves(task): onto.execute_task( task, consent_scopes=["tools:web.fetch", "tools:http_request"], ) ``` The second run is where side effects happen. It carries the execute scopes; the Plan run did not. ## Why a runtime mode, not a wrapper You can build something like Plan Mode in any SDK by tagging tools and intercepting them. We tried this. It breaks in three places. **First, the agent doesn't know it's in "plan mode."** A wrapper sits *outside* the agent. The agent sees tools, calls them, and gets back "denied, ask the user first." This produces a long agent retry loop and confused model behavior. The model writes apologies and tries to call the tool again with the same arguments. You waste tokens. **Second, the contract leaks.** If a tool isn't tagged, it fires. If a developer forgets the tag, it fires. If a new tool is added without the tag, it fires. The system depends on every tool author remembering a thing they could forget. The guarantee is best-effort. **Third, you can't distinguish proposal from execution in the audit log.** Both look like tool calls. You can grep for "denied" but you can't reconstruct the human's review trail. Making Plan Mode a runtime mode fixes all three. The agent gets a system message saying "you are in plan_only mode; create tasks for what you would do." The runtime refuses any tool whose `required_consent` isn't covered. The audit log has separate Plan and Execute rows tied by `plan_id`. "This tool only fires if it's tagged for execute" is a convention. "This tool only fires if the run carries scopes that cover its required_consent" is a runtime invariant. The second is what an auditor will trust. ## The Tasks primitive For Plan Mode to be useful, "what the agent proposes" has to be a thing you can hold. ONTO ships a Task primitive. A Task has: - `tool`: the tool name and arguments the agent wants to call - `rationale`: the model's own explanation of why - `cost_estimate`: tokens or dollars, computed at plan time - `depends_on`: other task IDs this one needs first - `state`: `proposed` → `approved` | `rejected` | `executed` This is data, not chat. You can render it in a UI. You can pipe it through your existing approval workflow. You can store the approval decision next to the task. You can replay the whole plan run when an auditor asks why a refund was issued. ## Where this maps to real work Three use cases that fail without something like Plan Mode: **Healthcare orders.** A clinical copilot proposes "order ECG, order troponin panel, schedule cardiology consult." Each is irreversible-ish (billing fires, the lab runs). The clinician must see and approve the list before any order is placed. Plan Mode → human approval → Execute with `patient:order` scopes. **Customer support refunds.** An agent proposes "refund $500, send apology email, flag account for ops review." For a free-tier account this needs human approval; for a gold tier it auto-approves under policy. Plan Mode lets you write that policy in plain code, outside the LLM. **Personal AI booking.** An assistant proposes "book this hotel, reserve this restaurant, decline these two meetings." The user reads the plan in their email, taps approve, and the second run fires. The user is in control of which actions happen, in what order, with what arguments. In every case the structure is the same: the agent gets to *think* — including thinking about external state via reads — without getting to *act*. Acting is a separate run with separate scopes. ## What about agents that just respond? If your agent only answers questions and never writes to the world, you don't need Plan Mode. A read-only chat agent runs with `["memory:read"]` and no execute scopes. There is nothing to gate. Plan Mode is for agents that *do* things. Once you cross that line — your agent calls APIs, books rooms, places orders, sends emails, files tickets, mutates the database — the question of "did the human approve this" goes from theoretical to load-bearing. Plan Mode is the line. ## The escape hatch For agents that need to act without a human (a cron job, a background processor), you skip Plan Mode entirely. Run with execute scopes. The agent acts. The audit trail records every action with its consent scopes, so when something goes wrong, you can answer "who authorized this" with "the cron job, here are the scopes it had." This is the right design: Plan Mode is opt-in *per run*, not a global setting. You pick the right mode for the right caller. ## What you gain when this is built in The honest answer for what changes is twofold: 1. **You stop writing apology code.** "Sorry, the agent did X and we didn't mean it to" is a class of incident that mostly comes from agents acting before someone said yes. Plan Mode by construction eliminates the agent-acted-before-approval bucket. 2. **Auditors stop asking the same question.** Every regulated workflow eventually gets "show me the human approval for action Y." With Plan Mode you produce the plan, the approval, and the execution rows. With a wrapper-based approach you produce a denied row and ask the auditor to trust you. Neither of these is the kind of feature you'd find exciting at a hackathon. Both are the kind of feature that lets you sleep when your agent is in production. --- *[Read the Plan Mode reference](https://onto-framework.github.io/onto-framework/concepts/tasks-and-plan-mode/) for the full task schema and execute pattern. The [comparison page](/comparison) shows how Claude, OpenAI, and Google ADK handle planning differently.* --- ### Typed Memory vs Vector Embeddings for AI Agents Published: 2026-05-20 · 10 min · thesis URL: https://onto.neullabs.com/blog/typed-memory-vs-embeddings Tags: typed memory, vector search, agent memory import Callout from '@components/article/Callout.astro'; Pop quiz. The user says "I'm in Lyon for the week." Your agent has to remember this. How do you store it? **Option A.** Append the message to the conversation buffer. **Option B.** Embed the message and put it in a vector store. **Option C.** Extract `Alice.current_location = "Lyon"` with `decay=session_scoped, confidence=0.9, provenance=msg_42, consent_scopes=["memory:write"]` and store it as a typed assertion. If your answer is A or B, your agent will be asking "where are you again?" by Wednesday. ## What vectors are good at, and what they aren't Vector embeddings encode a meaning-vector for text. Similar meanings land near each other in the space. This is wonderful for one task: finding things that are *like* your query. "Find me notes similar to this email." "Find me docs that mention this concept." Vectors crush this. Here's what they're not good at: answering questions about state. "What's the user's timezone right now?" is not a similarity query. It is a state lookup. Asking "what's the most similar prior message to 'where is the user'" and reading off the answer is a Rube Goldberg machine for what should be a row lookup. If you can write the question as "given (subject, predicate), what is the current object?" — it is a state question. Use typed memory. If you can only write it as "find things like this," it is a similarity question. Use vectors. ## What typed memory looks like A typed assertion is a row with structure. In ONTO, the minimum is: ```python Assertion( subject="alice", predicate="current_location", object="Lyon", level="session", # session | profile | domain | tool confidence=0.9, decay=Decay.SESSION, provenance=Source(message_id="m_42", session_id="s_2026_05_20"), consent_scopes=["memory:write"], tags=["pii"], created_at=..., ) ``` This row has properties a vector embedding does not have: - **Identity.** `(subject, predicate)` is a key. The next time the user says "I'm in Berlin," this row is *superseded* — not appended near, replaced. - **Decay.** Session-scoped facts disappear with the session. Profile facts (like timezone) decay slowly. Domain facts (a customer's tier) decay on a policy curve. - **Provenance.** We know exactly which message produced this fact and which extraction prompt. We can replay. - **Confidence.** Low-confidence facts ("I think I'm in Lyon") sit below high-confidence facts ("I am in Lyon"). The agent can weight them. - **Consent scopes.** The write required `memory:write`. Reads require `memory:read`. Cross-tenant reads fail by construction. - **Regulation tags.** `[pii]` lets the compliance layer route this row to encrypted storage and apply retention rules. A vector embedding has none of these. It has a number array and (maybe) a metadata payload that you bolted on. To get the equivalent you re-invent every property above, less robustly. ## The mistake everyone makes once The reason most agents end up with vector-only memory is that vectors are easy. Embed everything; the LLM will figure it out at retrieval time. This works in the demo. It fails in production because the LLM can't actually figure it out — it just confabulates plausible answers from whatever fragments it retrieves, and you don't notice until a customer asks why the agent thinks they live in two cities. The fix is not "better RAG." The fix is to recognize that some things are facts and some things are documents, and to store them differently. ## When to use both A real agent has a memory layer that uses both: - **Typed memory (UPO + SMO)** for the things you can name. User profile, account state, session intent, current document being worked on, recently completed tasks. Anything where (subject, predicate) is a sensible question. - **Vector store** for the things you can't pre-typify. Notes, emails, documents, code, knowledge-base articles. Anything where "find me similar content" is the right query. ONTO ships both. Typed assertions go into UPO/SMO; documents go into the vector index. The agent's prompt loads typed facts at the start of every turn (one query) and pulls in relevant documents on demand (vector search). The system uses each primitive for what it's good at. ```python # Typed lookup: O(1) — is this true now? user = onto.memory.get_profile("alice") print(user.timezone) # "UTC" # Vector search: top-k similar — find content like this notes = onto.search.query( "deployment notes for the auth migration", tenant_id="acme", top_k=5, ) ``` Two primitives. Two queries. No confusion about which is for which. ## The conflict resolution story Typed memory enables something vectors can't: structured conflict resolution. When the user said "I'm in Lyon" on Monday and "I'm in Berlin" on Wednesday, both assertions point at `(alice, current_location)`. The system detects this conflict — same key, different objects — and creates a Plan Mode task: "user previously said Lyon; now says Berlin. Supersede?" Approve and the Lyon row is marked `superseded_by=berlin_row_id`. The agent now reads Berlin and ignores Lyon. Vectors can't do this. Two near-duplicate embeddings both retrieve. The model gets both and picks one (badly). The contradiction never surfaces. For documents that should both exist, that's fine. For facts that should resolve to one current value, it's not. ## The compliance story Auditors ask three questions about your AI: 1. What data did the model use? 2. What did it conclude? 3. Who approved the action? Typed memory answers all three by construction. Every assertion has provenance (data source), confidence (conclusion), and `[pii]`/`[phi]`/`[mnpi]` tags (compliance routing). The Plan Mode log answers approval. Vector-only memory answers none of these directly. You can get to "the model retrieved these chunks" but not to "the model believed this fact at this confidence." ## What you give up Typed memory has a real cost: schema. You have to define your predicates. You have to write extraction prompts that target them. You have to handle evolution as the schema changes. This is genuine work. The reason it's worth doing is that the alternative — storing everything as text and asking the model to derive facts on every call — costs more in tokens, latency, and confusion. Once the schema settles, extraction is automated (set `extract=True` and the runtime routes things to the right level), and you stop paying for re-derivation. ## A small principle Use typed memory for facts. Use vectors for content. Use both, deliberately. Stop treating embeddings as a universal memory primitive — they're not, and the bug shows up at exactly the moment your agent has been running long enough to matter. --- *ONTO's typed memory model is described in detail on the [features page](/features#typed-memory) and in [the broader memory thesis](/blog/why-agent-memory-is-broken). The [comparison page](/comparison) shows how other SDKs handle this.* --- ### The Case for Open-Weight Defaults in Agent Frameworks Published: 2026-05-18 · 9 min · thesis URL: https://onto.neullabs.com/blog/open-weight-defaults Tags: open-weight, Ollama, model providers import Callout from '@components/article/Callout.astro'; When you install an agent SDK and `import` it for the first time, the SDK has to make a choice for you. Which model does it call by default? The answer to that question is a statement about whose problem the SDK is solving. Most agent SDKs default to their parent company's hosted proprietary model. Claude SDK defaults to Claude. OpenAI SDK defaults to GPT. Google ADK defaults to Gemini. This is reasonable from each vendor's perspective. It is unreasonable from yours. ## What "default" actually does The default model in an SDK shapes three things: 1. **Onboarding.** A new user runs the quickstart. Did it work? If the default needed a paid API key, the answer is "not yet." 2. **Lock-in.** People build for the default. Switching providers later means changing prompts, retesting tools, and re-evaluating quality. Most teams never do it. 3. **Deployment options.** If the default can't run on-prem, the SDK's on-prem story is theoretical. Production teams notice this on day one. Defaulting to a proprietary hosted model optimizes for the vendor on all three. It minimizes the friction of *trying* the SDK, but only if you already have the vendor's API key. It locks people in to that model. And it makes on-prem an afterthought. ## What an open-weight default does Defaulting to an open-weight model — running either locally or on a free cloud tier — flips the optimization toward the user: 1. **Onboarding works without an account.** ONTO defaults to Ollama Cloud's free open-weight tier. `pip install onto-core`, set one env var, you're running. No credit card, no vendor relationship. 2. **No lock-in to start.** When you build on an open-weight default, your prompts and tools are tested against a model whose weights are public. Swapping to a different open-weight model is a config change. 3. **On-prem is a one-line switch.** Point at a local Ollama. Same model class, same prompts. Done. The proprietary models are still there — Claude and GPT are one env var away. They're upgrades, not requirements. A proprietary default makes the open-weight path a workaround. An open-weight default makes the proprietary path an upgrade. The SDK communicates whose interest it serves through which one it picked. ## "Are open-weight models even good enough" Three years ago this question had a clear answer: no. Today the answer is "for what." For frontier reasoning — multi-step math, long-context comprehension on hard documents, very subtle instruction following — proprietary frontier models are still the strongest. There's a real gap. For the bulk of agent work — extracting typed facts from user messages, generating a plan of tool calls, summarizing a session, structured output to a JSON schema, calling a single tool with the right arguments — modern open-weight models like gpt-oss:20b, Llama 3.1, and Qwen 2.5 are good enough. We've measured. They handle ONTO's extraction loop, plan mode, and tool-calling guardrails without trouble. The right pattern is to **default to open-weight, upgrade where you measure a lift**. Build, ship, measure. If a specific workload measurably benefits from Claude or GPT, route that workload — and only that workload — to the upgraded provider. The SDK should let you make this routing decision at the call site, not the install site. ## The on-prem dimension For regulated industries this isn't a preference, it's a requirement. Healthcare data can't leave the VPC. Financial data has retention rules that hosted vendors won't sign. Legal work has confidentiality requirements. If your SDK can't run with everything on-prem, you're not building for these users. You're building a hosted-only product and asking these users to make an exception. Most of them won't. ONTO's on-prem path is straightforward: - Run Ollama or vLLM locally with an open-weight model - Store assertions in Postgres + pgvector inside your VPC - Embed with fastembed locally - Deploy with the included Helm chart on your own Kubernetes No third-party API hits. The cost is the model serving infrastructure you'd run anyway. ## The cost dimension Open-weight models on a free or low-cost tier change the unit economics of agents that do a lot of small tasks. Memory summarization, conflict detection, extraction — these can run hundreds of times per session in a busy agent. If each of those goes to a $0.005 GPT-4 call, your bill is a problem. If they go to a free Ollama Cloud open-weight call (or a self-hosted one), your bill is the foreground turns only. ONTO's background memory worker pattern leans on this: route background summarization to a cheaper model by default. ## The "ecosystem" objection The pushback against open-weight defaults is usually "but the ecosystem and quality and tools are better on the proprietary side." This is sometimes true. It's also irrelevant to the *default* choice. The default is what runs when you don't pick. Picking should be an active decision, not the by-product of an install. An SDK that asks you "by the way, please configure a paid API key before this works" is asking for a commitment before you've evaluated. An SDK that runs out of the box on free open-weight, then lets you upgrade to Claude or GPT with one env var, is letting you evaluate first and commit second. ## What this looks like in practice ```python # Default — Ollama Cloud free open-weight. No key needed. onto = Onto(model=Models.GPT_OSS_20B) # Same code, Claude. Set ANTHROPIC_API_KEY. onto = Onto(model=Models.CLAUDE_SONNET_46) # Same code, OpenAI. Set OPENAI_API_KEY. onto = Onto(model=Models.GPT_5) # Same code, local Ollama. Set OLLAMA_BASE_URL. onto = Onto(model=Models.GPT_OSS_20B) # routes to local ``` The library doesn't care which one you pick. The library cares that the default lets you start without asking permission. ## A final argument There's a structural reason this matters beyond ergonomics. When an SDK's default is its parent company's proprietary model, every tutorial, every example, every blog post is implicitly an advertisement for that model. The ecosystem the SDK builds is an ecosystem of users who tested only against the proprietary model. That ecosystem then has trouble running anywhere else. The SDK is technically provider-agnostic; in practice it's lock-in by inertia. An open-weight default breaks that. Tutorials work on the free tier. Examples ship with hashes that match what the open-weight model produces. New users build prompts that aren't covertly tuned to a particular proprietary quirk. The ecosystem becomes portable by construction. If you're building agent infrastructure that you want to outlive the current generation of frontier models, defaulting to open-weight is the only choice that doesn't quietly bet on one vendor. --- *ONTO defaults to Ollama Cloud's free open-weight tier. Swap to Claude or GPT with one env var — details on the [docs page](/docs) and the [features page](/features#open-weight).* --- ### Stateless Agents, Stateful Humans: A New Architecture for AI Agents Published: 2026-05-15 · 11 min · thesis URL: https://onto.neullabs.com/blog/stateless-agents-stateful-humans Tags: agent architecture, thesis, design The standard agent architecture, in one sentence: a loop where the model receives observations, picks an action, calls a tool, observes the result, and repeats until done. This works because LLMs are good at the "pick an action" step. The problem is that in this architecture, picking and acting are the same step. The model decides what to do *and* the system does it, with no gap between them. State changes happen inside reasoning. For one-shot tasks that's fine. For long-running, regulated, multi-tenant, or just-shipping-to-real-users work, it's a structural mistake. ## The split The architecture ONTO is built on: - **The agent is stateless about the world.** It reads typed state at the start of each run, proposes actions and memory writes, and returns. It does not mutate. - **The human (or a deterministic gate) is stateful.** They hold the durable memory, the approval log, the policy decisions. They — not the model — write to the world. The model's job becomes proposing. The human's job becomes deciding. The runtime's job is to enforce that the model can only propose and the human can only execute. Three runtime primitives make the split structural rather than conventional: 1. **Memory writes are explicit.** Every UPO write is an `add_assertion` call with consent scopes and audit metadata. The runtime — not the model — does the write. 2. **Tool execution carries consent scopes.** Every tool declares what scopes it needs. Every run carries what scopes you grant. The runtime refuses tools whose scopes don't cover what the call needs. 3. **Plan Mode is a runtime invariant.** A run in `mode="plan_only"` creates tasks but does not fire side-effect tools. The split between Plan and Execute is enforced at the runtime layer, not the prompt layer. ## Why this is more than aesthetics Three concrete consequences. ### You stop hallucinating side effects In an autonomous loop, the model can "decide" to call a tool you didn't intend it to call. The standard mitigation is a tool allowlist — the agent can only call tools in a list. This helps but doesn't fix the fundamental issue: even an allowed tool, called at the wrong time with the wrong arguments, can cause harm. With Plan Mode, the LLM's intent goes into a typed Task object first. The arguments are visible. The cost is estimated. A human (or policy) decides whether to fire the tool. The LLM never gets the satisfaction of acting on its own decision — it gets the satisfaction of having proposed correctly. ### You get an audit trail for free Auditors ask "show me the AI's reasoning, the data it used, and the human approval for action X." In the autonomous loop, the answer is "here's the transcript, please trust us." In the stateless-agent / stateful-human architecture, the answer is a structured record: - The Run row, with input, scopes, model_id - The Assertion writes, with provenance and tags - The Task proposals, with arguments and cost - The approval decisions - The Execute Run, with the executed tools You produce these by construction, because the system already has to track them to enforce the split. ### You decouple memory from the model In the autonomous loop, memory lives in the context window. The model sees what it can see. Anything past the window is gone. In the stateless-agent architecture, memory lives in typed storage you control. The model loads relevant facts at the start of a turn, proposes new ones, and returns. The window is just a buffer for the current turn — the durable state is somewhere else. This is the thing that lets agents live longer than a session. The model can change. The memory persists. ## What it costs The architecture has real costs. Three of them. **More moving parts.** A Plan run, an approval step, an Execute run is more code than a single autonomous loop. The flow is more explicit. **Latency for the approval step.** If your use case is "user types message, agent acts in 2 seconds," and approval is needed, you've blown the latency budget. The right pattern for these cases is to run with execute scopes directly (no Plan Mode), accepting the autonomous loop's risk model in exchange for speed. **Schema design.** Typed memory means you have to think about what predicates to track. There's no free schema-on-write — the cost of the schema is real. These costs are the price of getting the architecture's benefits. For one-shot creative tasks they're not worth paying. For long-running personal assistants and regulated industry work they pay back many times over. ## The escape valves The architecture has two important escape valves that keep it practical. **Run with execute scopes when it's safe.** A read-only chat agent runs with `["memory:read"]`. There's nothing to gate. Plan Mode is opt-in per run, not a global mode. **Auto-approve under policy.** When the gate is "is this a gold-tier customer with a refund under $50," you don't need a human. Plan Mode produces tasks; a policy function approves them; Execute fires. The architecture doesn't say humans must be involved — it says the *decision to execute* must be separate from the *decision to propose*. These let the same SDK serve the read-only chat use case and the high-stakes-clinical-order use case without code duplication. ## Where this came from This isn't an invention. Operating systems split user-space (the program proposes a syscall) from kernel-space (the kernel executes it under privilege checks). Databases split clients (which propose writes) from the engine (which executes them under integrity constraints). The pattern of "untrusted thing proposes, trusted thing decides" is everywhere good systems are built. What's new is that agent SDKs have, mostly, *not* applied this pattern. The autonomous loop treats the model as both the proposer and the executor. That's user-space being allowed to do its own syscalls. It works in toy examples and fails in production. ONTO is what you get when you take this OS lesson seriously and apply it at the runtime level. The model proposes. The runtime executes — under consent checks, with audit, with Plan Mode as a halt point. ## Where it doesn't apply If you're shipping a one-shot creative tool — "write me a poem about my cat" — none of this matters. There is no durable state, no compliance question, no multi-session memory, no risk of irreversible side effects. The autonomous loop is fine. The architecture matters precisely when those things show up. Once your agent has memory across sessions, or scopes data across tenants, or fires tools that touch the world, the split is worth its costs. ## A summary you can use If someone asks what's different about ONTO's architecture, the one-liner is: **the agent reads state and proposes actions; the human writes state and approves them.** Everything else — Plan Mode, consent scopes, typed memory, audit logs — is structure built around that split. It's a small reframing with large consequences. Once you see it you can't unsee it. Once you build on it you can't go back to the autonomous loop. --- *The [features page](/features) walks through each runtime primitive in detail. [Why agent memory is broken](/blog/why-agent-memory-is-broken) covers the memory side; [Plan Mode explained](/blog/plan-mode-explained) covers the execution side.* --- ### Agent SDK Comparison: ONTO vs Claude vs OpenAI vs Google ADK (2026) Published: 2026-05-12 · 12 min · comparison URL: https://onto.neullabs.com/blog/agent-sdk-comparison-2026 Tags: comparison, Claude SDK, OpenAI SDK, Google ADK import Callout from '@components/article/Callout.astro'; There are roughly four agent SDKs worth considering today, plus a long tail of orchestration libraries. This post compares the four head-to-head: Claude Agent SDK, OpenAI Agents SDK, Google ADK, and ONTO. Same structure for each: what they're built for, where they win, where they lose. At the end, a decision flow. ## Claude Agent SDK **Built for:** code-editing agents and long-context reasoning workflows with Anthropic's models. **Wins:** - Best-in-class code tooling. The `Read`, `Write`, `Edit`, `Bash`, and search tools are the most refined. - Deepest MCP (Model Context Protocol) ecosystem. Most community MCP servers target Claude first. - Claude Skills as a packaging format for reusable agent capabilities. - Anthropic's hosted tracing and prompt caching. **Loses on:** - Memory model. Conversation buffer only; you bolt on your own typed store. - Lock-in to Anthropic models — Claude is the only first-class provider. - No on-prem story. Hosted API is the only path. - Single-language SDK ecosystem (Python + TypeScript). - Plan mode exists inside Claude Code, but is not a runtime invariant available to third-party agent builders. **Pick it when:** you're building a code-editing agent or sitting deep inside the Anthropic ecosystem with Claude Skills. ## OpenAI Agents SDK **Built for:** OpenAI-native agents, especially voice and Realtime API. **Wins:** - Best voice and realtime support. The Realtime API integration is genuinely good. - Tightest integration with OpenAI features (Responses, file tools, computer use). - Battle-tested for high-traffic chat workloads. **Loses on:** - OpenAI-only. The whole SDK assumes you're calling OpenAI endpoints. - Memory is conversation-buffer; you bolt on the rest. - No on-prem story. - Single language (Python, with community TypeScript ports). - Guardrails are useful but bolt-on, not runtime invariants. **Pick it when:** voice / realtime is the use case, or you're locked into OpenAI for other reasons. ## Google ADK **Built for:** GCP-native agents with declarative graph workflows. **Wins:** - Workflow-as-code with explicit graphs (good for compliance-heavy deterministic flows). - Go and Java SDKs alongside Python and TypeScript — the only SDK with serious Go support. - Tight integration with Vertex AI, BigQuery, and other GCP services. **Loses on:** - GCP lock-in. The deployment story is heavy on Google services. - Workflow-as-graph fits some problems and forces others into an awkward shape. - Memory: conversation buffer. - Gemini is the default model; provider-agnostic but tilted toward Google. **Pick it when:** you're on GCP, your team writes Go or Java, and your workflows fit a declarative graph cleanly. ## ONTO **Built for:** long-running agents with durable memory, in regulated and multi-tenant environments, that need to swap models freely. **Wins:** - Typed memory (UPO + SMO) with provenance, decay, consent scopes, and regulation tags. - Plan Mode is a first-class runtime mode — not a prompting convention. - Per-call consent scopes (Policy Guard) enforced by the runtime. - Polyglot: native Python, TypeScript, and Rust SDKs from one Rust core. - Open-weight default. No API key relationship needed to start. - On-prem path: Sled or Postgres + pgvector + local Ollama + Helm. - Honest cost meter that splits foreground vs background per user / per tenant. **Loses on:** - No `fs.write` / `Bash` first-party tools yet — bring your own or wire MCP. - No voice / realtime path. - Pre-1.0 (0.1.0). We break the SDK when it makes the SDK better, tracked in the public roadmap. - Fewer community tutorials than Claude or OpenAI SDKs (catching up). **Pick it when:** you need durable memory, on-prem deployment, polyglot SDKs, or honest cost observability. ## The decision flow A pragmatic flow if you're still picking: ``` Is your agent editing code? → Claude Agent SDK. Is voice / realtime in the requirement? → OpenAI Agents SDK. Are you all-in on GCP with Go/Java? → Google ADK. Do you need durable cross-session memory? → ONTO. Do you need on-prem deployment? → ONTO. Do you need a Rust SDK? → ONTO. None of the above strongly? → Default to ONTO. Easier to migrate out of than into. ``` ## The honest matrix | Capability | Claude SDK | OpenAI SDK | Google ADK | ONTO | | ------------------ | ---------- | ---------- | ---------- | ---- | | Default model | Claude | GPT | Gemini | Open-weight (Ollama Cloud) | | Memory model | Conversation | Conversation | Conversation | Typed UPO + SMO | | Plan mode | Claude Code feature | Pattern | Workflow-as-plan | First-class runtime mode | | Languages | Py, TS | Py | Py, TS, Go, Java | Py, TS, Rust | | Policy / consent | Tool allowlist | Guardrails | Tool allowlist | Per-call consent scopes | | Ontology extraction | DIY | DIY | DIY | Native, 4-level | | On-premise | No | No | GCP only | Yes — Helm + Postgres | | License | Anthropic Commercial | MIT | Apache-2.0 | MIT or Apache-2.0 | ## What this comparison is *not* about We're not ranking the *models*. Claude, GPT, and Gemini are all strong frontier models. This comparison is about the SDKs around them — how they let you structure memory, gate execution, deploy on-prem, and avoid lock-in. The model conversation is a separate one. Several teams build directly on LangChain/LangGraph or the OpenAI/Anthropic libraries without using an "agent SDK" at all. That's a valid choice too — you get maximum flexibility and minimum guarantees. The agent SDKs above exist to give you guarantees in exchange for some opinions. ## Where we'd be wrong This matrix is a snapshot from mid-2026. All four SDKs are moving. If you're reading this in late 2026 or 2027, expect Claude SDK to have stronger memory primitives, OpenAI to have caught up on tracing tooling, Google to have rounded out the non-GCP story, and ONTO to be past 1.0. The honest matrix lives at [/comparison](/comparison) and gets updated. If you spot something inaccurate, [open an issue](https://github.com/onto-framework/onto-framework/issues/new). The comparison is more useful when it's correct, even when correct hurts. --- *The [comparison page](/comparison) has the live matrix and per-competitor breakdowns. The [features page](/features) covers what each ONTO primitive does in detail.* --- ### ONTO vs LangChain: When Typed Memory Beats Chains Published: 2026-05-10 · 10 min · comparison URL: https://onto.neullabs.com/blog/onto-vs-langchain Tags: comparison, LangChain, agent memory LangChain is, by orders of magnitude, the largest agent orchestration ecosystem. It has the most integrations, the most tutorials, the most stars, and the most production deployments. So the obvious question: why use ONTO instead? The honest answer is that LangChain and ONTO solve overlapping but different problems. This post says what those differences are and helps you pick. ## What LangChain is good at LangChain (and LangGraph, its newer graph-based orchestration layer) is best at: - **Composing chains.** Pipelines of LLM calls, transformations, retrievals, and tools that fit naturally into a directed graph. - **Integration breadth.** Hundreds of community-maintained connectors to vector stores, databases, APIs, and tools. - **Workflow flexibility.** If your application is "input → LLM → retrieve → LLM → tool → LLM → output," LangChain has a primitive for every node. - **LangSmith.** The observability and eval tooling around LangChain is mature. For "I have a workflow that I can draw on a whiteboard, and I want to wire it up," LangChain is the right answer. ## Where LangChain doesn't help What LangChain doesn't ship as a built-in primitive: - **Typed durable memory** with provenance and decay. There's a `ConversationBufferMemory` class and a hundred third-party memory implementations. None ships as a structurally typed, provenance-carrying, consent-scoped store. - **Plan Mode as a runtime invariant.** LangGraph lets you write a plan-execute pattern; the runtime doesn't enforce that side-effect tools refuse outside of execute scopes. - **Per-call consent scopes.** Tool allowlists exist; per-call consent scopes that cascade through the runtime do not. - **Polyglot SDKs from one core.** LangChain is Python-first with a separate TypeScript port (langchain-js); there's no Rust core. For applications where these matter — long-running agents, regulated industries, multi-tenant SaaS, on-prem deployments — you end up building the typed-memory + consent layer yourself on top of LangChain. That's a real engineering project. ## What ONTO ships out of the box The things you'd build on top of LangChain to get production-grade behavior: - UPO + SMO for typed durable memory. - Plan Mode as a runtime mode, not a pattern you implement. - Policy Guard for per-call consent scopes. - Cost meter with per-user / per-tenant rollups. - Four-level extraction routing. - Async writes and background memory worker. ONTO ships these as first-class primitives. You don't have to design them; you have to use them. ## Where ONTO is behind Worth being clear about: - **Integration count.** LangChain has more out-of-the-box integrations. ONTO closes the gap via MCP, but raw breadth is still LangChain's. - **Community.** LangChain's community is bigger. More tutorials, more Stack Overflow answers, more YouTube videos. - **Pre-1.0 churn.** ONTO is at 0.1.0 and will break the SDK to improve it. LangChain is more stable across versions. ## The interoperability path You don't have to pick one. The most productive pattern we see: - **LangChain for the orchestration graph.** If you've already drawn your workflow as a graph and built nodes for retrieval, transformation, and routing. - **ONTO for the memory and consent layer underneath.** Wrap ONTO's memory APIs as LangChain tools. Use the Policy Guard to gate which LangChain nodes can run with which scopes. Or the reverse: use ONTO for the agent runtime and call LangChain components from inside ONTO tools when you need a particular integration. The two ecosystems are not zero-sum. ## Where they actually compete Where LangChain and ONTO genuinely overlap is the "let me run an agent that calls tools and has memory" use case. For *that* use case: - LangChain offers maximum flexibility — every primitive is a building block you compose. - ONTO offers fewer primitives that solve more problems by construction — Plan Mode, typed memory, consent scopes are built in. If your team values flexibility over guarantees, LangChain. If your team values guarantees over flexibility, ONTO. Both are valid trade-offs. ## A test Here's a question that's clarifying. Imagine your agent is in production for six months. A user files a complaint. The auditor (internal or external) asks: "show me what data the agent used to make this decision, when each piece was learned, who approved the action, and the trail from input to side effect." Can you answer that today, with your stack? If yes — great. Whatever you're using is working. If you'd need to instrument that capability after the fact, ONTO's typed memory + Plan Mode + consent log give you the answer by construction. That's the structural benefit you'd pay for in switching costs. ## The pragmatic recommendation - New project, regulated industry: start with ONTO. - New project, lots of integrations, less compliance pressure: start with LangChain. - Existing LangChain project: don't migrate. Add ONTO as the memory and consent layer where it helps. - Existing ONTO project: add LangChain components where you need a specific integration. The choice isn't binary; the choice is which one is the spine of your system and which one supplements. --- *The full ONTO [comparison page](/comparison) covers all four major agent SDKs side by side. The [features page](/features) covers what each ONTO primitive does in detail.* --- ### ONTO vs LlamaIndex: Agents vs RAG, Explained Published: 2026-05-08 · 9 min · comparison URL: https://onto.neullabs.com/blog/onto-vs-llamaindex Tags: comparison, LlamaIndex, RAG The first thing to get straight: LlamaIndex and ONTO are not really competitors. They solve adjacent problems that look similar from far away. LlamaIndex is the leading framework for **retrieval-augmented generation (RAG)** — indexing documents, retrieving the relevant chunks, and synthesizing answers from them. It's deep, well-engineered, and has an enormous integration ecosystem on the data side. ONTO is an **agent SDK** — typed memory, Plan Mode, consent scopes, tool execution. The thing that does work on behalf of a user, often with side effects. You can use them together. Most non-trivial real systems do. ## When LlamaIndex is the right primary tool If your application looks like this: - Index a corpus of documents. - A user asks a question. - Retrieve the relevant chunks. - Synthesize an answer with citations. - Repeat. LlamaIndex is the answer. It will save you months of work over building this from scratch. Plain LlamaIndex (no agent layer) is the right shape for: - Document Q&A applications - Internal knowledge-base assistants - Customer-facing FAQ bots backed by a corpus - Research assistants that summarize a known body of text In these cases ONTO doesn't add a lot — there's no durable user state to manage, no side effects to gate, no Plan Mode flow. ## When ONTO is the right primary tool If your application looks like this: - A user has an ongoing relationship with the assistant. - The assistant has to remember things across sessions. - The assistant takes actions (books, refunds, orders, writes). - Some actions need human approval; others are auto-approved by policy. - The assistant runs in a regulated environment (HIPAA, SOC 2, GDPR, etc.). ONTO is the answer. The typed memory layer, Plan Mode, and Policy Guard scopes are designed for exactly this shape. ## When you need both The interesting case is the overlap: an agent that *also* needs to retrieve from a document corpus. A research copilot. A legal assistant that pulls from case law. A support bot that searches a knowledge base. Here the right architecture is: - **LlamaIndex for the retrieval layer.** Index your corpus, build the right retrievers, expose them as tools. - **ONTO for the agent layer.** Typed memory of the user / case / customer, Plan Mode for any actions, consent scopes for tool calls. - **The LlamaIndex retrievers are ONTO tools.** Wrap them in `Tool(...)` calls with appropriate `required_consent` (e.g., `["search:legal"]`). The two ecosystems compose cleanly. You get RAG quality from LlamaIndex and agent guarantees from ONTO. ```python # Wrap a LlamaIndex retriever as an ONTO tool. from llama_index.core import VectorStoreIndex from onto import Onto, Tool case_index = VectorStoreIndex.from_documents(legal_docs) @Tool(name="search_case_law", required_consent=["search:legal"]) def search_case_law(query: str) -> list[dict]: nodes = case_index.as_retriever().retrieve(query) return [{"source": n.metadata["source"], "text": n.text} for n in nodes] onto = Onto(tools=[search_case_law]) ``` The agent now has a typed memory of the user and the case (ONTO's UPO), and a tool that retrieves from the LlamaIndex-managed corpus. ## The mistake people make The wrong pattern is treating LlamaIndex's vector store as your *agent memory*. Embedding "Alice is in UTC" and retrieving it later by similarity is the typed-memory-vs-embeddings mistake covered in [a separate article](/blog/typed-memory-vs-embeddings). A vector store should hold documents. An agent memory should hold facts. LlamaIndex is the right tool for the first; ONTO is the right tool for the second. If you use a vector store as agent memory, you'll find: - The agent re-derives the user's profile every session. - Facts that should have a single current value get duplicated and conflict. - "Is this true now" questions return confidently wrong answers. - Compliance audits become very hard, because the audit trail doesn't carry provenance per fact. The fix is layered storage: vector for content, typed for state. ## Where LlamaIndex Agents fit LlamaIndex now ships an Agents layer and a Workflows system. Both are good, and they're getting better fast. They're optimized for retrieval-driven workflows — the use case where the agent's job is mostly "retrieve and synthesize." If your agent's job is mostly retrieval, LlamaIndex Agents is a fine choice. If your agent's job is to *act* — book, refund, order, write — the retrieval-centric framing is a poor fit, and ONTO's primitives (Plan Mode, consent scopes, durable typed memory) map better to the problem. ## The pragmatic recommendation - Document Q&A: LlamaIndex alone. - Agent that acts on the world: ONTO, with LlamaIndex retrievers as tools when you need them. - Agent that lives in a regulated domain: ONTO. Add LlamaIndex retrievers for any document corpora you have to search. - Pure RAG over a fixed corpus: LlamaIndex. ONTO would be over-engineering. Use each for what it's good at. Stop treating "framework" as a single layer; treat it as a stack. --- *The [features page](/features) walks through ONTO's primitives in detail. The full [comparison page](/comparison) covers ONTO vs Claude / OpenAI / Google ADK SDKs.* --- ### ONTO vs Mem0: Choosing a Memory Layer for AI Agents Published: 2026-05-06 · 9 min · comparison URL: https://onto.neullabs.com/blog/onto-vs-mem0 Tags: comparison, Mem0, agent memory Mem0 has built one of the more interesting dedicated memory layers for LLMs in the past year. If you're shopping for "agent memory" as a standalone primitive, it's a serious option. ONTO takes a different shape: an agent SDK where typed memory (UPO + SMO) is one of the runtime primitives, alongside Plan Mode, Policy Guard, and the rest. You can't fully separate them. This post compares them as memory layers and is honest about when each is the right choice. ## What Mem0 ships - A standalone memory layer with REST, Python, and TS SDKs. - Automatic fact extraction from conversations. - Categorization and dedup. - Vector + graph hybrid storage. - Hosted cloud option and self-hosted option. Mem0 is essentially "add memory to your existing agent" as a service. You bring your own agent loop and call Mem0 for writes and reads. ## What ONTO ships - UPO + SMO typed memory with provenance, decay, confidence, and consent scopes. - Four-level extraction routing (session / profile / domain / tool). - Conflict detection and resolution via Plan Mode tasks. - Memory is one component of a broader agent runtime that also includes Plan Mode, Policy Guard, async writes, and cost meter. ONTO is "the whole agent SDK." Memory is built in but not separable from the rest. ## When Mem0 makes more sense - **You already have an agent framework you like.** If you're committed to LangChain or a custom loop and just need a memory store, Mem0 slots in cleanly. ONTO would mean migrating the whole agent runtime. - **Memory is your only structural concern.** No regulated industry pressure, no multi-tenant SaaS billing requirements, no Plan Mode requirement. - **You want a hosted memory service.** Mem0 has a managed cloud; ONTO is self-hosted (your Postgres, your Sled). ## When ONTO makes more sense - **You're starting fresh.** Building an agent SDK is a big bite. ONTO gives you the whole runtime with memory included, not bolted on. - **You need consent scopes on tool calls, not just memory writes.** Mem0 handles memory access control, but the consent question for *tools* sits at the agent layer. - **You need Plan Mode.** Mem0 doesn't gate execution; ONTO does. - **You need multi-tenant cost rollup.** ONTO's cost meter is built around tenants; Mem0 bills you for memory but you do the multi-tenant attribution yourself. - **You need on-prem with open-weight defaults.** Both can run on-prem; ONTO's defaults are open-weight, Mem0 typically pairs with OpenAI/Claude. ## The architectural difference The fundamental difference is *boundaries*. Mem0's boundary is "memory layer below the agent." Your agent code holds the loop; Mem0 holds the facts. This is a clean separation. It also means the agent layer makes its own choices about Plan Mode, consent scopes, and audit — Mem0 doesn't have an opinion. ONTO's boundary is "the runtime owns the agent loop, the memory, the consent enforcement, and the audit trail." This is a larger commitment but produces stronger guarantees. The same consent scope that gates a memory write gates a tool call. The audit log threads through all of it. Neither boundary is wrong. They're suited to different teams. ## Performance and scale Both can scale to millions of users / assertions when run on Postgres. Mem0's hybrid vector + graph store is good for the "find related memories" query; ONTO's typed-by-default approach is good for the "what is true about X" query. As covered in the [typed memory vs embeddings](/blog/typed-memory-vs-embeddings) piece, these are different shapes of question. For the typical agent workload, you want both: ONTO-style typed lookups for facts, vector retrieval for content. ONTO ships both internally; with Mem0 you'd pair it with a document index like LlamaIndex. ## The interop question Can you use Mem0 inside an ONTO agent? Yes — wrap Mem0's API as ONTO tools. The consent scope mapping is straightforward. This makes sense if you've already standardized on Mem0 in your org and want to add ONTO's runtime guarantees on top. Can you use ONTO's memory inside a Mem0-style architecture? Less clean today, because ONTO's memory APIs assume the broader runtime. We're working on better separability for a 1.0. ## A simple decision flow ``` Just need a memory layer to drop into your existing agent? → Mem0. Need consent scopes on tools, not just memory? → ONTO. Need Plan Mode? → ONTO. Want a hosted service to reduce ops burden? → Mem0 cloud. Need on-prem with open-weight defaults? → ONTO. Building from scratch and want fewer moving parts? → ONTO. ``` ## The bigger point The right question isn't "ONTO or Mem0." It's "what's the right boundary for memory in my system?" If you draw the line at "memory is a service my agent calls," Mem0 is well-suited. If you draw the line at "memory is one primitive in a runtime that also handles consent, Plan Mode, and audit," ONTO is well-suited. Both lines are defensible. Both teams are making the agent ecosystem better. Pick the one whose line matches yours. --- *The [features page](/features) covers ONTO's typed memory model in detail. The thesis post on [why agent memory is broken](/blog/why-agent-memory-is-broken) explains the underlying argument for typed memory in any agent system.* --- ### ONTO vs Letta (MemGPT): Two Approaches to Long-Term Agent Memory Published: 2026-05-04 · 9 min · comparison URL: https://onto.neullabs.com/blog/onto-vs-letta Tags: comparison, Letta, MemGPT, agent memory Letta (formerly MemGPT) is one of the most thoughtful research projects on agent memory in the past two years. The core idea: the agent itself manages its memory through tool calls. It writes to a "working memory" it can see directly, archives older entries to a searchable store, and pulls from the archive when needed. ONTO takes a different approach: typed memory with explicit extraction, managed by the runtime rather than the agent. This post compares the two architectures honestly. ## Letta's architecture in one paragraph The agent's prompt includes its working memory. It can call tools to write to working memory, search the archive, or move items between the two. Memory management is part of the agent's task — the model decides what to remember and what to archive. The system provides storage and search; the agent provides the policy. This is elegant. It means a smart model can build sophisticated memory behaviors without you writing schema. It also means the *model* is responsible for memory hygiene. ## ONTO's architecture in one paragraph The agent reads typed memory at the start of each turn and proposes typed assertions through extraction. The runtime — not the agent — writes them, with provenance, decay, consent scopes, and routing by extraction level. Conflicts surface as Plan Mode tasks. Memory management is a system concern, not an agent concern. This is more rigid. It also gives you an audit trail by construction, with provenance per fact, and lets you reason about "what does this agent know" outside the model. ## Where Letta's approach wins - **No schema required.** The agent figures out what to remember. You don't have to pre-define predicates. - **Memory behavior emerges from the model.** A smarter model produces smarter memory behavior, automatically. - **Conversational fluency.** Memory operations happen inline with reasoning, so the agent can incorporate memory state into its responses naturally. For use cases where you want a single agent personality with long-term coherence — companion chat, creative collaboration, ongoing research dialogue — Letta's approach is well-suited. ## Where ONTO's approach wins - **Auditability.** Every assertion has provenance and routing through Policy Guard. You can answer "what did the agent learn, when, and how" with structured data, not a transcript. - **Compliance.** Regulation tags (`[pii]`, `[phi]`, `[mnpi]`) ride on every assertion. Retention and decay policies are properties of the typed row. - **Multi-tenant scoping.** Domain-level extraction partitions memory by tenant. Cross-tenant reads refuse structurally. - **Cost discipline.** Memory management doesn't burn foreground tokens — extraction can run async or on a cheaper background model. For use cases where memory is a *system property* you have to answer for — regulated industries, multi-tenant SaaS, long-running personal assistants where the user owns their data — ONTO's approach maps better. ## The model-agnostic question Letta's approach assumes the model is good at managing memory. When the model is changed, the memory behavior changes. With strong models, this is great. With weaker open-weight models, the memory behavior degrades — the agent forgets to archive, accumulates clutter, or paraphrases facts. ONTO's approach is largely model-agnostic. Extraction quality varies by model, but the *structure* of memory (typed, routed, audited) doesn't change. This matters for ONTO's open-weight default: extraction prompts that work on a 20B parameter model produce the same memory shape as extraction prompts on a frontier model. ## The "what should the agent know" question A subtle difference. In Letta, the agent's working memory is what the agent decided to remember. In ONTO, the agent reads UPO at the start of each turn — a set of facts you can query independently. This means: - In Letta, "what does the agent know about user X" is a question you ask the agent (and trust the agent's answer). - In ONTO, "what does the system have stored for user X" is a Postgres query that returns the same answer regardless of which model runs. For systems where the answer to that question has to be reliable independent of the model's mood, ONTO's externalized memory is more robust. ## Can you use them together? Less obviously than ONTO + LangChain or ONTO + LlamaIndex. Letta and ONTO are both opinionated about how memory should work, and the opinions don't perfectly compose. The most practical pattern is to pick one as the spine. If you choose Letta, you get the agent-managed memory model. If you choose ONTO, you get the typed-graph model with audit. ## A simple choice flow ``` Need a typed, auditable, multi-tenant memory layer? → ONTO. Need model-managed memory with conversational fluency? → Letta. Need on-prem with open-weight defaults? → ONTO. Want memory behavior to scale with model quality automatically? → Letta. Need consent scopes on tools, not just memory? → ONTO. Building a companion / ongoing-dialogue product? → Letta. Building an agent that acts on the world? → ONTO. ``` ## A bigger question Both projects are placing real bets on how agent memory should work. Letta bets on the model: smarter models produce smarter memory. ONTO bets on the system: explicit structure gives you auditability and stability across models. The bets compose at the field level. If frontier models keep getting much smarter, Letta's approach will look more attractive — let the model do the work. If the production cost question keeps mattering and regulatory pressure keeps growing, ONTO's approach will look more attractive — make the structure explicit so the system can answer for the model. It's not yet obvious which way the field will go. The healthy answer is "both, for different use cases," and that's where we are today. --- *See [the typed memory thesis](/blog/why-agent-memory-is-broken) for the longer argument behind ONTO's approach, and the [features page](/features#typed-memory) for what UPO + SMO actually look like.* --- ### ONTO vs CrewAI: Multi-Agent Without the Black Box Published: 2026-05-02 · 8 min · comparison URL: https://onto.neullabs.com/blog/onto-vs-crewai Tags: comparison, CrewAI, multi-agent CrewAI broke through by making multi-agent orchestration feel approachable. Pick a "crew" of agents with roles, give them tasks, watch them collaborate. For a class of demos and prototypes it's been the right tool. ONTO has multi-agent support too, but it sits inside a different design. Subagents are scoped delegations; the parent agent is responsible for them; cost rolls up; depth is bounded. The starting shape in ONTO is one agent doing more, not many agents doing each less. This post compares them where they overlap. ## What CrewAI optimizes for - **Multi-agent collaboration as the central pattern.** Define personas (researcher, writer, critic), give them shared context, let them iterate. - **Approachability.** The mental model is intuitive: agents are like co-workers. - **Workflow flexibility.** Sequential, hierarchical, and consensus-style processes are built in. For prototypes and creative workflows, CrewAI's structure is genuinely useful. The "let three personas collaborate" pattern is hard to express elsewhere. ## What ONTO optimizes for - **Single-agent depth.** Most ONTO users build one agent and use the runtime's primitives (typed memory, Plan Mode, policy gates) to handle complexity. - **Subagents as scoped delegations.** A parent can spawn a child with a constrained consent set. The child is a worker, not a peer. - **Bounded delegation.** A depth budget prevents agents from infinitely delegating to subagents. - **Cost rollup.** A child subagent's token usage adds to the parent's cost meter row. ```python calendar = onto.subagent( name="calendar", tools=["mcp:gcal:list", "mcp:gcal:create"], consent_scopes=["calendar:read", "calendar:write"], ) result = onto.run_sync( "What's on my calendar this week, and what should I prep for?", user_id="alice", consent_scopes=["calendar:read", "memory:write"], ) # The parent agent calls the calendar subagent. The child's cost rolls up. ``` ## When CrewAI is the right shape - **Creative or research workflows.** Multiple personas critiquing each other genuinely helps quality on open-ended tasks. - **Demos and prototypes.** The "crew" abstraction is fun to show and easy to iterate on. - **You like the collaborative-personas mental model.** It's a coherent frame for thinking about agent design. ## When ONTO is the right shape - **Production deployments with audit requirements.** Multi-agent flows are hard to audit when every persona has its own state. ONTO's typed memory + consent scopes give you a single, queryable state. - **Regulated industries.** "Three agents discussed and decided" is not an audit story regulators accept. "Plan Mode produced these tasks; human approved these; the runtime executed under these scopes" is. - **Multi-tenant SaaS.** Cost rollup per tenant matters. Multi-agent CrewAI workflows can amplify cost in opaque ways; ONTO's cost meter splits it by parent vs subagent and by tenant. - **When you only need one agent.** Most production agent deployments are single-agent. Multi-agent is sometimes the cool answer to a problem that didn't need it. ## The multi-agent counterargument There's a real case for multi-agent that ONTO doesn't try to compete on hard. If your problem genuinely benefits from multiple personas critiquing each other — code review by a panel, scientific writing with a researcher / editor / critic — CrewAI's structure is well-suited. ONTO can do this with subagents, but the design point is "scoped worker," not "peer collaborator." If "peer collaborator" is the right model for your problem, CrewAI fits better. ## The cost / latency trap A practical caution about multi-agent in general (true for CrewAI, true for any framework that makes it easy): every agent in the loop is an LLM call. Five agents discussing means five LLM calls per turn instead of one. The token bill multiplies, latency multiplies, and the marginal quality lift is often small. ONTO's design implicitly discourages this — subagents are deliberate delegations with a depth budget, not casual collaborators. Whether that's a feature or a limitation depends on what you're building. ## A simple decision flow ``` Multi-agent collaboration is the *point* of your product? → CrewAI. You need typed memory + audit + consent across agents? → ONTO. You're building a single-agent product that occasionally delegates to a specialized worker? → ONTO (with subagents). You want demos that show multiple personas working? → CrewAI. ``` ## A note on direction Both projects are evolving. CrewAI is adding more production controls (memory, audit). ONTO is adding richer multi-agent ergonomics. Don't pick based on today's feature gap if it's a feature both are working toward. Pick based on the *shape* of your product. Multi-agent at the center → CrewAI. Single agent with depth → ONTO. The shape is harder to change than the features. --- *The [features page](/features) covers ONTO's subagent model and consent scopes. The [comparison page](/comparison) has the full matrix.* --- ### ONTO vs AutoGen: Human-in-the-Loop, Done Right Published: 2026-04-30 · 8 min · comparison URL: https://onto.neullabs.com/blog/onto-vs-autogen Tags: comparison, AutoGen, human-in-the-loop AutoGen pioneered the conversational multi-agent paradigm — agents talking to each other and to humans through a chat interface. Its `UserProxyAgent` pattern made human-in-the-loop a first-class concept in 2023, before most other frameworks had a real story for it. ONTO and AutoGen both take human-in-the-loop seriously. They take it differently. This post compares the approaches. ## How AutoGen does human-in-the-loop The central abstraction is `UserProxyAgent` — an agent that represents the human. In a multi-agent conversation, the UserProxyAgent can be configured to prompt the human at certain steps. The agents around it can ask it for input, treat it as a tool, or hand control to it. ```python # Conceptual AutoGen pattern user_proxy = UserProxyAgent(name="user", human_input_mode="ALWAYS") assistant = AssistantAgent(name="assistant", llm_config=...) user_proxy.initiate_chat(assistant, message="Plan a trip to Lyon") # AutoGen runs the conversation, prompting the human via user_proxy. ``` This is conversationally elegant. The human is "in" the conversation, not "outside" it. The pattern fits well when collaboration with the human is iterative and frequent. ## How ONTO does human-in-the-loop ONTO uses Plan Mode + Execute Mode. Plan runs produce a typed list of proposed tasks; the human reviews them; Execute runs (with execute consent scopes) actually do work. ```python # ONTO pattern plan = onto.run_sync("Plan a trip to Lyon.", mode="plan_only", consent_scopes=["tasks:write"]) # Human approves tasks via your UI. for task in plan.tasks: if approved(task): onto.execute_task(task, consent_scopes=["tools:web.fetch"]) ``` The human reviews structured data, not a transcript. Approval is per-task. The execution step is a separate run with separate scopes. ## Where AutoGen's approach wins - **Conversational workflows.** When the human's interactions are short, frequent, and feel like chat ("can you also book the train?"), AutoGen's structure is natural. - **Multi-agent collaboration with a human peer.** "Three agents and a human collaborating" is a coherent shape in AutoGen. - **AutoGen Studio.** The UI layer for building and inspecting flows is mature. ## Where ONTO's approach wins - **Auditable approval.** Plan Mode produces typed Task objects with arguments, costs, and rationales. The approval decision is a row, not a turn in a chat. For regulated workflows, this is decisive. - **Structural guarantees.** ONTO's Plan Mode is enforced by consent scopes. The runtime refuses execute tools in plan mode. AutoGen's UserProxyAgent relies on the agents *honoring* the convention of asking the human — there's no runtime gate. - **Asynchronous review.** ONTO's Plan / Execute split is naturally async. The human can review a plan an hour later, in a queue, in a different UI. AutoGen's conversational model is fundamentally synchronous. - **Bulk approval workflows.** "Here are 50 proposed tasks across 50 customers, approve / reject / edit in bulk" maps cleanly to ONTO's Task primitive. Conversational HITL doesn't. ## The structural vs conventional split The core philosophical difference: AutoGen's human-in-the-loop is *conventional* — agents are instructed to ask the human and they do. ONTO's is *structural* — the runtime refuses certain tools unless execute scopes are granted, regardless of what the LLM tries to do. For some use cases (creative collaboration, exploratory work, anything where speed of iteration matters more than guarantees) the conventional approach is better. The friction of a runtime gate gets in the way. For other use cases (regulated workflows, high-stakes decisions, anything where "the agent asked the human" needs to be provable) the structural approach is required. Without it, your audit story is "we instructed the model to ask, and we hope it did." ## Memory and state A quick aside since this matters for choosing. AutoGen's state model is per-conversation; long-running memory across conversations needs a bolt-on. ONTO ships UPO + SMO as the runtime's memory layer. For agents that need to remember the user across sessions, ONTO has the better story by default. ## Open-weight and on-prem Both can run with open-weight models and on-prem. AutoGen is provider-agnostic and works with any chat-completion-style endpoint. ONTO defaults to open-weight and has Helm + Postgres deployment recipes; AutoGen has fewer turnkey patterns for production on-prem. ## A decision flow ``` Conversational HITL where the human is a peer? → AutoGen. Structured approval over typed tasks? → ONTO. Multi-agent prototype with a human in the chat? → AutoGen. Production agent with audit + memory + on-prem? → ONTO. Want a UI-builder for the flow? → AutoGen Studio. Want a runtime that refuses tools without consent? → ONTO. ``` ## A summary AutoGen and ONTO both take human involvement seriously; they just bet differently on what shape the involvement should take. AutoGen bets on chat. ONTO bets on structured tasks. For experimental, conversational, and creative work, AutoGen's bet pays off. For production, regulated, and audit-bound work, ONTO's bet pays off. It's not a "which is better" question. It's a "which shape matches your workflow" question. --- *The [Plan Mode explainer](/blog/plan-mode-explained) covers ONTO's approach in depth. The [comparison page](/comparison) has the full matrix.* --- ### Building HIPAA-Ready AI Agents: A Compliance Engineer's Checklist Published: 2026-04-28 · 13 min · industry URL: https://onto.neullabs.com/blog/hipaa-ready-ai-agents Tags: healthcare, HIPAA, compliance import Callout from '@components/article/Callout.astro'; Taking an AI agent into a HIPAA-covered environment is not impossible. It is exacting. The OCR (Office for Civil Rights) doesn't read your code; they read your audit trail, your access controls, and your incident logs. The job of the engineering team is to produce those artifacts by construction, not by retrofit. This is a practical checklist. We'll go through what an AI agent in a HIPAA environment has to do, with examples from ONTO where they apply. ## 1. Decide where PHI lives PHI lives in three places in an agent system: - **The prompt.** Patient name, condition, history in the model's input. - **The memory.** Anything the agent retains across sessions. - **The logs.** Inputs, outputs, intermediate tool calls. For each, you need a story. Not a story like "we encrypt at rest" (that's the table stakes); a story like "this specific data type has this specific retention, lives in this specific store, accessible only to these specific scopes." In ONTO, the primitive for this is regulation tags. Every Assertion carries tags (`[phi]`, `[pii]`, `[mnpi]`, custom). Tags drive routing, retention, decay, and which consent scopes can read or write. A typical PHI memory write looks like: ```python onto.memory.add_assertion( subject="patient_a912", predicate="reported_symptom", object="chest_pain", tags=["phi"], consent_scopes=["patient:intake"], provenance=Source(message_id="m_42", session_id="s_2026_04_28"), ) ``` The `[phi]` tag triggers your retention policy. The consent scope governs who can read it later. ## 2. Sign a BAA with every vendor in the data path If a hosted service sees PHI, you need a Business Associate Agreement with the vendor. The path of PHI is: - The LLM provider (Claude / GPT) — needs a BAA, only available on enterprise tiers. - The vector store (if PHI is embedded) — needs a BAA. - The logging / observability service — needs a BAA. - The hosting platform — needs a BAA. This is the lift that pushes many health systems to on-prem. With ONTO's default deployment (local Ollama + Postgres in your VPC), there are no external vendors in the PHI path. The BAA list shortens to "internal." ## 3. Role-based access at the call site HIPAA distinguishes between users with different access levels — an intake nurse, a treating clinician, a billing analyst. A single agent serving all three needs to behave differently for each. The pattern is per-call consent scopes: ```python # Intake nurse: can write patient data. onto.run_sync( intake_message, user_id="patient_a912", consent_scopes=["memory:write", "patient:intake"], ) # Clinician: can read patient data and place orders. onto.run_sync( "Summarize the case and propose orders.", user_id="patient_a912", consent_scopes=["memory:read", "patient:order", "audit:write"], mode="plan_only", ) ``` The Policy Guard refuses any tool whose `required_consent` isn't covered. If the clinician tries to call `remember()` (which requires `memory:write`), the runtime refuses. The audit log records the refusal — which is itself evidence that the access control is working. ## 4. Plan Mode before any irreversible clinical action Some agent outputs are reversible (a draft note). Some are not (a placed order, a scheduled procedure, a shared record). For the irreversible ones, an autonomous loop is unsafe — by the time the human reviews, the action has already happened. Plan Mode separates proposal from execution. The agent proposes orders in `mode="plan_only"`. The clinician reviews the typed Task list. They approve task by task. A second run, with execute scopes, actually places the orders. ```python plan = onto.run_sync( "Propose next steps for patient_a912.", user_id="patient_a912", mode="plan_only", consent_scopes=["memory:read", "tasks:write"], ) # clinician_ui.show(plan.tasks) # clinician approves... for task in approved_tasks: onto.execute_task(task, consent_scopes=["patient:order", "audit:write"]) ``` The audit log distinguishes Plan rows from Execute rows by `plan_id`. The clinician's approval is recorded. ## 5. Append-only audit log at the database layer The HIPAA Security Rule requires recording specific events: access attempts, modifications, and access by user. The right place to enforce append-only is the database, not the application. ```sql CREATE OR REPLACE FUNCTION reject_audit_mutation() RETURNS trigger LANGUAGE plpgsql AS $$ BEGIN RAISE EXCEPTION 'audit_log is append-only'; END; $$; CREATE TRIGGER audit_no_update BEFORE UPDATE OR DELETE ON audit_log FOR EACH ROW EXECUTE FUNCTION reject_audit_mutation(); ``` If the application has a bug that tries to UPDATE the audit log, the database refuses. Compliance properties shouldn't depend on the application being correct. ## 6. Encrypt at rest, encrypt in transit, rotate keys Standard but easy to forget: - TLS for all in-network traffic, including between ONTO and Postgres. - Postgres `pgcrypto` for column-level encryption of PHI-tagged columns, or full-disk encryption on the volume. - KMS-managed keys with rotation. Don't roll your own. - Audit log columns that contain PHI excerpts: encrypted. ONTO's storage layer accepts encrypted columns; configure your Postgres deployment accordingly. ## 7. Retention and disposal policies HIPAA requires retention of certain records for specific durations and disposal of PHI when the retention period ends. The agent's memory layer needs: - Per-tag retention policy (`[phi]` data retained N years from last access). - Automated purge or anonymization at expiry. - A documented disposal procedure. ONTO's decay curves attach to assertions. A nightly job purges expired `[phi]` rows; the audit log records the purges. ## 8. Anonymize for evals and training Your eval set may need to look like real PHI for the model to behave realistically. It must not actually be PHI. Three options, increasing in rigor: - **Synthetic data**: generate plausible cases that aren't real patients. Hardest to get right; safest. - **De-identification**: strip the 18 HIPAA identifiers from real cases. Still PHI-derived; lower risk. - **k-anonymization**: ensure no record is identifiable to fewer than k patients. Stronger but more complex. ONTO's eval harness reads test cases from a separate directory you can keep on a non-PHI store. Don't run your evals against your production memory. ## 9. Incident response Every covered entity has an incident response plan. Your AI agent's piece: - **Detect.** OpenTelemetry traces with PHI-tagged tool calls. Alerts when consent scope mismatches spike (indicates someone is probing). - **Contain.** Disable specific consent scopes globally; pause the agent. Have a kill switch. - **Investigate.** Audit log query by user_id, session_id, and tool. Plan / Execute rows linked by plan_id. The whole story is reconstructible. - **Notify.** OCR has timelines. Don't wait until you find the entire scope; report on the OCR-required schedule. ## 10. Documentation, documentation, documentation The OCR auditor reads your policies before they look at your code. Have: - A data flow diagram showing every place PHI lives. - A consent scope matrix (which role gets which scopes). - A retention policy per tag. - An incident response runbook. - A list of every system in the PHI path with the BAA on file. This is paperwork that engineers don't love, and it's the difference between a green audit and a yellow one. ## What ONTO gives you for free To be concrete, here's what the ONTO primitives map to on the HIPAA checklist: - Regulation tags on memory → §164.308(a)(1) risk management and §164.312(b) audit controls - Consent scopes → §164.308(a)(4) access management - Plan Mode → §164.310(a) physical safeguards (analogue: gating high-impact actions) - Append-only audit log → §164.312(b) - On-prem deployment story → minimizes the BAA list It does not give you compliance. Compliance is your policies, your training, your physical safeguards, your incident response. It gives you the technical primitives that produce the evidence those policies require. Imagine an OCR audit tomorrow morning. Can you produce, for any patient, the list of every agent interaction, the scope set each interaction carried, the approval log for any clinical action, and the data flow diagram showing where the data lives? If yes, your AI agent is HIPAA-ready. If no, this checklist is the to-do list. --- *See the [healthcare industry page](/industries/healthcare) for the runnable triage example, or the [features page](/features#policy-guard) for the consent scope details.* --- ### Financial Services AI Agents: Audit Trails That Pass SOC 2 and FINRA Published: 2026-04-26 · 12 min · industry URL: https://onto.neullabs.com/blog/financial-services-ai-agents Tags: financial services, FINRA, SOC 2, compliance Financial services AI deployments come with three control regimes that interact: SOC 2 (security and availability), FINRA (broker-dealer communications, books-and-records), and increasingly the SEC's AI-specific guidance. Plus GDPR if you operate in Europe and MiFID II if you operate in the UK or EU markets. The good news is that the controls these regimes ask for are largely the same controls — audit trail, access management, retention, incident response. The bad news is that you have to demonstrate them to multiple bodies under different formats. This is a practical guide. ## What's actually required The high-impact controls for AI agents in financial services: **SOC 2 Trust Services Criteria (especially CC6, CC7, CC8):** - Logical access controls with role separation - Change management with audit - Monitoring and detection of anomalous activity - Incident response with documented timelines **FINRA Rule 4511 (books and records) and Rule 3110 (supervision):** - Retention of communications with customers - Supervisory review of communications - Documented review procedures **FINRA Rule 3210 / NASD 3010:** - Restrictions on outside business activity, including AI-generated communications that might constitute advice **SEC marketing rule (Rule 206(4)-1) for RIAs:** - AI-generated client communications can be advertisements; same disclosure / fair-and-balanced rules apply **MNPI controls:** - Material non-public information must not flow into models or systems that could leak it ## The agent control surface Map those to what an agent system has to provide: 1. **Per-user, per-role access controls** with enforcement at the runtime, not just the UI 2. **Append-only audit log** with retention longer than your longest applicable window 3. **MNPI tagging** that prevents specific memory writes from being visible to specific scopes 4. **Plan Mode** for any communication that constitutes advice or solicitation 5. **Supervisory review queue** with approval / rejection tied to a named principal 6. **Model version pinning** with a documented change procedure ONTO's primitives map cleanly: ```python # Tagged memory write — MNPI. onto.memory.add_assertion( subject="client_4421", predicate="position_request", object="long XYZ Corp", tags=["mnpi"], consent_scopes=["advisor:write"], ) # A general-access tool refuses to read [mnpi] memory. @Tool( name="generate_marketing_email", required_consent=["marketing:write"], excluded_tags=["mnpi", "phi", "pii"], # ← cannot read these. ) def generate_marketing_email(client_id: str) -> str: ... ``` The `excluded_tags` is the structural primitive that prevents MNPI from flowing into a marketing context — the same memory store, different access rules, enforced at the runtime. ## The Plan Mode case for FINRA Financial communications to clients are supervisable. Under Rule 3110, a registered principal has to review communications before they go out. A fully autonomous AI agent that sends client emails is a supervision gap. Plan Mode is the structural answer: ```python # Advisor proposes a portfolio recommendation. plan = onto.run_sync( "Recommend rebalance for client_4421.", user_id="client_4421", consent_scopes=["advisor:read", "tasks:write"], mode="plan_only", ) # The plan goes to a supervisor's queue. supervisor.review(plan) # After approval, execute with appropriate scopes. if approved: onto.execute_task( plan.tasks["send_recommendation_email"], consent_scopes=["client:communicate", "audit:write"], ) ``` The supervisor's approval is a row in the audit log linked to the plan_id. Books and records satisfied; supervision satisfied. ## The on-prem decision Several factors push financial firms toward on-prem AI: - **MNPI controls**: easier to demonstrate that MNPI didn't leave the network if there is no external API call. - **Audit complexity**: every external vendor adds a sub-processor to your SOC 2 scope. - **Data residency**: GDPR data-localization rules; MiFID II conditions. - **Predictable cost**: hosted models bill by token; on-prem GPU cost is fixed. ONTO's default deployment runs entirely on your infrastructure: local Ollama (or vLLM) + Postgres + pgvector + Helm chart. No external API call required. ## Model version pinning For SOC 2 change management, the version of the LLM you call is configuration that has to be managed. Hosted model providers often update their models silently; this is incompatible with strict change management. Two paths: - **Pin to a specific hosted model version** with vendor commitment to retain it. Anthropic and OpenAI both offer this on enterprise tiers. - **Self-host open-weight models.** The version is whatever you deploy. ONTO supports both — it's an env var. ## The eval discipline Most financial firms require pre-production testing for any AI change. The right shape: - An eval set of representative inputs (anonymized, never real client data). - Expected outputs or properties (e.g., "must not give a buy / sell recommendation without disclaimer"). - CI gate that fails if regressions exceed threshold. - Change-approval workflow that ties to the eval results. ONTO's eval harness reads TOML/JSON test cases and exits non-zero on failure. Integrate it into your CI; gate releases on it. ## What you can't do (yet) Even with the right primitives, some patterns remain hard: - **Real-time MNPI screening**: detecting in-flight that user input contains MNPI requires careful prompting + post-hoc review. No magic. - **Generative tax advice**: most firms forbid this entirely; not an SDK question. - **Cross-border data movement**: requires careful network architecture; the SDK can pin storage to a region but you have to enforce the rest. These are deployment patterns, not code patterns. The SDK supports them; the policy enforces them. ## A minimal compliance kit The artifacts you should be able to produce on demand: 1. Data flow diagram (what data is where, what scopes can touch it) 2. Consent scope matrix (which role has which scopes; signed off by compliance) 3. Audit log retention policy (how long for each tag) 4. Eval set + last run results 5. Model version inventory + change log 6. Incident response runbook with AI-specific scenarios 7. Sub-processor list (every external vendor in the data path) 8. BAA / DPA list (signed agreements with each) ONTO gives you the technical primitives that the artifacts reference. The artifacts themselves are your work. ## The financial-services bottom line The right way to ship AI in a regulated financial workflow is: - Default to on-prem with open-weight models. - Tag memory by regulation class; gate tools by tag. - Plan Mode every client-facing or trade-affecting communication. - Pin model versions and gate changes through CI. - Treat the audit trail as a first-class deliverable, not an afterthought. If your stack supports these by construction, your compliance burden becomes manageable. If they're bolt-ons, every audit becomes an engineering project. --- *The [features page](/features) covers consent scopes and Plan Mode. The thesis post on [stateless agents](/blog/stateless-agents-stateful-humans) explains why structural separation matters more than conventions.* --- ### Legal AI Agents Without the Confidentiality Headache Published: 2026-04-24 · 10 min · industry URL: https://onto.neullabs.com/blog/legal-ai-agents Tags: legal, confidentiality, compliance import Callout from '@components/article/Callout.astro'; Legal AI has four hard problems that don't have the same shape as healthcare or financial services: 1. **Attorney-client privilege** is broken by disclosure to third parties. 2. **Conflicts of interest** require not even *appearing* to use one client's data on another. 3. **Unauthorized practice of law (UPL)** rules forbid providing legal advice without a license. 4. **Confidentiality obligations** under state bar rules are independent of privilege and apply broadly. This post is about how to design an AI agent system that respects these by construction. ## The privilege problem Privilege protects communications between attorney and client from compelled disclosure. Disclosure to a third party — including a hosted AI vendor — can constitute a waiver, depending on jurisdiction and circumstances. The conservative path is to keep all privileged material inside the firm's network. That eliminates the "is sharing with vendor X a waiver?" question by not sharing. For ONTO, this maps to the on-prem deployment story: open-weight models running locally (Ollama / vLLM), Postgres + pgvector for typed memory, your VPC, your KMS. No third-party API call touches privileged matter. If you do want hosted models, do it under a vendor agreement that addresses confidentiality, with client consent where required. The ABA Formal Opinion 512 (2024) gives some structure here. But the cleanest answer is "no hosted vendor sees privileged material." ## The conflicts problem A law firm can't represent two clients with adverse interests on the same matter. Discovering a conflict is a deliberate process — every new matter is run against the conflicts database. An AI agent that's processed Client A's matter cannot, even theoretically, contaminate Client B's matter. The structural answer is domain-scoped memory. ONTO's four-level extraction includes a `domain` scope: ```python # Matter intake for Client A. onto.run_sync( intake_text, user_id="attorney_williams", domain_id="matter_4421_clientA", consent_scopes=["memory:write", "matter:read", "matter:write"], extract=True, ) # Different matter, different client. onto.run_sync( different_intake, user_id="attorney_williams", domain_id="matter_5512_clientB", consent_scopes=["memory:write", "matter:read", "matter:write"], extract=True, ) ``` The runtime stores Client A's typed assertions in a separate partition from Client B's. A read query without the right `domain_id` returns nothing. A tool that needs to cross domains has to declare `required_consent=["matter:cross-domain"]` — which you don't grant to general matter work. For the conflicts database itself, that's a separate system (matter intake software, with its own controls). The AI agent's job is not to *be* the conflicts check; it's to not leak across matters in normal use. "The agent will not mention Client A's matter while working on Client B's" is a property best enforced by the runtime, not the prompt. Domain-scoped extraction makes the cross-matter read structurally impossible. ## The UPL problem In most US states, providing legal advice without a license is unauthorized practice of law. An AI agent that talks to non-lawyer end users and produces legal advice can be UPL. The right pattern is Plan Mode: the agent proposes; a licensed attorney reviews; only after attorney approval does the output go to the client. ```python # AI drafts a response to a client's question. draft = onto.run_sync( "Draft a response to client question 14.", user_id="matter_4421", consent_scopes=["matter:read", "tasks:write"], mode="plan_only", ) # Attorney reviews and edits. approved_draft = attorney_review(draft) # Approved draft goes out. if approved_draft: onto.execute_task( approved_draft, consent_scopes=["matter:read", "client:communicate", "audit:write"], ) ``` The attorney remains the legal decision-maker. The AI accelerates the work; it doesn't replace the judgment. The audit log captures the attorney's approval — useful for both UPL defense and malpractice insurance. ## The confidentiality obligation State bar rules treat *all* client information as confidential, not just privileged communications. This is broader than privilege. Disclosure of metadata, of the fact of representation, of even the existence of the matter can be a confidentiality breach. For an AI system this means: - **No logging of client identifiers in third-party telemetry.** Plausible / Mixpanel / etc. — opaque the data. - **No model training on client data without consent.** ONTO doesn't train; if you use a hosted model, the vendor agreement must prohibit training. - **No transcript leak through error messages.** Logs that contain prompt content must be in a controlled store with the same access rules as the matter. ONTO's telemetry exports per-run metadata (model_id, scopes, tool calls) but not prompt content by default. Configure the OpenTelemetry exporter accordingly. ## The retention question Different states have different rules on client file retention. Some require 5 years; some 7; some specific to matter type. The retention applies to AI-generated work product if it's part of the file. ONTO's decay curves attach to assertions. A retention policy per `domain_id` purges expired assertions on schedule. The audit log records purges so you can demonstrate compliance. ## A minimal legal AI checklist 1. **On-prem deployment** with open-weight models for privileged matter. 2. **Domain-scoped memory** partitioned by matter. 3. **Plan Mode** for any output that goes to a client. 4. **Attorney approval** captured in the audit log per output. 5. **Confidentiality-aware logging** (no client content in third-party telemetry). 6. **Retention policy** per matter, with documented purge procedure. 7. **Vendor agreements** with appropriate confidentiality terms if any hosted service is in the path. If those seven are in place, your legal AI deployment is in defensible shape. ## What ONTO doesn't do To be honest: - **Conflicts checking is a separate system.** ONTO partitions memory to prevent accidental leak; it does not screen new matters against the conflicts database. That's still your matter management software. - **Privilege log generation is a separate workflow.** ONTO records what the agent saw; constructing a privilege log for discovery is a manual or semi-automated process on top. - **State-by-state UPL nuance is your problem.** ONTO supports the supervised-attorney pattern; whether that pattern satisfies *your* state's specific UPL rules is a legal question, not a technical one. The job of the SDK is to give you the primitives that make the technical part defensible. The legal part remains a human judgment. ## The summary Privilege, confidentiality, conflicts, and UPL aren't insurmountable. They're solved structurally — domain scoping, Plan Mode, attorney supervision, retention policies — rather than prompt-engineered around. If your legal AI deployment fights these problems at the prompt layer, you've built a fragile system. If it solves them at the runtime layer, you've built one that can survive an audit and a malpractice claim. --- *See [the comparison page](/comparison) for how other agent SDKs handle this. The [features page](/features#policy-guard) covers consent scopes in detail.* --- ### AI Customer Support That Doesn't Hallucinate Refunds Published: 2026-04-22 · 9 min · industry URL: https://onto.neullabs.com/blog/customer-support-no-hallucinated-refunds Tags: customer support, multi-tenant, Plan Mode The standard customer support AI pattern fails in two specific ways. First, the agent forgets the customer between sessions. Second, when it does take action — like issuing a refund — it does so autonomously, and sometimes wrongly. This post walks through a customer support agent architecture that fixes both with ONTO's runtime primitives. It's based on the customer-support example in the framework repo. ## The two failure modes **Forgetting the customer.** Conversation buffer memory dies with the session. Vector retrieval finds "similar past conversations" but not "what this customer's tier is right now." Result: the agent re-derives the customer's account state from scratch each time, badly. **Autonomous refunds.** A customer asks for a refund. The agent looks at the case, decides "yes, refund $500," and calls the refund tool. There was no policy check. Maybe the customer asked too many times this quarter. Maybe their account is flagged. Maybe the model just hallucinated the right thing to do. By the time anyone notices, the refund is processed. Both failures share a root cause: the agent has authority it shouldn't have. The fix is structural — separate "agent proposes" from "system decides." ## The agent loop, sketched ```ts import { Onto, Models, BuiltinTools } from "@onto/core"; const onto = new Onto({ model: Models.GPT_OSS_20B, builtinTools: BuiltinTools.STANDARD, }); async function handleMessage(tenantId: string, accountId: string, message: string) { // 1. Load the account's durable UPO. Extract any new facts from the message. await onto.run(message, { userId: accountId, tenantId, consentScopes: ["memory:write", "support:read"], extract: true, }); // 2. Plan: propose actions (refund, escalation, knowledge base reply, ...). const plan = await onto.run("Plan response to this customer.", { userId: accountId, tenantId, mode: "plan_only", consentScopes: ["tasks:write", "support:read"], }); // 3. Apply policy. The LLM doesn't decide; the policy function does. const account = await loadAccount(accountId); const decision = policy.classify(plan, account); if (decision === "auto-approve") { await onto.run("Execute the approved plan.", { userId: accountId, tenantId, consentScopes: ["support:refund", "support:notify"], }); } else { await escalateToHuman(plan, account); } } ``` ## The memory layer Each account has a UPO partition (`tenant_id + user_id`). At conversation start, the agent loads the account's typed facts: ``` account.tier = "gold" account.signup_year = 2023 account.escalation_history = ["billing_2025_03", "shipping_2025_11"] account.preferred_channel = "email" account.refunds_ytd = 1 ``` These are typed assertions, not chat history. They're queryable with one Postgres lookup. The agent's prompt is a few hundred tokens — not 50k of conversation buffer. When the customer says something new ("I never received the package"), `extract=True` writes a typed assertion: `(account_42, claimed_non_delivery, package_X44, confidence=0.85)`. Next session, this is part of the account state. ## The policy function Policy lives outside the LLM. It's plain TypeScript: ```ts function classify(plan: Plan, account: Account): "auto-approve" | "human-review" { const refund = plan.tasks.find(t => t.tool === "issue_refund"); if (!refund) return "auto-approve"; // Nothing to gate. // Auto-approval gate: gold tier, low value, not already refunded this quarter. const value = refund.args.amount; const recentRefunds = account.refunds_ytd; if (account.tier === "gold" && value < 50 && recentRefunds < 3) { return "auto-approve"; } if (account.tier !== "gold" && value < 10 && recentRefunds < 1) { return "auto-approve"; } return "human-review"; } ``` The model doesn't know this logic. The model proposes; the policy decides. Two consequences: 1. You can change the policy without retraining or re-prompting the model. 2. The audit trail records the policy decision separately from the model proposal. When something goes wrong, you know whether it was a bad proposal or a bad policy — they're different fixes. ## The multi-tenant story In SaaS support, the worst-case failure is cross-tenant data leak. The agent talks to a Tenant A user; the model's context somehow includes a snippet from Tenant B. Catastrophic. ONTO's structural fix: every UPO read and write carries `tenant_id`. Cross-tenant reads refuse with a structured error — no silent fall-through. The Policy Guard checks tenant_id on every tool call. For shared resources (like a knowledge base that all tenants can read), declare a tool with `required_consent=["kb:read"]` and grant it broadly. For per-tenant resources, the tenant_id must match. This means you can audit "did any cross-tenant read happen this month" with a simple query against the runtime's denial log. The answer is "no" by construction unless an explicit grant was made. ## The cost meter Multi-tenant SaaS bills the customer for the usage. The vendor bill — OpenAI / Anthropic / Ollama Cloud — is gross. You need to attribute it per tenant. ```ts const cost = await onto.cost({ tenantId: "acme-corp" }); // { // foreground: { tokens: 1_234_567, dollars: 4.92 }, // background: { tokens: 8_456_021, dollars: 1.08 }, // cheaper model // } ``` Foreground vs background split matters: foreground is the user-facing turn; background is summarization, conflict detection, and consolidation. They can use different models. They're billed separately. This is how you turn an AI vendor bill into a tenant invoice. ## What still requires human review Three categories where Plan Mode should never auto-approve: 1. **High-dollar refunds.** Threshold depends on your business; choose conservatively. 2. **Account terminations.** Always human-reviewed. 3. **Anything legal-adjacent.** Disputes, complaints, regulatory matters — escalate. For these, Plan Mode produces tasks; the human review UI shows them; an authorized agent approves. The audit log records the human approval. ## What ships for free Building this from scratch on a generic LLM API would take months. With ONTO: - Multi-tenant memory: built in - Plan Mode with typed tasks: built in - Policy Guard with consent scopes: built in - Cost meter with foreground/background split: built in - Audit log: built in - Background summarization on a cheaper model: built in You write the policy function and the human review UI. Everything else is configuration. ## A summary you can paste in a design doc > The support agent runs in two modes. In Plan Mode, it loads typed per-account memory, extracts new facts from the customer's message, and proposes a list of tasks (reply, refund, escalation). A policy function — outside the LLM — classifies each task as auto-approve or human-review. Auto-approve tasks fire in Execute Mode with appropriate consent scopes. Human-review tasks go to a queue. The agent has no authority to take action without classification; it has authority to propose actions for classification. That's the architecture. It generalizes to every "agent acts on the world" use case in support. --- *See the [customer support industry page](/industries/customer-support) for a deeper writeup or the [multi-tenant memory article](/blog/multi-tenant-agent-memory) for the partitioning details.* --- ### Building a Personal AI Assistant That Actually Remembers You Published: 2026-04-20 · 9 min · industry URL: https://onto.neullabs.com/blog/personal-ai-that-remembers Tags: personal AI, agent memory, async writes The hardest problem in personal AI isn't quality. It's continuity. After three months of using an assistant, you want it to know you. Not in the "model has been fine-tuned on me" sense — in the "it remembers what we worked out in March" sense. Standard chat agents can't deliver this. Conversation buffer forgets when the window rolls. Vector retrieval surfaces "similar past content" but not "the thing you decided." The user re-explains the same context every month. This post sketches the architecture for a personal assistant that doesn't. ## What the assistant has to know A useful personal assistant knows things in four categories: 1. **Identity facts.** Name, timezone, employer, location, primary calendar. 2. **Preferences.** "Mornings, not evenings." "Reply to emails the same day." "Don't book meetings on Fridays." 3. **Project context.** Current work-in-progress, key people involved, deadlines. 4. **Relationship facts.** Who is who, how you know them, last interaction. These aren't chat history. They're typed facts. And they have different lifecycles — identity facts decay slowly, project context turns over weekly, relationship facts age but don't disappear. ONTO's UPO + SMO split is built for this: ```python # Identity — durable, profile-level. profile = onto.memory.get_profile("you") # profile.name, profile.timezone, profile.employer # Preferences — durable, profile-level. # profile.prefers, profile.meeting_window # Project — domain-scoped to the current work. projects = onto.memory.query_domain("you", domain="work") # returns active projects with their context # Relationships — domain-scoped to a "contacts" graph. contacts = onto.memory.query_domain("you", domain="contacts") ``` Each query is O(1) lookup against typed storage. The model's prompt loads what's relevant, no more. ## The async write pattern For personal AI, latency is everything. If the assistant takes three seconds to reply because it's writing memory synchronously, it feels sluggish. ONTO's `extract_async=True` flag spawns extraction writes into a background task: ```python result = onto.run_sync( user_message, user_id="you", consent_scopes=["memory:write", "calendar:read"], extract=True, extract_async=True, ) ``` The user gets the reply immediately. The memory writes happen behind the scenes. The next turn loads them when it starts — and because storage is fast, they're ready in time. Combined with `background_memory=True`, session summarization and conflict detection also move to a background task on a cheaper model: ```python onto = Onto( model=Models.GPT_OSS_20B, # foreground background_memory=True, # background uses a cheaper model by default background_model=Models.LLAMA_3_8B, # explicit override ) ``` The foreground bill stays bounded; the background bill is small per task because the model is small. ## The four-level extraction When the user says something, where does it go? The four-level routing does the work: ``` User: "Move my Wednesday standup to 10am from now on." → Extracted assertions: session: (you, current_request, reschedule_standup, conf=0.95) profile: (you, prefers_standup_time, "10:00", conf=0.85) domain[work]: (standup_meeting, time, "10:00", conf=0.9, supersedes=prior) tool: (calendar, last_reschedule_target, "standup", conf=0.6) ``` The LLM labels each candidate assertion with a level. The runtime writes them to the appropriate stores. Next month, when you ask "what time is my standup," the answer comes from the work-domain assertion, not a buffer search. ## The conflict resolution People change their minds. The user might say "I prefer mornings" in March and "I want to switch to afternoons" in July. Both are honest at the time. The agent has to handle the contradiction. ONTO's conflict detection: when two assertions point at the same `(subject, predicate)` with different objects, the system creates a Plan Mode task: ``` Task: "User previously preferred mornings (March 2026, confidence 0.85). Now states preference for afternoons (July 2026, confidence 0.92). Supersede?" ``` The user can resolve it ("yes, supersede"), the agent can resolve it (auto-supersede high-confidence over low-confidence), or it can sit until someone decides. The point is the contradiction *surfaces* — it doesn't get silently averaged. ## The on-device path For users who want maximum privacy, the whole stack runs locally: - A local Ollama serving an open-weight model (gpt-oss:20b runs on a M-series Mac). - ONTO's Sled storage for typed assertions (embedded; no separate database). - Local fastembed for any vector search. The setup: ```sh # Run ollama locally ollama serve # Pull a model ollama pull gpt-oss:20b # Configure ONTO export OLLAMA_BASE_URL=http://localhost:11434 export ONTO_STORAGE=sled export ONTO_STORAGE_PATH=./my-assistant-memory pip install onto-core ``` After this, the assistant runs on the user's machine. No data leaves. The model is fully local. ## The multi-device sync If users want their assistant on multiple devices, swap Sled for a Postgres backend. The Postgres lives on a server the user controls (or a managed service with appropriate controls). Each device runs ONTO pointed at the same Postgres. ```sh export ONTO_STORAGE=postgres export DATABASE_URL=postgres://user:pass@user-server/onto ``` The user owns the database. Their facts live in their Postgres, not in a vendor's. They can back it up, export it, query it, delete it. ## What the subagent pattern enables A daily-digest assistant has multiple jobs — calendar, email, notes. Each has its own tools. ONTO's subagent pattern lets you decompose: ```python calendar = onto.subagent("calendar", tools=["mcp:gcal:list", "mcp:gcal:create"]) email = onto.subagent("email", tools=["mcp:gmail:search", "mcp:gmail:reply"]) notes = onto.subagent("notes", tools=["mcp:notion:query"]) digest = onto.run_sync( "Give me my morning digest.", user_id="you", consent_scopes=["memory:write", "calendar:read", "email:read", "notes:read"], ) ``` Each subagent has constrained tools and scopes. The parent agent dispatches and synthesizes. The cost rolls up to the parent. The depth budget prevents infinite delegation. ## What still requires the user Personal AI does well on accumulation; it does poorly on judgment. The assistant should not unilaterally: - Reply to emails on the user's behalf without review. - Accept meeting invites that involve commitments. - Make purchases. - Share information with anyone. Plan Mode handles this. The assistant proposes "draft a reply to Alex"; the user reviews; the user sends. Or the user configures an auto-approval policy ("auto-approve drafts to close colleagues, request review for everyone else"). ## A summary A personal assistant that actually remembers you is built on three primitives: 1. Typed memory (UPO + SMO) instead of conversation buffers or vector-only stores. 2. Async writes so memory persistence doesn't slow turns. 3. Background summarization on a cheaper model so consolidation doesn't dominate cost. ONTO ships all three. The user's facts stay on the user's side. The assistant gets smarter about you over time, not because the model trained on you, but because the system stored what was learned. That's the bar for a useful personal AI. Everything else — frontier-model-of-the-month, voice support, novel UI — sits on top of it. --- *The [personal AI industry page](/industries/personal) walks through the runnable daily-digest example. The [thesis on agent memory](/blog/why-agent-memory-is-broken) explains why typed memory is the right primitive.* --- ### Research Agents: Mining Typed Facts from a Stack of Papers Published: 2026-04-18 · 9 min · industry URL: https://onto.neullabs.com/blog/research-agents-typed-claims Tags: research, knowledge graph, extraction The standard research-agent pattern looks like this. Ingest a corpus. Embed each chunk. At query time, retrieve the top-k similar chunks and ask the model to synthesize an answer. Repeat. This produces fluent paragraphs. It does not produce knowledge. The difference matters when you're doing research at scale: a literature review across 200 papers, an evidence synthesis for a meta-analysis, an environmental scan across a domain. At that scale the question becomes "what does the corpus *claim*" — and "show me the most similar paragraphs" is the wrong abstraction. This post is about an alternative architecture: extract typed claims, attach provenance, detect conflicts, and have the researcher curate at the conflict point. ## The four levels in research ONTO's four extraction levels map onto research in a way that's worth making explicit: - **session**: the current ingestion run. "We're processing the 2024 oncology corpus right now." - **profile**: identity of the researcher. "This researcher cares about survival outcomes." - **domain**: the field-specific knowledge graph. *This is where research claims live.* - **tool**: tool-local state. "The semantic-scholar search filter." The domain layer is the interesting one for research. A typed claim there looks like: ```python Assertion( subject="drug_X42", predicate="reduces_progression_in", object="cohort_Y", confidence=0.78, level="domain", domain_id="oncology_progression", provenance=Source( document_id="paper_5512", page=4, passage="...significantly reduced progression in cohort Y (p<0.05)...", extraction_prompt="claim-extract-v2", ), tags=["claim", "from_paper"], ) ``` That's the unit. Not a chunk; a claim with a source. Confidence is here because not every extracted claim is high-confidence. Provenance is here because the auditor (or co-author) will ask. ## The ingestion run For a corpus of 200 papers, you don't want sequential single-paper extraction. You want parallel: ```python papers = load_corpus("./corpus/") for paper in papers: onto.run_sync( paper.text, user_id="research_session_2026_q2", consent_scopes=["memory:write", "domain:claims:write"], extract=True, extract_async=True, # ← async write; parallelism is real domain_id=paper.field, ) ``` Each `run_sync` returns when the LLM extraction finishes. The persistence happens in the background. The next paper starts as soon as the LLM is done with this one. On a small Ollama instance you can saturate. `background_memory=True` routes the conflict detection itself to a background worker. By the time the ingestion finishes, the typed graph and the conflict task list are both ready. ## The conflict detection When two papers make claims that point at the same `(subject, predicate)` with different objects, ONTO creates a Plan Mode task: ``` Conflict in domain[oncology_progression]: drug_X42.reduces_progression_in = "cohort_Y" (paper_5512, p<0.05) drug_X42.reduces_progression_in = "no significant effect" (paper_6890, p=0.12) Resolve? ``` The task carries both sources. The researcher reads them. They decide: - Supersede (one paper is methodologically better) - Both (the populations were different; assertion needs refinement) - Defer (need more evidence) - Reject one (sample size too small) This is the *point* of having a research copilot — surfacing genuine disagreements in the literature so the researcher can adjudicate. A standard RAG agent would have averaged the two and produced a wishy-washy paragraph. ## What you do with the typed graph Once you have domain-scoped typed assertions with provenance, you can: - **Export to a graph store.** Neo4j, GraphDB, Stardog. The JSON-LD serialization is standard. - **Drive downstream synthesis.** Ask the LLM "summarize all claims about drug_X42, weighted by confidence." Now it's working from a structured graph, not a chunk soup. - **Generate citations.** Every claim has its source. Citation lists write themselves. - **Re-run extraction.** Schema changes? Re-extract from the same documents and diff the typed assertions. Reproducible by construction. - **Hand to a human reviewer.** The reviewer can read claims-with-sources and add their own assertions. The graph becomes a living research artifact. ## The cost of doing it right Realistically: extraction is more expensive than retrieval. You're asking the model to produce structured output per document, not just embed it. For a 200-paper corpus, expect ~2x the LLM bill of a pure RAG pipeline. The savings come at the *use* phase. Every downstream query against the typed graph is fast and cheap. Synthesis happens once, against typed data, rather than re-synthesizing every query. For research the trade is usually worth it. For one-shot Q&A over the corpus, RAG is fine. ## The reproducibility property Research reviewers care about reproducibility. With typed extraction: - Every assertion has `extraction_prompt` and `model_id` metadata. - Re-running with the same prompt and model regenerates the same (or near-same) assertions. - Diffs against an earlier extraction are visible at the assertion level. A traditional RAG pipeline can't promise this — the retrieval order can shift, the model can paraphrase differently, the answer drifts. Typed graphs are static unless you actively change them. A research artifact you can re-derive. ## The use case for non-research This architecture isn't only for academic research. Other places it fits: - **Competitive analysis.** Extract claims from competitor white papers and product pages; surface contradictions. - **Due diligence.** Extract typed facts from a target company's disclosures; flag inconsistencies. - **Document review for legal.** Extract typed claims from case law; route conflicts to attorneys. - **Regulatory monitoring.** Extract from regulator publications; conflicts surface when a rule contradicts an earlier interpretation. Any time you have a body of text and need to know *what it claims* — not just retrieve passages — typed extraction is the right primitive. ## A small caution The schema matters. If your domain predicates are poorly chosen, extraction produces noise. The work of designing good predicates is genuine and not LLM-automatable. Plan to iterate on the schema as you see what comes out of the first few hundred extractions. ## Summary Research copilots that extract typed claims: - Replace "top-k chunks" with "structured assertions with provenance" - Detect cross-document conflicts instead of silently averaging them - Produce a queryable, exportable knowledge graph as a byproduct - Cost more at extraction time; save many times that at synthesis time - Are reproducible by construction If you're building a tool for serious literature work, this is the architecture worth investing in. --- *See the [research industry page](/industries/research) for the runnable example, or [why agent memory is broken](/blog/why-agent-memory-is-broken) for the deeper case for typed memory.* --- ### HR & Recruiting AI Agents That Stay GDPR-Compliant Published: 2026-04-16 · 10 min · industry URL: https://onto.neullabs.com/blog/hr-ai-gdpr Tags: HR, recruiting, GDPR, compliance AI in HR is high-risk in the GDPR sense — automated decisions about people. Article 22 grants candidates the right not to be subject to a decision based solely on automated processing where the decision produces legal or similarly significant effects. A hiring rejection produces a "similarly significant effect." So the rule isn't "no AI" — it's "AI assists, humans decide." That maps onto Plan Mode neatly. This post walks through an HR/recruiting agent that uses the right primitives. ## What GDPR actually requires Three intersecting obligations: **Article 5 (principles).** Data minimization, purpose limitation, storage limitation. The agent can't accumulate candidate data forever, and can only use it for the purpose disclosed. **Article 6 (lawful basis).** Processing requires a lawful basis. For recruiting, usually consent or legitimate interest. Both require disclosure. **Article 22 (automated decision-making).** Restricted. With limited exceptions (consent, contract necessity), candidates have the right to human review. Plus state-level rules in the US (NYC AEDT, California's various employment AI laws) and the EU AI Act's high-risk classification for hiring AI. ## The architecture The HR/recruiting agent has to: 1. Process candidate data under explicit consent scopes. 2. Tag every candidate-derived assertion as `[pii]`. 3. Run any hiring-affecting decision through Plan Mode → human approval. 4. Enforce retention via decay. 5. Provide a deletion path (GDPR Article 17, the right to erasure). ONTO maps cleanly: ```python # Candidate ingestion. onto.run_sync( cv_text + "\n\n" + cover_letter, user_id="candidate_4421", consent_scopes=["memory:write", "candidate:intake"], extract=True, # Candidate-derived assertions get [pii] tag automatically via the # extraction prompt config; you can also force tags. ) # Recruiter requests a screening summary — read-only. summary = onto.run_sync( "Summarize candidate_4421 for the role.", user_id="candidate_4421", consent_scopes=["candidate:read", "tasks:write"], mode="plan_only", ) # Hiring decision is *never* autonomous. Plan Mode → recruiter decides. decision_task = next(t for t in summary.tasks if t.tool == "propose_hire_decision") recruiter_choice = recruiter_ui.review(decision_task) # advance / reject / interview # Recruiter's decision is logged with their identity, scopes, and reasoning. onto.audit.record_decision( candidate_id="candidate_4421", decided_by="recruiter_williams", decision=recruiter_choice, plan_id=summary.plan_id, ) ``` The audit log answers Article 22's "was this fully automated" question with "no — see the decision row signed by recruiter_williams." ## The bias question Algorithmic bias is the other major HR AI concern. Most major AI hiring guidance (NYC AEDT, EEOC) requires bias audits. ONTO doesn't run the audit for you. What it does: - **Stable input set.** The eval framework lets you run the agent against a curated set of candidate profiles and compare outcomes by protected class. - **Reproducible runs.** Pinned model versions, recorded extraction prompts, deterministic temperature settings. Re-running the eval next quarter is meaningful. - **Per-protected-class output tracking.** Custom guardrails can flag outputs that pattern-match exclusionary language. The audit is your responsibility. The primitives that make the audit possible are the SDK's. ## Retention and erasure GDPR Article 17 gives candidates the right to erasure ("right to be forgotten"). Article 5 requires storage limitation regardless of request. ONTO's decay curves attach to assertions. Candidate-tagged data gets a retention policy: ```python # In your decay config DECAY_POLICIES = { "candidate_unsuccessful": Decay( type="absolute", ttl_days=180, # 6 months for unsuccessful candidates on_expiry="purge", ), "candidate_hired": Decay( type="absolute", ttl_days=365 * 7, # 7 years for hired (varies by jurisdiction) on_expiry="purge", ), } ``` A nightly purge job removes expired assertions. The audit log records the purge so you can demonstrate compliance. For Article 17 requests, ONTO exposes a `purge_user(user_id)` API that removes all assertions for the user, with a confirmation row in the audit log. ## The consent disclosure Candidates need to know: - What data is collected. - What it's used for. - Who can access it. - How long it's kept. - Whether automated decisions are involved (if so, the logic and significance). - Their rights (access, rectification, erasure, portability). This is mostly a privacy notice question, not an SDK question. What the SDK provides is the data flow — which lets you write an accurate privacy notice instead of a generic one. Your privacy notice and your code should match; the audit log proves they match. ## The on-prem advantage For EU candidates especially, the simplest privacy story is "data never leaves the EU." Hosted models with US-based providers add data-transfer complexity (Schrems II, the EU-US Data Privacy Framework, SCCs). Self-hosted open-weight models in an EU data center sidestep this. ONTO's default deployment supports this directly — set up Postgres in Frankfurt, run Ollama in the same region, configure ONTO to use them. No data transfer occurs. For US-based recruiting that also processes EU candidates, the on-prem option is operationally cleaner than maintaining DPF certifications. ## The recruiter UX The agent shouldn't replace recruiters. It should accelerate them. The right UX: - Auto-screening: agent reads CV + cover letter, surfaces top candidates with rationale. - Recruiter reviews and decides. - Agent drafts outreach emails, interview questions, decision rationales. - Recruiter reviews, edits, sends. Plan Mode supports all of this. The recruiter sees what the agent proposes, the rationale, and the underlying assertions. They approve or modify. The agent's value is *velocity*, not *autonomy*. Same number of hires, same quality bar, less recruiter time per hire. ## The bias-rate metric A concrete suggestion: track the bias rate of your AI agent the same way you track its accuracy. Set up: - A held-out set of candidate profiles annotated with protected class (where allowed). - A monthly eval that measures advancement rate by class. - An alert when the rate drift exceeds threshold. This is a CI job using ONTO's eval framework. It's not the only thing you need (a real bias audit is more involved), but it's the kind of ongoing monitoring that catches regressions early. ## What still feels hard Two areas where the primitives help but don't solve: - **Disparate impact in the underlying model.** If the LLM is biased on its priors, your agent inherits it. Open-weight models you can fine-tune help; pre-deployment evals are mandatory. - **Hot-button decisions like accommodations or terminations.** These shouldn't touch an autonomous loop at all. Plan Mode here means "agent drafts; legal + HR review heavily." ## A summary HR AI under GDPR works when: - Every candidate-derived assertion is `[pii]` tagged. - Retention curves enforce data minimization. - Plan Mode keeps the *decision* human even when the *drafting* is AI. - Bias monitoring runs continuously in CI. - Privacy notices match the audit log. The SDK provides the primitives that make all of this concrete. The policy and the audits are your responsibility — and they're a lot easier to write against a system that has them built in. --- *The [features page](/features) walks through consent scopes and Plan Mode in detail. The [comparison page](/comparison) covers how other SDKs handle this.* --- ### On-Premise AI Agents: How to Run Without Sending Data to OpenAI Published: 2026-04-14 · 11 min · engineering URL: https://onto.neullabs.com/blog/on-premise-ai-agents Tags: on-prem, Ollama, self-hosted, open-weight import Callout from '@components/article/Callout.astro'; There are three reasons to run AI agents on-prem in 2026: 1. **Data residency or confidentiality requirements** that forbid sending data to hosted providers (healthcare PHI, financial MNPI, legal privilege, EU data localization). 2. **Cost** at sustained scale — once you're past a few million tokens per day, a self-hosted GPU is cheaper than a hosted API. 3. **Latency** for tight loops where round-trip to a hosted vendor is the bottleneck. This is a practical guide to all of it. The stack is the one ONTO defaults to, but the architecture applies to any agent SDK that supports on-prem deployment. ## The stack A complete on-prem agent stack has five parts: 1. **Model serving.** Ollama or vLLM running an open-weight model. 2. **Embedding model.** fastembed or sentence-transformers, locally. 3. **Vector store.** pgvector inside Postgres, or qdrant locally. 4. **Typed memory store.** Postgres for production, Sled for single-machine dev. 5. **Agent runtime.** ONTO (or your SDK of choice) running on the same network. Everything inside your VPC. No external API call required for any agent operation. ## Model serving For a single-machine setup, Ollama is the easiest: ```sh # Install Ollama curl -fsSL https://ollama.com/install.sh | sh # Pull a model ollama pull gpt-oss:20b # Verify it serves curl http://localhost:11434/api/generate -d '{ "model": "gpt-oss:20b", "prompt": "Hello" }' ``` For multi-machine or higher-throughput deployments, vLLM gives better serving performance. The OpenAI-compatible API means it's a drop-in for any SDK that talks to OpenAI: ```sh # vLLM with OpenAI-compatible server python -m vllm.entrypoints.openai.api_server \ --model gpt-oss:20b \ --tensor-parallel-size 2 \ --port 8000 ``` Configure ONTO to point at either: ```sh # Ollama export OLLAMA_BASE_URL=http://localhost:11434 # vLLM (OpenAI-compatible) export OPENAI_BASE_URL=http://localhost:8000/v1 export OPENAI_API_KEY=anything # vLLM doesn't check; required by SDK ``` The same agent code runs against either. ## Choosing a model For 2026, three open-weight model families are practical for production agents: - **gpt-oss family (OpenAI's open-weight models).** 20B and 120B variants. Strong on tool calling and structured output. Default for ONTO. - **Llama 3 family (Meta).** 8B, 70B, 405B. Stable; broad community support. - **Qwen 2.5 family (Alibaba).** 7B, 32B, 72B. Strong on multilingual. For the bulk of agent workload (extraction, tool calling, plan generation, summarization), the 20B-class models are sufficient. For complex reasoning that the 20B class struggles with, route those specific calls to a 70B or larger. ## Embeddings For vector search (LlamaIndex-style RAG, semantic search over notes), use a local embedding model: ```python from fastembed import TextEmbedding emb = TextEmbedding("BAAI/bge-small-en-v1.5") vectors = list(emb.embed(["text 1", "text 2"])) ``` fastembed runs in CPU at acceptable speeds for moderate volumes. For high volume, sentence-transformers on a GPU. ONTO's vector layer accepts any embedding function; configure yours: ```python from onto import Onto onto = Onto( embedding_provider=fastembed_provider("BAAI/bge-small-en-v1.5"), ... ) ``` ## Typed memory store For production, Postgres + pgvector: ```sh # Set up Postgres docker run -d -p 5432:5432 \ -e POSTGRES_PASSWORD=secret \ pgvector/pgvector:pg16 # Configure ONTO export ONTO_STORAGE=postgres export DATABASE_URL=postgres://postgres:secret@localhost/onto ``` ONTO's schema migration creates the tables; the rest is operational. Index `(tenant_id, subject, predicate)` for fast lookups. Index the vector column for ANN. For dev or single-machine deployments, Sled (embedded key-value): ```sh export ONTO_STORAGE=sled export ONTO_STORAGE_PATH=./onto-data ``` Sled needs no separate process. Backups are file copies. Good for dev and small-scale personal AI. ## Kubernetes deployment For production, a Helm chart deploys ONTO + Postgres + Ollama + your application: ```yaml # values.yaml (sketch) onto: replicas: 3 storage: postgres databaseUrl: postgres://onto:...@postgres:5432/onto ollama: replicas: 2 modelName: gpt-oss:20b gpuLimit: 1 postgres: storageSize: 200Gi resources: requests: memory: 4Gi cpu: 2 ``` The framework repo includes a chart at `deploy/helm/onto/`. For most production deployments the chart is a starting point you customize. ## The latency picture Round-trip times for a typical agent turn, on-prem vs hosted: - Hosted Claude/GPT: 800-2500ms for a typical agent turn - On-prem Ollama 20B on A100: 400-1200ms for the same turn - On-prem vLLM 20B on A100 with batching: 200-700ms The latency win is real for moderate-complexity prompts. For very large prompts or complex reasoning, the larger hosted models can be faster despite the round-trip because their inference is faster. The right pattern for latency-sensitive agents: keep the bulk of work on the 20B local model; route only the hardest reasoning to a larger model (local or hosted). ## The cost picture For an A100 (40 GB) on AWS at $2.10/hour (p4d.24xlarge breakdown), running 24/7, you pay ~$1,500/month. That's amortized across whatever throughput you serve. vLLM on an A100 serves roughly 20-40 requests/second for a 20B model with reasonable prompt sizes. At an average of 1000 tokens per turn, that's ~24M tokens/hour at peak, ~600M tokens/day at peak. A hosted API at $3/MTok (typical for a mid-size model) would bill ~$1,800/day for the same throughput. The break-even is roughly 10-20% utilization of the GPU. If your traffic is bursty, hosted is more cost-effective. If your traffic is steady at moderate volume, on-prem wins. This is the calculation every team should run. The strongest reasons for on-prem are compliance and data control. Cost savings are a nice secondary benefit at high utilization. If your only motivation is "I want to save money on the OpenAI bill," do the spreadsheet first. ## The operational surface What you take on with on-prem: - GPU monitoring (utilization, temperature, OOM events) - Model serving health checks - Postgres operations (backups, replication, vacuum) - Network security (don't expose the model server to the internet) - Model upgrades (when a new gpt-oss version ships, you redeploy) - Cost attribution per workload (a per-tenant token meter) A small SRE team can run this. A single application engineer with no SRE support cannot. ## The hybrid pattern Most production deployments end up hybrid: - Default: on-prem open-weight for the bulk of traffic. - Burst: route to a hosted endpoint when GPU capacity is exhausted. - Specialized: route hardest reasoning calls to Claude/GPT for higher quality. ONTO supports this with provider-aware routing. The agent code is unchanged; the provider config decides where calls go. ## Where on-prem doesn't help To be honest: - **One-shot research projects.** The setup cost isn't worth it for a 3-week project. - **Frontier reasoning workloads.** The 20B class is below frontier; if your agent needs maximal reasoning quality, hosted frontier models are still ahead. - **Voice/realtime.** OpenAI's realtime API is fundamentally hosted; no equivalent on-prem story today. For those, hosted is the right answer. ## A summary On-prem AI agents in 2026 are a real option, not a workaround. The stack is mature: Ollama or vLLM for serving, fastembed for embeddings, Postgres + pgvector for memory, Helm for deployment, ONTO (or any well-designed agent SDK) for the runtime. The decision factors are compliance, data control, sustained-traffic cost, and your team's operational capacity. If three of the four point on-prem, ship it. --- *ONTO defaults to local-friendly deployment — the [docs page](/docs) walks through the Ollama setup. The [features page](/features#open-weight) explains the provider model.* --- ### Multi-Tenant Memory: Scoping AI Agents for SaaS Published: 2026-04-12 · 9 min · engineering URL: https://onto.neullabs.com/blog/multi-tenant-agent-memory Tags: multi-tenant, SaaS, agent memory Multi-tenant SaaS has one bug nobody recovers from: tenant A sees tenant B's data. Customers tolerate slow days, broken features, and bad design. They do not tolerate "we leaked your competitor's information into your agent's prompt." If your agent has memory, this risk is real, and it's structurally different from API-level multi-tenancy. The memory layer has to enforce the boundary, and it has to do it at every read and every write, on every code path. This post is about how to design that boundary so it doesn't depend on every code change being correct. ## The four leak vectors Where can tenant A's data end up in tenant B's context? 1. **Direct read.** Code reads memory without filtering by tenant_id. 2. **Vector retrieval.** Embeddings retrieve cross-tenant chunks because the index isn't tenant-scoped. 3. **Background worker confusion.** A summarization job runs against the wrong tenant's data because the job spec didn't carry tenant_id. 4. **Shared cache.** A response cache keyed only on prompt content returns tenant A's cached response to tenant B. Each of these has happened in production at companies you've heard of. The fix in every case is the same: tenant_id is a required field on every memory operation, enforced by the runtime, not by convention. ## The structural fix In ONTO, every Assertion carries `tenant_id`. Every query requires it. Every Tool that reads memory declares which tenant_id it operates on. Cross-tenant operations require an explicit `cross-tenant` consent scope that you grant in narrow contexts. ```python # Write — must carry tenant_id. onto.memory.add_assertion( tenant_id="acme", subject="customer_42", predicate="tier", object="gold", consent_scopes=["memory:write"], ) # Read — must carry tenant_id. asserts = onto.memory.query( tenant_id="acme", subject="customer_42", consent_scopes=["memory:read"], ) # Cross-tenant read — refused unless explicit scope. try: onto.memory.query( tenant_id=None, # request across tenants subject="customer_42", ) except ConsentDenied as e: # e.code == "tenant_mismatch" pass ``` The runtime refuses any cross-tenant operation that lacks the special scope. The denial is logged. You can grep for denials weekly to confirm nobody is trying. ## The agent loop A multi-tenant agent loop passes tenant_id through every call: ```python async def handle_request(tenant_id: str, user_id: str, message: str): result = await onto.run( message, tenant_id=tenant_id, user_id=user_id, consent_scopes=["memory:read", "memory:write", "support:read"], ) return result.output ``` Every tool the agent calls inherits the tenant_id from the run. If a tool tries to read memory without it, the runtime refuses. There's no fall-through to "default tenant" or "any tenant" — you either have a tenant_id or you don't operate. ## The vector store Vector search needs the same discipline. Each vector carries tenant_id metadata; queries filter on it: ```sql SELECT *, vector <-> $query AS distance FROM documents WHERE tenant_id = $tenant ORDER BY distance LIMIT 10; ``` The query plan should use a partial index on `(tenant_id, vector)` so the filter pushes down before the ANN search runs. Performance is good up to very large fanout. For tenants with vast document corpora, partition the vector table or move to per-tenant collections in a vector database. The application code doesn't change — only the storage strategy. ## The background worker problem Background workers (summarization, conflict detection, batch extraction) need tenant_id baked into the job spec: ```python @background_job def summarize_session(tenant_id: str, session_id: str): # Loaded with tenant_id; can't accidentally pull from another tenant. session = onto.memory.query(tenant_id=tenant_id, session_id=session_id) summary = llm.summarize(session) onto.memory.add_assertion( tenant_id=tenant_id, subject=session.user_id, predicate="session_summary", object=summary, ) # Enqueue with tenant_id required. queue.enqueue(summarize_session, tenant_id="acme", session_id="s_2026_04_12") ``` If the function signature requires tenant_id, the caller must provide it. The runtime checks it. The bug class "background worker ran without tenant context" disappears. ## The cache problem Prompt response caching is a real performance win — same prompt, same model, same response. But the cache key must include tenant_id, or the cache leaks: ```python # BAD — cache key is just prompt content. cache_key = hash(prompt) # GOOD — cache key includes tenant. cache_key = hash((tenant_id, prompt, model_id)) ``` This is so obvious that it's the bug people make anyway. A cache decorator that doesn't take tenant_id into its key is the kind of code review issue that gets missed in a hurry. The structural fix: a cache wrapper that requires `tenant_id` in its function signature. The compiler / type checker enforces the call site. ## The cross-tenant read use case Sometimes you need cross-tenant operations: - Global analytics ("how many active tenants this month"). - Anonymized model evals ("performance across all production traffic"). - Cross-tenant deduplication ("this email appears in many tenants — likely public"). For these, declare a `cross-tenant` consent scope and grant it only to the specific code path: ```python # Analytics service — gets explicit cross-tenant scope. analytics_run = onto.run( "Compute monthly active tenants.", consent_scopes=["cross-tenant:read", "analytics:write"], ) ``` The audit log records every cross-tenant operation. A monthly review confirms the only cross-tenant operations were the legitimate analytics jobs. ## The audit story Two queries you should be able to run any time: ```sql -- Any cross-tenant operation in the last 30 days? SELECT count(*), code FROM audit_log WHERE event_type = 'consent_denied' AND code IN ('tenant_mismatch', 'cross_tenant_unauthorized') AND timestamp > now() - interval '30 days' GROUP BY code; -- Any successful cross-tenant read? SELECT * FROM audit_log WHERE event_type = 'tool_call' AND 'cross-tenant:read' = ANY(consent_scopes) AND timestamp > now() - interval '30 days'; ``` The first answers "were there leak attempts." The second answers "were there legitimate cross-tenant operations." Together they answer the security review. ## The cost meter angle Multi-tenant cost attribution is the other side of this story. You want to bill or analyze per tenant: ```python cost = onto.cost(tenant_id="acme") # { # foreground: { tokens: 1_234_567, dollars: 4.92 }, # background: { tokens: 8_456_021, dollars: 1.08 }, # tools: { "search": 200_000, "calendar": 50_000 }, # } ``` This lets you turn "the AI bill was $5000 last month" into "tenant X accounted for $420, tenant Y for $300, ..." Which means you can charge them, throttle them, or investigate them. ONTO's cost meter splits foreground vs background because the background work uses cheaper models — accounting that detail honestly matters for SaaS pricing. ## A test before you ship Run an integration test: ```python def test_no_cross_tenant_leak(): onto.memory.add_assertion(tenant_id="A", subject="x", predicate="p", object="A_secret") onto.memory.add_assertion(tenant_id="B", subject="x", predicate="p", object="B_secret") a_results = onto.memory.query(tenant_id="A", subject="x") assert all(r.object == "A_secret" for r in a_results) assert not any(r.object == "B_secret" for r in a_results) b_results = onto.memory.query(tenant_id="B", subject="x") assert all(r.object == "B_secret" for r in b_results) assert not any(r.object == "A_secret" for r in b_results) with pytest.raises(ConsentDenied): onto.memory.query(tenant_id=None, subject="x") ``` If this test passes, your multi-tenant isolation is at least one structural level beyond "trust the code." ## Summary Multi-tenant memory isolation is a bug class you can engineer out. Make tenant_id a required field on every memory operation. Make the runtime check it. Make cross-tenant operations require an explicit scope. Log every denial. Test the leak case. The cost of building this in upfront is small. The cost of fixing it after a leak is your company. --- *The [customer support industry page](/industries/customer-support) walks through a multi-tenant agent end-to-end. The [features page](/features#policy-guard) covers consent scopes.* --- ### Cost Observability for AI Agents: Per-User, Per-Tenant Billing Published: 2026-04-10 · 9 min · engineering URL: https://onto.neullabs.com/blog/cost-observability-ai-agents Tags: cost observability, FinOps, multi-tenant Every agent product hits this moment. The model bill arrives. It's bigger than expected. The finance team asks: which customer drove it? Which feature? Which background process? If your answer is "let me dig through the logs for a few days," you didn't build observability — you built hope. This post is about instrumenting an agent system so cost attribution is structural. The answer to "which tenant ran up the bill" is one query. ## What you actually need Per-LLM-call you need: - `tenant_id` - `user_id` - `session_id` - `run_id` - `tool_call_id` (if applicable) - `model_id` - `cost_category`: `foreground` | `background` | `eval` | `internal` - `input_tokens`, `output_tokens`, `cost_usd` - `cache_hit`: boolean - `timestamp` If you have those for every call, every cost question is answerable with SQL. ## The cost meter primitive ONTO ships this as a first-class primitive: ```python # At the end of a billing cycle. cost = onto.cost( tenant_id="acme", start=datetime(2026, 4, 1), end=datetime(2026, 5, 1), ) # { # "foreground": { # "tokens": {"input": 12_300_000, "output": 4_500_000}, # "dollars": 124.30, # "calls": 4_280, # }, # "background": { # "tokens": {"input": 89_000_000, "output": 6_700_000}, # cheaper model # "dollars": 14.20, # "calls": 22_100, # }, # "by_model": { # "gpt-oss:20b": 80.10, # "claude-sonnet-4.5": 58.40, # }, # "by_tool": { # "search": 22.40, # "calendar_create": 4.10, # ... # }, # } ``` These numbers are computed from the run database, not reconstructed from log scrapes. ## The foreground / background split The split is more important than it sounds. A typical agent has two cost categories: - **Foreground**: turns the user actually waited for. The cost the user "experienced." - **Background**: summarization, conflict detection, consolidation. Work that improves future turns but isn't directly tied to a user request. These usually use different models. Foreground might run on Claude or gpt-oss:20b; background runs on a cheaper open-weight model. The split matters because: - **Billing.** If you bill the customer, you bill foreground, not background. - **Optimization.** Background can be made arbitrarily cheap by routing to smaller models; foreground latency / quality trade-offs are different. - **Capacity planning.** Background can run at off-peak; foreground can't. ONTO sets `cost_category` automatically based on which code path made the call. You don't have to instrument it. ## The cache problem Prompt caching changes the picture. Anthropic's prompt cache, OpenAI's prefix cache, and similar features mean some LLM calls don't bill at the full rate. You should track: - **Billed cost**: what the provider charged you. - **Effective cost**: what you'd pay without cache hits. The ratio (billed / effective) is your cache hit rate, financially. Track it; optimize it. For your customer billing, use billed cost (you're not going to charge them for tokens you didn't pay for). For capacity planning and what-if analysis, use effective cost. ## The per-tool view Tools differ wildly in cost. A simple web search might be 200 tokens; a long-context document summary might be 30,000. Per-tool cost surfaces hot spots: ```sql SELECT tool_name, sum(cost_usd), avg(cost_usd), count(*) FROM tool_calls WHERE tenant_id = 'acme' AND timestamp > now() - interval '30 days' GROUP BY tool_name ORDER BY sum(cost_usd) DESC; ``` Often the surprise is a tool that's called a lot, not the one with high per-call cost. "Search costs $20/month" beats "summarize costs $80/month" if search is called 100x more often. ## The per-user distribution For a SaaS, you don't want average-user cost; you want the distribution. ```sql SELECT percentile_cont(0.50) WITHIN GROUP (ORDER BY user_cost) as p50, percentile_cont(0.95) WITHIN GROUP (ORDER BY user_cost) as p95, percentile_cont(0.99) WITHIN GROUP (ORDER BY user_cost) as p99, max(user_cost) as max FROM ( SELECT user_id, sum(cost_usd) as user_cost FROM tool_calls WHERE tenant_id = 'acme' AND timestamp > now() - interval '30 days' GROUP BY user_id ) u; ``` The p99 / p50 ratio tells you whether you have a "long tail" cost problem. Usually you do — a small number of power users drive disproportionate cost. The question is whether they're high-value users (fine) or runaway-loop bugs (alert). ## The unit economics question Putting it together: your unit economics on a per-customer basis depend on whether you can match per-customer LLM cost against the per-customer revenue. ``` Customer revenue (monthly): $200 Customer LLM cost (monthly): $35 ← from onto.cost(tenant_id=...) Customer infrastructure share: $10 Customer support load (estimated): $15 Customer gross margin: $140 / $200 = 70% ``` If you can pull the LLM cost number with one query, you can run this analysis monthly. If you can't, you're flying blind on whether each customer is profitable. ## The runaway-loop alarm A specific failure mode: an agent gets stuck in a loop and burns thousands of tokens before anyone notices. Detection has to be in the cost path: ```python # In the runtime if session_cost_so_far > session_budget * 0.8: log.warn("approaching session budget", tenant=t, user=u, session=s) if session_cost_so_far > session_budget: raise BudgetExceeded(...) ``` ONTO's runtime supports per-session cost budgets natively. Set them low enough that the alarm fires before the bill blows up. ## The integration with OpenTelemetry For broader observability, ONTO exports cost as OpenTelemetry attributes on each span: ``` span.llm.call tenant_id: acme user_id: alice cost_category: foreground model_id: gpt-oss:20b input_tokens: 1200 output_tokens: 400 cost_usd: 0.012 cache_hit: false ``` Drop these into Grafana or Honeycomb. Build a per-tenant cost dashboard. Alert on anomalies. The cost meter and the trace are the same data shaped two ways. ## What to do if you're not on ONTO If you're using a different SDK, the principles are the same: 1. **Wrap your LLM client.** Every call goes through a wrapper that tags it with tenant_id, user_id, cost_category. 2. **Persist the calls to a metering table.** Cheap; row per call. 3. **Build the queries you need.** Per-tenant, per-user, per-tool, per-category. 4. **Set up budgets and alarms.** Per-session, per-day, per-tenant. 5. **Wire the cost into OpenTelemetry.** Make the data show up where your existing observability lives. None of this is glamorous. All of it is cheap to do upfront and expensive to retrofit. ## A summary Per-tenant, per-user, per-tool LLM cost attribution is the table-stakes observability story for any agent product running in production. Either you instrument it upfront and your CFO is happy, or you instrument it after the first surprising bill and your CFO is unhappy. The data model isn't complicated. The discipline is making sure every call carries the right tags and lands in a queryable store. Pick an SDK that ships this, or build it yourself — but build it. --- *The [features page](/features#cost-meter) covers ONTO's built-in cost meter. The [multi-tenant memory article](/blog/multi-tenant-agent-memory) covers the related tenant-isolation work.* --- ### Ontology Extraction Explained: The Four Levels Every Agent SDK Should Have Published: 2026-04-08 · 8 min · engineering URL: https://onto.neullabs.com/blog/ontology-extraction-four-levels Tags: ontology extraction, typed memory, explainer When an agent reads "I'm Alice, in UTC, I prefer mornings, and right now I'm in Lyon planning a trip," what should happen? Most agent SDKs store the whole thing as a chat turn and move on. The next session, the agent has forgotten everything. A smarter agent extracts typed facts from that sentence. But where do those facts go? - "Alice's timezone is UTC" → durable; should survive forever. - "Alice prefers mornings" → durable; should survive forever. - "Alice is currently in Lyon" → transient; should evaporate when the trip ends. Three different lifecycles in one sentence. A single "memory" bucket can't hold them right. You need levels. ## The four levels ONTO's extraction routes assertions to one of four levels: **Session.** Lives within the current session. Examples: current location during a trip, current intent ("rebook"), current document being worked on. Evaporates at session end unless promoted. **Profile.** Durable identity. Examples: timezone, name, employer, preferences, primary language. Long decay; survives across sessions. **Domain.** App-scoped facts in a particular domain. Examples: a customer's tier in your CRM, a patient's allergy list, a research paper's claim about drug X. Scoped by `domain_id` (tenant, matter, project). **Tool.** Tool-local state. Examples: a saved search filter, a pinned dashboard, a configured threshold. Belongs to the tool's behavior rather than to the user or domain. Each level has its own decay curve, retention policy, and consent model. Each is queried by a different lookup pattern. Each has different audit obligations. ## The extraction flow When you set `extract=True` on a run: 1. The LLM reads the user input + relevant memory. 2. It proposes a list of typed assertion candidates, each with a level label. 3. The runtime audits the routing (does the assertion's content match the level's expected shape?). 4. Policy Guard checks consent scopes for the writes. 5. Assertions are persisted to the appropriate stores. ```python result = onto.run_sync( "Hi, I'm Alice. UTC, mornings. Currently in Lyon planning a trip.", user_id="alice", consent_scopes=["memory:write"], extract=True, ) for ea in result.extracted_assertions: print(f"{ea.level:8} {ea.subject}.{ea.predicate} = {ea.object!r}") # session alice.current_location = "Lyon" # session alice.current_intent = "plan_trip" # profile alice.timezone = "UTC" # profile alice.prefers = "mornings" ``` Four assertions, four levels, each routed correctly. ## Why session vs profile matters Imagine the user wrote "I'm in Lyon" and you stored it as a profile fact. Six months from now the agent thinks Alice lives in Lyon. The reverse — storing "Alice's timezone is UTC" as a session fact — would mean the agent forgets her timezone next session. The level *is* the decay policy. Conflating them is the underlying cause of "the agent doesn't really remember me" complaints. ## Why domain matters Domain-level extraction is what makes multi-tenant memory clean. A customer-support agent extracts assertions about Tenant A's customer; those go into the `domain="tenant_A"` partition. Tenant B's data lives in `domain="tenant_B"`. Cross-domain reads refuse. This is also how research-agent corpus claims work. A claim about drug X42 goes into `domain="oncology_progression"`; a claim about a different topic lives elsewhere. The graph stays sane. ```python # Healthcare: each patient is a domain. onto.run_sync( intake_text, user_id="patient_a912", domain_id="patient_a912", consent_scopes=["memory:write", "patient:intake"], extract=True, ) # Research: each field is a domain. onto.run_sync( paper_text, user_id="researcher_lim", domain_id="oncology_progression", consent_scopes=["memory:write", "domain:claims:write"], extract=True, ) ``` The same SDK, the same call shape, different domain partitioning. ## Why tool matters The tool level is the one people forget. It holds *the tool's* state, not the user's. Examples: - A saved search filter the user configured in the search tool. - A pinned dashboard arrangement in the analytics tool. - A custom retry threshold a workflow uses. These aren't user facts (you wouldn't query "what is Alice's timezone" by going to tool-level memory). They aren't domain facts (they're independent of the domain). They're configuration that belongs to a tool. Storing them in tool-level memory means a single user can have tool configurations that survive across sessions without polluting their profile or any domain. ## The extraction prompt The LLM has to label each candidate assertion with a level. The extraction prompt is where the labeling is taught. A minimal one: ``` You are extracting typed assertions from user messages. For each fact you find, output an assertion with one of four levels: - session: transient context (current location, current intent, current task) - profile: durable identity (timezone, preferences, name, employer) - domain: app-scoped facts (customer tier, patient allergy, research claim) - tool: tool-configuration state (saved searches, pinned views, thresholds) Output JSON. Be conservative: prefer "session" over "profile" when unsure. ``` The "prefer session over profile when unsure" is important. Wrong-level "profile" assertions accumulate; wrong-level "session" assertions just evaporate. Bias toward the recoverable error. ## Conflicts within a level Within a level, two assertions on the same `(subject, predicate)` create a conflict. The runtime surfaces this as a Plan Mode task: ``` Conflict in profile: alice.timezone = "UTC" (March 2026) alice.timezone = "JST" (May 2026) Supersede or merge? ``` The resolution decides whether the new fact supersedes the old (Alice moved) or whether they coexist (Alice is in two timezones — possible for a digital nomad). ## When you don't extract `extract=True` is opt-in. A read-only chat agent skips it. A one-shot Q&A doesn't need it. The decision is per-call. For high-volume background ingestion (e.g., a research corpus), use `extract_async=True` to spawn the writes to a background task. The next ingestion call starts immediately. ## What if your SDK doesn't have levels? Build them as a wrapper. Tag every assertion you write with `level=session|profile|domain|tool` and enforce the routing in your application code. The LLM still labels; your code still partitions. You lose runtime enforcement but gain the architectural discipline. Or pick an SDK that ships this. ONTO is one; we expect others to follow as memory-typing matures. ## Summary The four-level model — session, profile, domain, tool — is the right vocabulary for routing typed assertions. Without it, all memory ends up as one flat bucket and the agent can't tell what should survive a session vs what should evaporate. It's not magic. It's just a small architecture pattern that, applied consistently, makes agent memory actually work over time. --- *The [features page](/features#four-level-extraction) covers extraction in detail. The thesis on [agent memory](/blog/why-agent-memory-is-broken) explains the underlying argument.* --- ### Getting Started with ONTO in Python: A 10-Minute Tutorial Published: 2026-04-06 · 8 min · engineering URL: https://onto.neullabs.com/blog/python-quickstart Tags: tutorial, Python, getting started import Callout from '@components/article/Callout.astro'; This is a 10-minute walk through your first ONTO agent in Python. By the end, you'll have an agent that extracts typed facts from natural language, plans actions, gets your approval, and executes them under consent scopes — the whole Extract → Plan → Approve → Execute loop. ## Step 1 — Install ```sh pip install onto-core ``` That's the whole install. ONTO is a single package with sensible defaults. ## Step 2 — Get a free Ollama API key ONTO defaults to Ollama Cloud, which has a free open-weight tier. Get a key: ```sh # Visit https://ollama.com/settings/keys — sign up, generate a key. export OLLAMA_API_KEY=... ``` If you'd rather use Claude or GPT, set `ANTHROPIC_API_KEY` or `OPENAI_API_KEY` instead. The code below works either way. ## Step 3 — Your first run Create `quickstart.py`: ```python from onto import Onto, Models, BuiltinTools # Initialize with the open-weight default. onto = Onto( model=Models.GPT_OSS_20B, builtin_tools=BuiltinTools.STANDARD, ) result = onto.run_sync( "Hello. I'm Alice. UTC, mornings.", user_id="alice", consent_scopes=["memory:write"], extract=True, # ← extract typed facts from this message ) print("Output:", result.final_output) print("\nExtracted assertions:") for ea in result.extracted_assertions: print(f" {ea.level:8} {ea.subject}.{ea.predicate} = {ea.object!r}") ``` Run it: ```sh python quickstart.py ``` You should see: ``` Output: Hi Alice! Nice to meet you. ... Extracted assertions: profile alice.timezone = "UTC" profile alice.prefers = "mornings" ``` Two typed assertions. The LLM identified them, the runtime routed them to the `profile` level, and they're now in your local memory store. Next time you run, the agent will already know. ## Step 4 — Memory persists Run a second call without re-introducing yourself: ```python result = onto.run_sync( "What do you know about me?", user_id="alice", consent_scopes=["memory:read"], ) print(result.final_output) ``` The agent loads alice's profile assertions at the start of the run and uses them in its response. The first answer might be something like "You're Alice, in UTC, and you prefer mornings." Memory survives. This is what a conversation buffer can't do. ## Step 5 — Plan Mode Now let's plan something with side effects: ```python plan = onto.run_sync( "Plan a 3-day trip to Lyon for next weekend.", user_id="alice", mode="plan_only", # ← no tools fire consent_scopes=["tasks:write", "tasks:read"], ) print("Plan tasks:") for task in plan.tasks: print(f" - [{task.tool}] {task.rationale}") ``` You'll see something like: ``` Plan tasks: - [search_hotels] Find hotels in Lyon for the dates. - [book_hotel] Book the chosen hotel. - [search_trains] Find trains from your origin to Lyon. - [book_train] Book the chosen train. ``` None of these have actually fired. The agent's reasoning is captured as typed `Task` objects you can inspect, approve, or modify. ## Step 6 — Execute approved tasks Now you decide which to run: ```python for task in plan.tasks: if input(f"Approve [{task.tool}]? (y/n): ").lower() == "y": result = onto.execute_task( task, consent_scopes=[ "tools:web.fetch", "tools:http_request", # In a real deployment, also: "booking:hotel", "booking:train" ], ) print(f" → {result.summary}") ``` This is the structural guarantee: the Plan run carried `tasks:write` (it can propose) but not execute scopes (so side-effect tools refuse). Only the Execute run, with explicit consent, actually fires the tools. ## Step 7 — Check the cost ```python cost = onto.cost(user_id="alice") print(f"Tokens: {cost.tokens.input + cost.tokens.output}") print(f"Cost: ${cost.dollars:.4f}") print(f"Foreground: ${cost.foreground.dollars:.4f}") print(f"Background: ${cost.background.dollars:.4f}") ``` ONTO tracks per-user cost split between foreground (your turns) and background (any summarization or memory consolidation that ran). On the free Ollama Cloud tier, the dollar cost is zero — but the token counts are real and useful for capacity planning. ## What just happened In ~25 lines of code you've used five primitives: 1. **`extract=True`** — typed-fact extraction at the right level (profile vs session vs domain vs tool). 2. **UPO** — durable user memory that persisted across calls. 3. **`mode="plan_only"`** — Plan Mode, the runtime invariant that proposed actions without firing them. 4. **`consent_scopes=[...]`** — Policy Guard, the per-call consent enforcement. 5. **`onto.cost(...)`** — the cost meter, per-user attribution. This is the whole agent runtime in miniature. Everything else — multi-tenant scoping, MCP tools, subagents, background workers, eval framework — composes on top. ## Where to go next - **[Industries](/industries)** — see how the primitives map to specific verticals (healthcare, customer support, personal AI, research). - **[Features](/features)** — deeper coverage of each primitive. - **[Comparison](/comparison)** — how ONTO stacks up against Claude, OpenAI, Google, LangChain. - **[Canonical docs](https://onto-framework.github.io/onto-framework)** — full SDK reference, deployment guides, model providers. The same agent code runs in TypeScript and Rust with identical semantics. If your team is polyglot, you can put the Python tutorial above in front of one engineer and the TypeScript equivalent in front of another — they'll arrive at the same architecture. --- *[Install ONTO](/docs) and try the rest of the examples. If something doesn't work, [file an issue](https://github.com/onto-framework/onto-framework/issues/new) — we'd rather fix the rough edges than ignore them.* --- ## Reference - Quickstart: https://onto.neullabs.com/quickstart - Architecture: https://onto.neullabs.com/architecture - FAQ: https://onto.neullabs.com/faq - Glossary: https://onto.neullabs.com/glossary - GitHub: https://github.com/onto-framework/onto-framework - Canonical docs: https://onto-framework.github.io/onto-framework - PyPI: https://pypi.org/project/onto-core/ - npm: https://www.npmjs.com/package/@onto/core - crates.io: https://crates.io/crates/onto-runtime - Built by Neul Labs: https://www.neullabs.com