ovat navigate

One YAML file.
One command.
A local agent.

OVAT turns a tool-calling AI agent into a config file. You describe the model, the tools and the behaviour in workflow.yml, run ovat run, and the agent answers on your own Intel hardware. No API keys, no cloud, nothing leaves the machine.

pip install ovat
Start the guide
6:21 The end-term demo on an Intel AI PC: indexing and a RAG answer, the same question through all four engines, a run trace and ovat bench, then the terminal UI chatting with audio, image and document tools. One note: some answers in the four-engine run misread the memory budget as 128 MB. The note they cite says a full run fits in 8 GB.

What a run prints

The commands from the guide, replayed as text: a RAG answer with its source, a dry run that needs no server, and ovat doctor on a Mac.

examples/rag
$ ovat run examples/rag/workflow.yml -i "What is OVAT's memory budget?"
engine: native loop (loop.py)
On the minimal tier a full agent run fits in 8 GB. That is the target the project set itself; the number to trust on your own machine is the one ovat run --telemetry reports there.
 
sources: examples/rag/docs/ovat-facts.md
Replayed from real CLI output in the OVAT repo and docs.

how-it-works

How one question travels through OVAT

Follow the light. A question enters as a command, becomes a checked config, gets wired into an agent, and loops between the model and your tools until there is an answer. Press any step to stop on it.

The OVAT request flow ovat run hands workflow.yml to the config loader, the factory builds an agent engine, the engine asks OVMS, OVMS runs the model on the device, tools run in Python, and a reply with no tool calls becomes the answer. the request a tool runs the answer cli/main.pyovat run--input "..." config/workflow.pyworkflow.ymlchecked, strict agent/factory.pybuild_agent()model + tools + RAG 1 of 4 enginesAgent loopagent.type tools/Your toolsbuiltin or MCP model serverOVMS /v3decodes tool_calls model.deviceIntel CPU, GPU, NPUOpenVINO runtime no tool callsAnswer+ sources, trace
1 / 8
  1. You type one command. The CLI hands your file to _load_config(), the one place a workflow is ever loaded.
  2. The YAML becomes a checked object. Unknown keys are errors, so a typo like max_iteration fails by name instead of being ignored.
  3. The factory wires the agent. It builds the model client, the tools and the optional document search, then returns one of four engines.
  4. The engine asks the model. It sends the conversation plus a menu of tools to OVMS, which speaks the OpenAI chat format on /v3.
  5. The model runs on your hardware. OVMS uses OpenVINO on the device you named. Its tool parser turns the model's text into a list of tool calls.
  6. The reply comes back. If it carries tool calls, the engine has work to do. If not, it is the answer.
  7. Your tool runs in Python, on your machine. The model never runs code; it only asks. The result goes into the conversation and the engine asks the model again.
  8. No tool calls means done. OVAT prints the answer, lists the source files it saw, and can write a trace of every turn.

Six terms, so the rest makes sense

Agent

A language model that can ask for your functions to be run. It reads the result and keeps going until it can answer.

Tool call

The model's request to run a function, written as structured text: a name plus arguments. Your code runs it, never the model.

Tool parser

The piece inside the model server that reads that text and turns it into a real call. Qwen3.5 needs qwen3coder; Qwen3 needs hermes3.

OVMS

OpenVINO Model Server. It loads the model, runs it on CPU, GPU or NPU and answers like the OpenAI API. OVAT starts and stops it for you.

openvino_genai

The library that runs a model inside your own Python process. OVAT uses it for local chat with no server, including on a Mac.

Intel AI PC

A laptop or desktop with an Intel CPU, an Arc GPU and an NPU, a chip built for AI work. OVAT routes each model to the right one.

The loop every project rewrites, replaced by a file

On the left is the smallest working tool-calling agent against a local server, from the OVAT walkthrough. It still has no iteration cap, no error handling, no history and no server management. On the right is the same agent in OVAT.

By handPython, about 30 lines
agent.py
from openai import OpenAI
client = OpenAI(base_url="http://localhost:8000/v3", api_key="not-needed")

tools = [{"type": "function", "function": {
    "name": "search_docs",
    "description": "Search the user's local documents",
    "parameters": {"type": "object",
                   "properties": {"query": {"type": "string"}},
                   "required": ["query"]}}}]   # hand-written schema

messages = [{"role": "user", "content": question}]
while True:                                    # the loop, by hand
    r = client.chat.completions.create(
        model="Qwen3.5-4B-int4-ov", messages=messages,
        tools=tools, tool_choice="auto")
    choice = r.choices[0]
    if choice.finish_reason != "tool_calls":
        print(choice.message.content)
        break
    messages.append(choice.message)
    for call in choice.message.tool_calls:     # dispatch, by hand
        args = json.loads(call.function.arguments)  # may be malformed
        result = run_my_tool(call.function.name, args)
        messages.append({"role": "tool",
                         "tool_call_id": call.id,
                         "content": str(result)})
With OVATYAML, then one command
workflow.yml
model:
  name: Qwen3.5-4B-int4-ov
  device: GPU
  tool_parser: qwen3coder
tools:
  - name: search_docs
    type: builtin
agent:
  type: native
  max_iterations: 10
terminal
$ ovat serve workflow.yml
$ ovat run workflow.yml -i "what do my notes say about Q3?"
  • The loop, with a max_iterations cap so a confused model cannot spin forever.
  • Tool schemas, read from each tool's own SCHEMA, never written twice.
  • Broken tool arguments, sent back to the model as a sentence it can fix.
  • The server: ovat serve starts OVMS, --stop ends it.
  • Silent failures, like a tool call that was never decoded, named in the trace.

The loop, the tool schemas, the history, the iteration cap and the error messages become the toolkit's job. Moving from a 16 GB GPU box to an 8 GB CPU laptop is a three-line edit: compare examples/workflow.yml with examples/minimal.yml.

four-engines

Change one word, keep everything else

agent.type picks the loop that drives your tools. The model, the tools and the prompt stay the same, and all four talk to the same server through one shared description of it.

Four engines, one backend native, react, llamaindex and openai-agents each connect to LLMBackend, one shared description of the OVMS connection, which talks to the same OVMS server. agent/loop.pynative LangChainreact LlamaIndexllamaindex OpenAI Agents SDKopenai-agents providers/backend.pyLLMBackendone shared description the same serverOVMS /v3same model, same tools
workflow.yml
agent:
  type: native
What
OVAT's own loop in agent/loop.py. No extra library.
Run
--native
Install
Included in pip install ovat
Tokens
Prompt and completion tokens from OVMS's usage field, per turn, plus tool counts.

measured

What the AI PC runs showed

Every figure below was measured on an Intel LunarLake AI PC (Arc 140V GPU and NPU) and is written down in the OVAT repo with how it was taken.

56/60vs 42/60 on OVMS 2026.2.1

Right answers, 15 tool questions through all four engines, on the pinned OVMS 2026.4.1. McNemar p = 0.0001.

4.9 GBvs 17.3 GB before

Peak memory of the GPU model server across the same run, after moving to 2026.4.1.

NPUtool calls work

A real tool-calling agent run on the NPU, from a channel-wise INT4 export (-int4-cw-ov).

800+tests, 4 CI platforms

Ubuntu 22.04 and 24.04, Windows and macOS on every push. No server needed to run them.

See every measurement

Build on a Mac, serve on an AI PC

OVMS ships for x86-64 Windows and Linux only. On macOS everything except serving works, and ovat chat answers from your documents with a local model and no server.

Platform support
CommandmacOSWindows 11Linux
init doctor index setup telemetryYesYesYes
chat and the terminal UIYesYesYes
serve models run benchNoYesYes
GPU and NPUCPU onlyVerifiedUntested