One YAML file.
One command.
A local agent.
OVAT turns a tool-calling AI agent into a config file. You describe the model, the tools and the behaviour in workflow.yml, run ovat run, and the agent answers on your own Intel hardware. No API keys, no cloud, nothing leaves the machine.
ovat bench, then the terminal UI chatting with audio, image and document tools.
One note: some answers in the four-engine run misread the memory budget as 128 MB. The note they cite says a full run fits in 8 GB.
What a run prints
The commands from the guide, replayed as text: a RAG answer with its source, a dry run that needs no server, and ovat doctor on a Mac.
how-it-works
How one question travels through OVAT
Follow the light. A question enters as a command, becomes a checked config, gets wired into an agent, and loops between the model and your tools until there is an answer. Press any step to stop on it.
- You type one command. The CLI hands your file to
_load_config(), the one place a workflow is ever loaded. - The YAML becomes a checked object. Unknown keys are errors, so a typo like
max_iterationfails by name instead of being ignored. - The factory wires the agent. It builds the model client, the tools and the optional document search, then returns one of four engines.
- The engine asks the model. It sends the conversation plus a menu of tools to OVMS, which speaks the OpenAI chat format on
/v3. - The model runs on your hardware. OVMS uses OpenVINO on the device you named. Its tool parser turns the model's text into a list of tool calls.
- The reply comes back. If it carries tool calls, the engine has work to do. If not, it is the answer.
- Your tool runs in Python, on your machine. The model never runs code; it only asks. The result goes into the conversation and the engine asks the model again.
- No tool calls means done. OVAT prints the answer, lists the source files it saw, and can write a trace of every turn.
Six terms, so the rest makes sense
- Agent
A language model that can ask for your functions to be run. It reads the result and keeps going until it can answer.
- Tool call
The model's request to run a function, written as structured text: a name plus arguments. Your code runs it, never the model.
- Tool parser
The piece inside the model server that reads that text and turns it into a real call. Qwen3.5 needs
qwen3coder; Qwen3 needshermes3.- OVMS
OpenVINO Model Server. It loads the model, runs it on CPU, GPU or NPU and answers like the OpenAI API. OVAT starts and stops it for you.
- openvino_genai
The library that runs a model inside your own Python process. OVAT uses it for local chat with no server, including on a Mac.
- Intel AI PC
A laptop or desktop with an Intel CPU, an Arc GPU and an NPU, a chip built for AI work. OVAT routes each model to the right one.
The loop every project rewrites, replaced by a file
On the left is the smallest working tool-calling agent against a local server, from the OVAT walkthrough. It still has no iteration cap, no error handling, no history and no server management. On the right is the same agent in OVAT.
from openai import OpenAI
client = OpenAI(base_url="http://localhost:8000/v3", api_key="not-needed")
tools = [{"type": "function", "function": {
"name": "search_docs",
"description": "Search the user's local documents",
"parameters": {"type": "object",
"properties": {"query": {"type": "string"}},
"required": ["query"]}}}] # hand-written schema
messages = [{"role": "user", "content": question}]
while True: # the loop, by hand
r = client.chat.completions.create(
model="Qwen3.5-4B-int4-ov", messages=messages,
tools=tools, tool_choice="auto")
choice = r.choices[0]
if choice.finish_reason != "tool_calls":
print(choice.message.content)
break
messages.append(choice.message)
for call in choice.message.tool_calls: # dispatch, by hand
args = json.loads(call.function.arguments) # may be malformed
result = run_my_tool(call.function.name, args)
messages.append({"role": "tool",
"tool_call_id": call.id,
"content": str(result)})
model:
name: Qwen3.5-4B-int4-ov
device: GPU
tool_parser: qwen3coder
tools:
- name: search_docs
type: builtin
agent:
type: native
max_iterations: 10
$ ovat serve workflow.yml
$ ovat run workflow.yml -i "what do my notes say about Q3?"
- The loop, with a
max_iterationscap so a confused model cannot spin forever. - Tool schemas, read from each tool's own
SCHEMA, never written twice. - Broken tool arguments, sent back to the model as a sentence it can fix.
- The server:
ovat servestarts OVMS,--stopends it. - Silent failures, like a tool call that was never decoded, named in the trace.
The loop, the tool schemas, the history, the iteration cap and the error messages become the toolkit's job. Moving from a 16 GB GPU box to an 8 GB CPU laptop is a three-line edit: compare examples/workflow.yml with examples/minimal.yml.
four-engines
Change one word, keep everything else
agent.type picks the loop that drives your tools. The model, the tools and the prompt stay the same, and all four talk to the same server through one shared description of it.
agent:
type: native- What
- OVAT's own loop in
agent/loop.py. No extra library. - Run
--native- Install
- Included in
pip install ovat - Tokens
- Prompt and completion tokens from OVMS's
usagefield, per turn, plus tool counts.
agent:
type: react- What
- LangChain's
create_agentwith aChatOpenAImodel pointed at OVMS. - Run
--reactor--langchain- Install
pip install "ovat[langchain]"- Tokens
- Prompt and completion tokens from OVMS's
usagefield, summed per run from LangChain'susage_metadata, plus tool counts.
agent:
type: llamaindex- What
- LlamaIndex's
FunctionAgentwith anOpenAILikemodel. Bothis_chat_modelandis_function_calling_modelmust be set, or it quietly stops calling tools. - Run
--llamaindex- Install
pip install "ovat[llamaindex]"- Tokens
- Prompt and completion tokens from OVMS's
usagefield, summed per run, plus tool counts. Requests are sent unstreamed so every reply carries its usage.
agent:
type: openai-agents- What
- The OpenAI Agents SDK, kept on your machine by three settings: an explicit client on the
/v3URL, the Chat Completions model, and tracing turned off. - Run
--openai-agentsor--openai-sdk- Install
pip install "ovat[openai-agents]"- Tokens
- Prompt and completion tokens from OVMS's
usagefield, summed per run fromraw_responses, plus tool counts.
measured
What the AI PC runs showed
Every figure below was measured on an Intel LunarLake AI PC (Arc 140V GPU and NPU) and is written down in the OVAT repo with how it was taken.
Right answers, 15 tool questions through all four engines, on the pinned OVMS 2026.4.1. McNemar p = 0.0001.
Peak memory of the GPU model server across the same run, after moving to 2026.4.1.
A real tool-calling agent run on the NPU, from a channel-wise INT4 export (-int4-cw-ov).
Ubuntu 22.04 and 24.04, Windows and macOS on every push. No server needed to run them.
Build on a Mac, serve on an AI PC
OVMS ships for x86-64 Windows and Linux only. On macOS everything except serving works, and ovat chat answers from your documents with a local model and no server.
| Command | macOS | Windows 11 | Linux |
|---|---|---|---|
init doctor index setup telemetry | Yes | Yes | Yes |
chat and the terminal UI | Yes | Yes | Yes |
serve models run bench | No | Yes | Yes |
| GPU and NPU | CPU only | Verified | Untested |