A local-first agent runtime in C++20.
Chat with a model that can read your files and run commands, transcribe speech, speak, draw and make 3D meshes, all offline.
Website · Docs · Releases · Issues · Sponsor
- Local. Prompts, files and voice recordings never leave the computer. Qwen, Llama, Gemma and any other GGUF model.
- Modular. One folder holds everything. llama.cpp, audio.cpp, stable-diffusion.cpp, onnxruntime and trellis.cpp come from their own releases at pinned versions, with SHA-256 checks, and the GPU backend (CUDA, ROCm, Vulkan, Metal) follows the machine.
- More than chat. Speech in and out, image and video generation, image to 3D, MCP servers, skills, scheduled graphs.
- Safe by default. Commands and file changes ask first and run in an OS sandbox.
- A C++ library too. One header, no exceptions, no virtual calls, no allocation in the per-turn path. The core adds under a microsecond per turn (benchmarks).
Linux and macOS:
curl -fsSL https://agent.qozu.fyi/install.sh | shWindows (PowerShell):
irm https://agent.qozu.fyi/install.ps1 | iexThe installer asks for one folder (default ~/.agent.cpp, stored as the environment variable AGENT_HOME), installs the program, then
offers the engines (llama, audiocpp, onnx, sdcpp, trellis), the runtimes (bun, uv, node, pnpm, nub) and a first model. Everything is
downloaded for this machine at the version the release pins, and the GPU backend follows the driver found. Later: agent engine pull NAME,
agent runtime install NAME, agent update, agent uninstall.
| Supported | |
|---|---|
| Linux | x86_64 and aarch64, glibc 2.39 or newer |
| Windows 11 | x64 and arm64 |
| macOS | Apple silicon and Intel, macOS 14 or newer |
The installer offers what exists for the machine and says why anything is missing.
| Engine | Linux x86_64 | Linux aarch64 | Windows x64 | Windows arm64 | macOS arm64 |
|---|---|---|---|---|---|
| llama (chat, embeddings, rerank, vision, decision models) | CPU, Vulkan, CUDA 12 / 13, ROCm | CPU, Vulkan, CUDA 13 | CPU, Vulkan, CUDA 12 / 13, ROCm | CPU, Vulkan, CUDA 13 | Metal |
| onnx | CPU, CUDA 12 / 13 | CPU | CPU, CUDA 12 / 13 | CPU | CPU |
| audiocpp (speech) | CPU, Vulkan, CUDA; glibc 2.43 or newer | no | CPU, Vulkan, CUDA | no | Metal |
| sdcpp (images, video) | CPU, Vulkan, ROCm; glibc 2.43 or newer | no | CPU, Vulkan, CUDA 12, ROCm | no | Metal |
| trellis (image to 3D) | CPU, Vulkan, CUDA 12 / 13, ROCm | no | CPU, Vulkan, CUDA 12 / 13, ROCm | no | no |
Every engine library is the upstream project's own release (llama.cpp, audio.cpp, stable-diffusion.cpp, onnxruntime, trellis.cpp) at the version the release pins; agent.cpp adds a small plugin over each.
agentSessions are grouped by project folder. The bottom bar shows CPU, RAM, the loaded model and context use, and the mic box starts voice input.
| Key | Action |
|---|---|
/ or Ctrl+P |
command palette: /model, /image, /open, /stats, /doctor, /clear, /help |
Ctrl+N, Ctrl+K |
new chat, switch session |
Ctrl+B |
hide or show the sidebar |
F1 |
every shortcut; Settings > Shortcuts rebinds them |
Esc |
stop an answer in progress |
Running a command, or deleting or moving a file, always asks first. Tools run inside an OS sandbox (Landlock and seccomp on Linux, an AppContainer on Windows 11).
Every capability is one command, and all of them run offline.
| Goal | Command |
|---|---|
| Talk to the agent by voice | agent voice |
| Transcribe a recording | agent task asr --file talk.wav |
| Read text aloud | agent task tts --text "Hello" |
| Generate or edit an image, make a short video | agent task image_gen |
| Turn a picture into a 3D mesh | agent task mesh_gen |
| Run a repeatable workflow, resumable | agent run flow.json |
| Add models, MCP servers and skills | agent model pull NAME, agent mcp, agent skill |
| Check the installation, free disk space | agent doctor, agent clean |
| Remove agent.cpp and its data folder | agent uninstall (--keep-data keeps the models and conversations) |
agent --help lists every command. The full reference is docs/cli.md.
#include <agent-cpp/agent.hpp>
int main() {
agent::AgentRuntime runtime;
auto session = agent::open_session(agent::engines::llama, "models/qwen2.5-0.5b-instruct-q4_k_m.gguf");
if (!session.ok) return 1;
agent::Conversation chat;
return agent::run_turn(session.value, chat, "Say hi in five words.").ok ? 0 : 1;
}In the library, engines run in the calling process unless Config::engine_isolation says otherwise (the agent program runs llama, audiocpp and sdcpp each in its own sandboxed host process, because their ggml libraries clash in one). Start with the
library guide. The API is documented under docs/api.md, and runnable programs
are in examples/.
Requires CMake 3.21+, Ninja and a C++20 compiler (MSVC 2022, GCC 12+ or Clang 15+).
git clone --recurse-submodules https://github.com/amitverma-cf/agent.cpp
cd agent.cpp
cmake -B build -G Ninja
cmake --build build
ctest --test-dir build --output-on-failure
./build/agentWindows prerequisites and build options: docs/getting-started.md.
| Topic | Page |
|---|---|
Using the agent command |
CLI reference, getting started, workspace layout, voice assistant, GPU support |
| Embedding the library | library guide, usage guide, API reference, C API, choosing parts |
| Extending | writing an engine, a tool, a graph |
| Background | architecture, packaging, sandbox, benchmarks, comparison |
Everything is indexed in docs/index.md. Contributing: CONTRIBUTING.md. Security reports: SECURITY.md.
Questions and bugs go to issues. If agent.cpp is useful to you, a star helps, and sponsoring supports its development.