A privacy-first, fully local AI assistant — voice, web UI, and CLI in one package.
🔒 100% local. No cloud. No telemetry. Your data never leaves your machine.
🚀 Quick Start · ✨ Features · 🏗️ Architecture · 📁 Project Structure · 🛠️ Tech Stack · 📖 Docs
Before installing, make sure you have:
- Python 3.11+ — python.org
- Ollama — ollama.com — must be running on
localhost:11434
# Start Ollama and pull a model
ollama serve
ollama pull llama3.2# Core install — text chat + web UI + API (no voice)
pip install jarvise
# With voice support (Whisper STT + Kokoro TTS)
pip install jarvise[voice]
# With Google Calendar / Gmail tools
pip install jarvise[google]
# With Microsoft Outlook tools
pip install jarvise[microsoft]
# Everything — voice, Google, Microsoft, browser, CLI
pip install jarvise[all]# 1. Copy the example env file and edit it
cp .env.example .env
# 2. Launch JARVIS
jarvisOpen http://localhost:8000 in your browser and start chatting.
# Clone the repo
git clone https://github.com/Ashutosh3021/JARVISE.git
cd JARVISE
# Create and activate a virtual environment
python -m venv .venv
# Windows
.venv\Scripts\activate
# Linux / macOS
source .venv/bin/activate
# Install dependencies
pip install -r requirements.txt
# Configure environment
cp .env.example .env
# Run
python main.pyStartup — hardware detection & boot sequence
JARVIS boots in under a second, detects your hardware, initializes the logger, and serves the UI — all locally.
|
Conversational AI powered by a ReAct agent loop — reasons step by step, uses tools, and responds with context awareness. No hallucinated shortcuts; it thinks before it speaks. Remembers you across sessions. ChromaDB stores vector embeddings of your conversations, while Speak naturally. Faster-Whisper transcribes your voice locally, and Kokoro TTS reads responses back. Fully offline — no API keys, no latency from the cloud. |
A polished React + TypeScript interface served at Power users can interact via a rich terminal interface — ideal for scripting, piping, or when you just prefer the keyboard. JARVIS exposes a clean Ask JARVIS "how's my CPU?" — the built-in |
graph TB
subgraph "🖥️ Interfaces"
UI[Web UI<br/>localhost:8000]
CLI[CLI Shell<br/>jarvis shell]
API[REST API<br/>/api/*]
end
subgraph "⚙️ JARVIS Core"
Backend[FastAPI Backend]
Agent[ReAct Agent<br/>brain/]
Memory[ChromaDB<br/>memory/]
Voice[Voice Pipeline<br/>voice/]
Tools[Tool Registry<br/>tools/]
end
subgraph "🤖 AI Layer"
Ollama[Ollama<br/>localhost:11434]
LLM[Llama 3.2]
end
UI -->|WebSocket + HTTP| Backend
CLI -->|HTTP| Backend
API -->|HTTP| Backend
Backend --> Agent
Agent --> Memory
Agent --> Voice
Agent --> Tools
Agent <-->|inference| Ollama
Ollama --- LLM
flowchart LR
subgraph "📥 Input"
MIC[🎙️ Microphone]
TXT[⌨️ Text / CLI]
WEB[🌐 Web UI]
end
subgraph "🔄 Processing"
STT[Whisper STT]
AGENT[ReAct Agent]
LLM[Ollama LLM]
end
subgraph "💾 Memory"
VEC[ChromaDB<br/>Vector Store]
FILE[MEMORY.md<br/>Key Facts]
end
subgraph "📤 Output"
TTS[Kokoro TTS 🔊]
RESP[Text Response 📝]
UI2[Web UI Update 🌐]
end
MIC --> STT --> AGENT
TXT --> AGENT
WEB --> AGENT
AGENT <--> VEC
AGENT <--> FILE
AGENT <-->|reason + act| LLM
AGENT --> TTS
AGENT --> RESP
AGENT --> UI2
JARVIS/
│
├── 🖥️ backend/ # FastAPI server, WebSocket handlers, routes
├── 🧠 brain/ # ReAct agent, chains, prompt templates
├── 💾 memory/ # ChromaDB vector store + MEMORY.md fact file
├── 🛠️ tools/ # Tool modules: browser, code_exec, system_monitor, etc.
├── 🎤 voice/ # STT (Faster-Whisper) + TTS (Kokoro) pipeline
├── 🌐 ui/ # React 18 + TypeScript + Vite frontend
├── 💻 cli/ # Python CLI package (argparse-based)
│
├── 📖 Docs/ # Extended documentation
├── 🧪 tests/ # Test suites and bug regression tests
│
├── main.py # 🚀 Application entry point
├── HowToRun.md # 📋 Step-by-step setup guide
├── .env.example # ⚙️ Environment template
└── requirements.txt # 📦 Python dependencies
| Layer | Technology | Purpose |
|---|---|---|
| Backend | FastAPI + Python 3.11+ | REST API, WebSocket, async server |
| AI Inference | Ollama (Llama 3.2) | Local LLM — no cloud needed |
| Agent Framework | ReAct loop | Step-by-step reasoning with tools |
| Memory / RAG | ChromaDB | Vector embeddings, semantic recall |
| Speech-to-Text | Faster-Whisper | Offline voice transcription |
| Text-to-Speech | Kokoro | Offline neural TTS |
| Frontend | React 18 + TypeScript + Vite | Snappy, modern web interface |
| CLI | Python (argparse) | Terminal interface |
JARVIS is configured via a .env file in the project root:
# Ollama
OLLAMA_HOST=http://localhost:11434
OLLAMA_MODEL=llama3.2:latest
# Server
UI_HOST=127.0.0.1
UI_PORT=8000
# Memory
CHROMA_PERSIST_DIR=./memory/chroma
MEMORY_FILE=./memory/MEMORY.md
# Voice (optional)
VOICE_ENABLED=true
WHISPER_MODEL=base# Full app — backend + web UI
python main.py
# Backend API only (no UI)
python -m backend.main
# CLI interactive shell
jarvis shell
# or
python -m cli shell
# CLI — single query
jarvis chat "What's the weather like today?"# Full test suite
pytest tests/
# Specific bug regression test
PYTHONIOENCODING=utf-8 python tests/Bugs_Testing/B1Test.py
# Verbose output
pytest tests/ -v --tb=short- Create a new module in
tools/your_tool.py - Implement the tool interface (see
tools/README.md) - Register it in
brain/tool_registry.py - The ReAct agent will automatically discover and use it
| Problem | Fix |
|---|---|
jarvis command not found |
Reinstall: pip install jarvise — check entry points in pyproject.toml |
| Ollama not responding | Run ollama serve and confirm ollama list shows your model |
| Port already in use | Set UI_PORT=8001 in .env |
| VRAM / memory issues | Switch to a smaller model: OLLAMA_MODEL=llama3.2:1b |
| Voice not working | Install voice extras: pip install jarvise[voice] |
| Document | Description |
|---|---|
| HowToRun.md | Prerequisites, installation, and first-run walkthrough |
| publish.md | PyPI deployment guide |
| Docs/ | Architecture deep-dives, API reference, tool guides |
- Multi-model support (swap LLMs without restart)
- Plugin system for third-party tools
- Mobile-responsive web UI improvements
- Long-term memory summarization
- Wake-word detection for hands-free activation
Contributions are welcome. Please open an issue first to discuss what you'd like to change. For bug fixes, feel free to submit a PR directly.
MIT License — see LICENSE for full details.
Built for privacy. Designed for speed. Made to be yours.
🤖 JARVIS — Your Personal AI Assistant