LMStudio Local Models Compatibility

[xerudro]

This tool should be compatible to run with local lm agents thru Ollama or LMStudio also. Why all the tools runs on paid LMs ? We have hardware to run them locally also.

[lschneidpro]

@xerudro — agentmemory already runs against Ollama / LM Studio / vLLM / llama.cpp / any OpenAI-API-compatible local server. The docs gap is real though — the only mention was buried in a one-line comment inside the env-example block, so it was easy to miss.

Quick copy-paste for the two most common setups:

Ollama (default port 11434):

ollama pull qwen2.5-coder:7b
ollama serve

# ~/.agentmemory/.env
OPENAI_API_KEY=ollama
OPENAI_BASE_URL=http://localhost:11434/v1
OPENAI_MODEL=qwen2.5-coder:7b

LM Studio (default port 1234):

Open LM Studio → Local Server tab → Start Server (any chat model in the picker).

# ~/.agentmemory/.env
OPENAI_API_KEY=lmstudio
OPENAI_BASE_URL=http://localhost:1234/v1
OPENAI_MODEL=qwen2.5-coder-7b-instruct

Restart agentmemory and the consolidation pipeline, compression, summarization, and graph extraction all run against your local server. Zero paid LLM calls. Embeddings are local-by-default too (@xenova/transformers ships a BGE-small that runs on-device — no extra config).

For memory work specifically, a 7B instruct model (Qwen 2.5 Coder, Llama 3.2, Mistral, DeepSeek-R1) is plenty — compression is short summarization, not full reasoning. The 3B/7B size range fits on consumer hardware (4-5 GB RAM) and runs faster than the paid APIs.

I've opened #697 to add a dedicated "Local models (Ollama / LM Studio / vLLM)" section under the LLM Providers docs so the next person doesn't have to ask. Includes the model-pick table and a callout for the reasoning-model empty-content pitfall (some local servers don't surface the reasoning field, so o1-style models can look like they return blank — switch to a non-reasoning model if extractions come back empty).

Heads-up if you're using a configured provider but seeing no graph nodes / lessons / crystals: that's a separate bug (consolidation defaulting to off). Fix landing in #696 — until that merges, set CONSOLIDATION_ENABLED=true in your .env.

[lschneidpro]

@rohitg00

Thanks for the clarification on local models.

While testing, I noticed that AgentMemory wasn’t picking up the default local embeddings via @xenova/transformers. I also came across this PR:
#412

Has that issue been fixed through another PR since then, or is the current recommendation to rely on a self-hosted embedding provider as well?

Thanks!

[rohitg00]

I have not released new version, this is fixed in new version.