VVoxGraph

LLM providers

Ollama (recommended for development)

  • No API quota — runs on your GPU/CPU
  • Full multi-turn chat history via /api/chat
  • Tune context with OLLAMA_NUM_CTX and reply length with OLLAMA_NUM_PREDICT
LLM_PROVIDER=ollama
OLLAMA_MODEL=qwen2.5:7b
OLLAMA_NUM_CTX=8192
OLLAMA_NUM_PREDICT=256

Smaller models (llama3.2:1b) are faster but weaker on context; 7B models balance quality and speed for voice demos.

Google Gemini

LLM_PROVIDER=gemini
GOOGLE_API_KEY=your_key
GEMINI_MODEL=gemini-2.5-flash

Uses a single prompt with episodic text block. Automatic model fallbacks if the primary model fails. Watch free-tier rate limits in production.

Customizing prompts

Edit build_system_prompt() and build_gemini_prompt() in voxgraph/providers/llm.py. The system prompt enforces:

  • Assistant vs user identity ("your name" vs "my name")
  • Stay on the current conversation thread
  • Voice-friendly reply length