Open source · MIT · Python 3.10+ · by Surya Raj

Voice AI pipeline you can fork, not rebuild

VoxGraph is an open-source Python voice AI starter kit by Surya Raj: WebSocket audio in, Deepgram STT, LLM (Gemini or Ollama), TTS out — episodic memory, barge-in, browser demo, and Docker deploy. Clone the repo, add your API keys, then customize STT, LLM, TTS, memory, and your client.

Try it live Quick start Read the docs

terminal

# Terminal 1 — server

python -u -m uvicorn voxgraph:app --host 0.0.0.0 --port 8001

# Terminal 2 — live mic

python scripts/live_mic_client.py --demo --url ws://127.0.0.1:8001/audio

Hey VoxGraph — what can you help me build today?

How it works

Real-time voice loop — from microphone to spoken reply in one WebSocket session.

Mic Input

linear16 PCM

WebSocket

/audio

Deepgram

STT

LLM

Ollama / Gemini

Deepgram

Aura TTS

Audio Out

24 kHz stream

Why developers use VoxGraph

A hackable foundation — pin your own features on top and ship fast.

Real-time audio

WebSocket /audio accepts linear16 PCM at 16 kHz. TTS streams back at 24 kHz.

Pluggable LLM

Google Gemini in the cloud or Ollama locally — swap via one env var.

Built-in memory

Semantic facts plus rolling episodic turns injected into every chat.

Starter kit

Not a black box — fork, add intents, tools, and your own client UI.

Demo mode

Low-latency debounce and endpointing tuned for live conversation.

Real-time audio

WebSocket /audio accepts linear16 PCM at 16 kHz. TTS streams back at 24 kHz.

Pluggable LLM

Google Gemini in the cloud or Ollama locally — swap via one env var.

Built-in memory

Semantic facts plus rolling episodic turns injected into every chat.

Starter kit

Not a black box — fork, add intents, tools, and your own client UI.

Demo mode

Low-latency debounce and endpointing tuned for live conversation.

Stack

Everything you need to run a voice agent locally or in production.

FastAPIDeepgram STTDeepgram Aura TTSOllama / GeminiSQLiteWebSockets

VoxGraph is a starter kit, not a hosted product. Clone the repo, add your API keys, then customize prompts, voice intents, memory, and your front-end.

How to extend