WebSocket API

Connect to ws://HOST:PORT/audio (default port 8000 or 8001 in docs examples).

Client → server (binary)

Send raw PCM frames continuously while the user is speaking:

Server → client (binary)

TTS audio chunks during a reply:

Server → client (JSON text)

Status and fallback text when voice fails:

{ "type": "status", "message": "Thinking…" }
{ "type": "status", "message": "Speaking…" }
{ "type": "status", "message": "Ready — speak anytime" }
{ "type": "reply_text", "message": "Full LLM text if TTS failed" }
{ "type": "error", "message": "Human-readable error" }

Utterance lifecycle

1. Client streams PCM while user talks
2. Deepgram emits interim + final transcripts; speech_final marks end of phrase
3. Server debounces (~120 ms in demo mode) then merges segments into one user question
4. LLM + TTS pipeline runs; client should play binary chunks and show JSON status
5. While AI speaks, reference client ducks mic (optional barge-in on loud speech)

Building your own client

• Use any WebSocket library; set max_size=None for large TTS frames
• Run one task for send(mic_pcm) and one for recv() (duplex)
• Parse string frames as JSON; treat everything else as PCM for playback
• See scripts/send_test_pcm.py for TtsPlayback and continuous_tts_playback