orpheus GitHub ↗

real-time · voice-to-voice · runs on your GPU

Talk to it. It talks back.
Almost none of it leaves your machine.

orpheus is a voice-to-voice AI pipeline built for low enough latency to feel like a real conversation — VAD, transcription, and speech synthesis all run locally on your own GPU (tested on a 4GB GTX 1650), with just the LLM outsourced to a free, fast cloud API.

View on GitHub Watch the demo ↓

See it work

A real conversation, running on a 4GB GTX 1650 — including a mid-response interruption.

Why local

Most voice assistants ship your audio to someone else's server for every stage of the pipeline. orpheus flips that: voice activity detection, speech-to-text, and text-to-speech all run on-device, so your voice never leaves your machine except as text sent to the LLM. A 4GB card can't run a usable local LLM alongside STT+TTS, so that one piece is outsourced to Groq's free API — the hybrid that gets you privacy and low latency where it matters most (your ear and your voice) while keeping response quality high.

Stack

StageModelWhere it runs
Voice activity detectionSilero VADCPU
Speech-to-textMoonshine (ONNX)GPU
LLMGroq-hosted openai/gpt-oss-20bCloud
Text-to-speechKokoro (82M, ONNX)GPU

Engineering notes

~85%
cut in TTS time-to-first-audio (1.9s → 290ms) by flushing on clause boundaries instead of waiting for a full sentence.
barge-in
interrupt it mid-response and it actually stops, cancels the in-flight turn, and starts listening again — fixed a race that used to drop your first words.
4GB
the whole local stack (VAD + STT + TTS) has been run and tested on a 4GB GTX 1650.
capped
conversation history sent to the LLM is turn-capped so long sessions don't balloon the request payload over time.

Get started

py -3.11 -m venv .venv
.venv\Scripts\python.exe -m pip install -r requirements.txt
copy .env.example .env
# paste a free Groq API key into .env, then:
.venv\Scripts\python.exe cli.py

Full setup — including Kokoro model downloads and mic/speaker selection — is in the README.