Instant Local Voice Pipeline

Speaks and Listens Without the Cloud · Shipped · workstation, home

On-device streaming ASR and TTS with sub-second latency, wake-word detection, barge-in, and speaker identification.

The entire speech pipeline runs locally on Sherpa-ONNX neural models. The system runs two concurrent tracks: a Speech Track (VAD → Wake Word Spotter → Streaming ASR → Speaker ID) and an Ambient Track (Energy Gate → Audio Tagger → Anomaly Detector). TTS with rapid barge-in support means Halbert stops speaking the instant you start.

Audio ingress adapters capture PCM from local microphones, Wyoming protocol satellites, RTSP streams, and WebRTC. Chunks feed into an async ring buffer. The speech track processes voice activity detection, wake word spotting, streaming transcription, and speaker identification in sequence. Results emit as VoiceTurnObservation events.

Limits and invariants

Push-to-talk by default: continuous ambient listening requires explicit owner opt-in. Gracefully degrades if sherpa-onnx is not installed.

Where this lives

• halbert_core/halbert_core/audio/pipeline.py
• halbert_core/halbert_core/audio/speech/
• halbert_core/halbert_core/audio/ingress/
• halbert_core/halbert_core/audio/buffer.py
• halbert_core/halbert_core/audio/config.py