Set up voice

Read this before you follow it

The audio subsystem starts, listens on a microphone uplink, and runs its pipeline. What it cannot do on a fresh install is notice that anyone spoke, understand them, or answer aloud — every one of those steps needs a model file that Halbert neither ships nor downloads. Until you put those files on the machine yourself, turning voice on gets you a running pipeline and silence.

This page says what works, shows how to check it on your own machine, and is honest about the rest.

What is in place and what is missing

Part	On a fresh install	Needs
Audio subsystem and pipeline	starts	the inference runtime, installed with the audio-inference extra
Browser microphone uplink	works	nothing
Voice activity detection	not working	a model file you supply
Speech recognition	not working	model files you supply
Text to speech	not working	a voice model file you supply
Wake word	not working	the audio-wake-word extra and a trained model file
Speaker identification	not working	a model file you supply
Privacy settings	stored, but not yet enforced	see step 5

Every line that says a model file you supply is the same missing directory. Fill it once and several of these come up together.

1. See what this machine has

$ TOKEN=$(cat ~/.local/state/halbert/api-token)
$ curl -s -H "Authorization: Bearer $TOKEN" http://localhost:8000/api/audio/status
{"enabled":false,"available":true,"sherpa_onnx_installed":true,"state":"idle",
 "engines":{"vad":true,"asr":true,"tts":false,"speaker_id":false,"audio_tagger":false}}

available and sherpa_onnx_installed are the same answer twice: is the inference runtime importable. If it is false, install the extra:

pip install 'halbert-core[audio-inference]'

vad and asr are not promises

Those two fields say the runtime is importable — and, once the pipeline is running, that an engine object was built. Neither engine opens its model files until the first utterance, so a machine with no model files at all still reports both as true. Step 4 is the check that actually answers the question. The tts field is different: it is reported only after the model really loaded, which is why it reads false here.

2. Turn it on

Everything audio is off by default, and each switch is separate. Settings → Audio & Voice has them, top to bottom: Enable audio subsystem is the master switch, and the rest — local microphone, satellite ingress, speaker identification, acoustic events, text to speech — are disabled until the master switch is on.

The same thing from the API:

$ curl -s -X POST http://localhost:8000/api/audio/config \
    -H "Authorization: Bearer $TOKEN" -H "Content-Type: application/json" \
    -d '{"enabled": true}'
{"status":"ok"}

That writes audio_config.yml in the configuration directory. Nothing else on the machine changes yet.

3. Restart

The audio pipeline is built once, when the dashboard starts. Turning the master switch on while the dashboard is running writes the file and nothing more; the status endpoint will report enabled: true and still have no pipeline behind it.

Restart the dashboard — see systemd units for a deployed instance — and the startup log says what came up:

WebRTC/dashboard ingress ready
Ingress registered: dashboard (area=dashboard_voice)
Starting audio pipeline...
Wake word model not found — hotkey-only activation
Audio pipeline started
Audio pipeline coordinator started (dashboard ingress attached)
Voice turn relay wired to the dashboard uplink

After the restart the status endpoint answers from the live pipeline rather than from the file, and grows the fields that only a running pipeline has:

$ curl -s -H "Authorization: Bearer $TOKEN" http://localhost:8000/api/audio/status
{"enabled":true,"running":true,"available":true,"state":"idle","speaker":null,
 "ingress_sources":[{"source_type":"dashboard","source_id":"mic:dashboard:dashboard_voice",
 "area_id":"dashboard_voice","running":true,"active_connections":0,"queue_size":0}],
 "engines":{"vad":true,"asr":true,"tts":false,"wake_word":false,"speaker_id":false,
 "audio_tagger":false}}

ingress_sources is the microphone uplink the voice screen opens from the browser. Audio is captured there, sent to this machine, and recognised here — recognition is never done in the browser — which is why the model files below decide whether any of it works.

4. The models are not included

Nothing in Halbert downloads a model, and none ship with it. This is the point at which a voice setup either continues or stops, so check it directly rather than trusting the status endpoint.

Voice activity detection and speech recognition. Both engines open their model files on first use, which is why the status endpoint cannot answer for them. From a clone, ask one directly:

python -c '
from halbert_core.audio.speech.asr_engine import StreamingASR
a = StreamingASR()
print("constructed:", a is not None)
a._ensure_initialized()'

On a machine with no model files, that prints constructed: True and then ends in a traceback whose last line is:

AssertionError: <data dir>/audio/models/<model dir>/tokens.txt does not exist

Which is the trap in two lines: the object exists, the status endpoint reports it, and the files are not there. The error names the exact directory the engine expects and the first file it looked for. Put the recognition model files there and the same command prints an initialisation line instead.

Substitute VoiceActivityDetector from halbert_core.audio.speech.vad for the same check on detection, which fails the same way and names a single .onnx file in the same directory. Until that one is in place nothing downstream ever runs, because the pipeline has no way to notice that anybody spoke.

Text to speech. A voice is three artefacts, not one: the model file, its tokens.txt, and an espeak-ng-data/ directory. Set tts.voice_model in audio_config.yml to the model file and leave the other two beside it — the layout published voice archives already use — or the startup log says exactly this:

WARNING  TTS engine constructed but its model will not load — no voice:
         ... Set tts.voice_model in audio_config.yml

With no voice that loads, engines.tts stays false and Halbert answers in text. That field is what decides whether a reply is spoken at all, so it fails towards silence rather than towards a turn that dies halfway through.

Wake word. Two things are missing, not one: the package, and a trained model file.

pip install 'halbert-core[audio-wake-word]'

The model file is not published. Without it, the startup line above — hotkey-only activation — is the permanent state: you start a turn by pressing something, never by saying a name.

Speaker identification. Also a model file you supply, at speaker_id.model or in the same models directory. Enrolment endpoints exist and the Settings tab has an enrolment dialog, but with no model to make an embedding from there is nothing to enrol into.

5. Privacy

The Settings tab offers three privacy switches, and audio_config.yml stores them: delete_raw_after_transcription and ignore_tv_media, both on by default, and quiet_hours, unset.

They are stored, not enforced

Nothing in the audio pipeline reads any of the three today. Setting them changes the file and nothing else. Do not treat a switch here as a control over what is captured.

What is true regardless of those switches, and is worth more than any of them:

• Captured audio is held in memory, not on disk. The speech path reads from a ring buffer and writes no audio file. The only code in the audio tree that writes a .wav at all belongs to the optional music-fingerprinting feature, which is off by default and needs a network service and a key of its own before it does anything.
• The satellite ingress, for microphones on other devices, listens on loopback only. It is not a service on your network, and pointing a device on the network at it will not work until you have deliberately changed that.

What is not there yet

• No spoken turn can complete on a fresh install. The capture, the uplink, the pipeline and the relay that carries a transcript back to the page are all wired; the detection and recognition in the middle have no models, so the turn ends before it starts.
• engines.vad and engines.asr report construction, not capability — see the caution in step
  1. Text to speech is reported honestly; the other two are not, and all three read identically in the same object.
• There is no model manager. No screen, no command and no endpoint fetches, verifies or lists voice models. Every path above is a file you place and a path you type.
• The privacy switches are stored and not enforced — see step 5.
• There is no voice command in the CLI. Voice is configured through the dashboard or through audio_config.yml, and nowhere else.
• audio_config.yml is not in the configuration reference. That page is generated, and no generator emits this file yet, so it says so and lists only being.yml.

Next

Capabilities, not variants explains why enabling audio turns a capability on with no flag edited in between — the probe behind it asks the same two questions this page opens with: is the switch on, and is the runtime installed.