HermesVoice
Talk to a local or remote LLM with your voice. Native macOS, on-device speech, zero dependencies.
HermesVoice is a native macOS app, styled after the J.A.R.V.I.S. HUD, that holds a spoken conversation with any OpenAI-compatible backend: a Hermes Agent instance from Nous Research, OpenAI, Ollama or LM Studio. Speech recognition and speech synthesis both run on-device with Apple’s Speech and AVFoundation frameworks, so the audio never leaves the Mac — only the transcribed text is sent to the model. It is written in Swift and SwiftUI with no third-party dependencies at all: no SPM packages, no CocoaPods, nothing to vendor.
How it works
- MicrophoneShared AVAudioEngine tap
- RecognitionSFSpeechRecognizer, on-device
- ModelPOST /v1/chat/completions, SSE
- Sentence bufferSpeaks on the first full sentence
- SynthesisAVSpeechSynthesizer, on-device
The five states
- ReadySlow breath, dim blue
- ListeningFilaments follow the mic
- ProcessingFast pulse, sweeping arc
- SpeakingWaves follow the voice
- ErrorContracted, flickering
What is interesting inside
On-device speech, both directions
SFSpeechRecognizer transcribes and AVSpeechSynthesizer speaks, using the Siri voices already installed on the machine. The language picker only lists languages the Mac can both hear and speak, and flags the ones that would need to leave the device to be transcribed.
Streams as the model thinks
The reply arrives as server-sent events and goes through a sentence buffer, so the app starts speaking on the first complete sentence instead of waiting for the full answer. Talk over it and it stops mid-sentence — barge-in reads the same audio buffers as the recognizer.
One audio engine for everything
A single shared AVAudioEngine serves the microphone tap, playback and metering. That is what makes the waveform a real FFT of the live audio (the mic while you talk, the synthesized voice while it answers) and what makes stopping instant.
Any language, switchable mid-conversation
One setting drives recognition, synthesis and the language the model replies in. The system prompt stays monolingual and appends a directive naming the reply language, so adding a language costs nothing but a voice the Mac already has.
Spec-first, one pull request per task
The architecture, API contract and UI were written down in SPEC.md before the code, then built as 15 tasks with acceptance criteria, one PR each. The README screenshots are rendered from the real SwiftUI views with ImageRenderer inside the test suite, so they cannot drift out of date with the UI.
Built with
WHY IT EXISTSI wanted to know how far a voice agent gets with no cloud speech services and no dependencies — the answer is: the whole loop, on a laptop, with the only running cost being the model itself. The state machine, the barge-in handling and the sentence buffering are the parts that transfer directly to client work.
When you need judgment,
not just code.
More than a decade shipping product across web, mobile and AI left me something more valuable than a stack: judgment. If your team is stuck on a technical decision, evaluating a stack, or wants a second opinion before sinking months into a direction, let's talk.
