HermesVoice
Talk to a local or remote LLM with your voice. Native macOS, on-device speech, zero dependencies.
HermesVoice is a native macOS app, styled after the J.A.R.V.I.S. HUD, that holds a spoken conversation with any OpenAI-compatible backend: a Hermes Agent instance from Nous Research, OpenAI, Ollama or LM Studio. Speech recognition and speech synthesis both run on-device with Apple’s Speech and AVFoundation frameworks, so the audio never leaves the Mac — only the transcribed text is sent to the model. It is written in Swift and SwiftUI with no third-party dependencies at all: no SPM packages, no CocoaPods, nothing to vendor.
How it works
- MicrophoneShared AVAudioEngine tap
- RecognitionSFSpeechRecognizer, on-device
- ModelPOST /v1/chat/completions, SSE
- Sentence bufferSpeaks on the first full sentence
- SynthesisAVSpeechSynthesizer, on-device
The five states
- ReadySlow breath, dim blue
- ListeningFilaments follow the mic
- ProcessingFast pulse, sweeping arc
- SpeakingWaves follow the voice
- ErrorContracted, flickering
What is interesting inside
On-device speech, both directions
SFSpeechRecognizer transcribes and AVSpeechSynthesizer speaks, using the Siri voices already installed on the machine. The language picker only lists languages the Mac can both hear and speak, and flags the ones that would need to leave the device to be transcribed.
Streams as the model thinks
The reply arrives as server-sent events and goes through a sentence buffer, so the app starts speaking on the first complete sentence instead of waiting for the full answer. Talk over it and it stops mid-sentence — barge-in reads the same audio buffers as the recognizer.
One audio engine for everything
A single shared AVAudioEngine serves the microphone tap, playback and metering. That is what makes the waveform a real FFT of the live audio (the mic while you talk, the synthesized voice while it answers) and what makes stopping instant.
Any language, switchable mid-conversation
One setting drives recognition, synthesis and the language the model replies in. The system prompt stays monolingual and appends a directive naming the reply language, so adding a language costs nothing but a voice the Mac already has.
Spec-first, one pull request per task
The architecture, API contract and UI were written down in SPEC.md before the code, then built as 15 tasks with acceptance criteria, one PR each. The README screenshots are rendered from the real SwiftUI views with ImageRenderer inside the test suite, so they cannot drift out of date with the UI.
Built with
WHY IT EXISTSI wanted to know how far a voice agent gets with no cloud speech services and no dependencies — the answer is: the whole loop, on a laptop, with the only running cost being the model itself. The state machine, the barge-in handling and the sentence buffering are the parts that transfer directly to client work.
When you need judgment,
not just code.
Not every engagement needs someone to take the whole product. Sometimes you already have a team and need someone who has shipped this before to look at it. A decision nobody can settle, a stack under evaluation, a roadmap that feels wrong, an estimate you do not trust. A decade of shipping across web, mobile and AI left me something more useful than a stack: judgment about what is worth building.
