Written by
Tim Janke
,
Head of Machine Learning
Product
/
The fastest way to break a voice agent is another voice. A colleague talking nearby, a TV in the background, residual echo – to a classic voice activity detection model (VAD), all of it is speech, and every trigger feeds your turn-taking logic. The agent stops mid-sentence for a voice on the radio, or waits patiently for a "user" who was never talking to it.
Today we are releasing VAD Voice Focus 2.0, a standalone VAD model that triggers only when the primary speaker is talking.
Why a primary-speaker VAD matters
A VAD identifies when speech is present in an audio stream, and voice agents rely on that signal for turn-taking, end-pointing and deciding when to send audio to speech-to-text (STT). Classic VAD models are built to trigger on all audible speech – by design, they cannot tell the user apart from a voice in the background.
When the VAD fires on background speech, the agent hears turns that never happened. The result is false barge-ins, premature end-of-turn decisions and audio segments sent to STT that pollute the conversation context. In production audio – call centers, drive-throughs, shared offices, homes – interfering speech is common, and a VAD that treats every voice as the user makes the whole pipeline unstable.
Until now, getting a VAD signal for the primary speaker only meant chaining two models: run Quail Voice Focus to isolate the primary speaker, then run a VAD on the enhanced output. VAD Voice Focus 2.0 collapses that chain into a single lightweight model.
What counts as the primary speaker
People talk to voice agents in all kinds of ways. Some hold the phone to their ear, others put the call on speakerphone, speak to a laptop from across the desk or address an assistant from elsewhere in the room. Often there is no competing voice at all – just one speaker who happens to be far from the microphone.
VAD Voice Focus 2.0 follows the same definition of the primary speaker as Quail Voice Focus: it identifies the dominant speaker in the scene without treating distance from the microphone as the deciding factor.
When a single speaker is the only voice present, near or far, the VAD recognizes them as the primary speaker and triggers reliably, so distance does not work against them.
When other voices compete, only the primary speaker triggers the VAD. Interfering speech is ignored.
Performance
We evaluated VAD Voice Focus 2.0 on an internal benchmark of challenging real-world scenes, including competing speakers, background media and far-field speech. It compares three setups: VAD Voice Focus 2.0 on the raw audio, a standard VAD on the same audio, and the chained alternative - Quail Voice Focus preprocessing followed by a standard VAD.
VAD Voice Focus 2.0 detects primary-speaker activity with 92.4% accuracy, compared to 86.5% for Quail Voice Focus preprocessing followed by a standard VAD and 71.0% for a standard VAD alone. The false positive rate shows the same pattern: 5.5% for VAD Voice Focus 2.0, 12% for the chained setup and 28.5% for a standard VAD alone. Fewer false triggers means fewer turns started by voices that were never talking to the agent – with one model in the pipeline instead of two.
Try VAD Voice Focus 2.0 in the ai-coustics SDK
With this release, the SDK offers two standalone VAD models, and the right choice depends on what you need for your use case. VAD 2.0 is the general-purpose option: highly noise-robust detection of any speech, for pipelines where every voice matters. VAD Voice Focus 2.0 is built for one-to-one interaction, where a single speaker drives turn-taking and other voices are ignored.
Option | Triggers on | Use when |
|---|---|---|
VAD 2.0 | Any speech | You need general, noise-robust speech detection |
VAD Voice Focus 2.0 | Primary speaker only | Turn-taking should follow a single primary speaker |
VAD Voice Focus 2.0 runs in real time on CPU through AirTen, our CPU-first inference runtime, with 30 ms latency and no GPU or ONNX dependency. It operates at 16 kHz and stays robust at lower sample rates, including 8 kHz telephony audio. Three tunable parameters – sensitivity, speech hold duration and minimum speech duration – let you adapt the trigger behavior to your turn-taking logic.
VAD Voice Focus 2.0 is available now in the ai-coustics SDK. Try the SDK for free or read the docs.
