/

/

Tyto 1.2: An audio decision model for Voice Agents

/

/

Tyto 1.2: An audio decision model for Voice Agents

Tyto 1.2: An audio decision model for Voice Agents

Tyto 1.2: An audio decision model for Voice Agents

Written by

Anil Keshwani

,

Senior Machine Learning Engineer

Product

/

Anil Keshwani

Written by

,

Senior Machine Learning Engineer

Product

/

Tyto 1.2 is our audio decision model for Voice Agents that listens to the human side of a call and scores the speaker’s audio. Tyto gives you a headline Risk Score, which tells you how likely this audio is to break turn-taking or speech-to-text (STT), and six dimensions that describe what it hears. Your application turns those seven numbers into decisions for the Voice Agent: whether to carry on, change how the agent takes turns, ask the caller to turn off a television, or hand the call to a person.

Decision models are the System One of an AI stack. They are small and fast, and they return a score or a choice where a language model would return text. Tyto brings that to audio. It runs on CPU beside your agent, needs no transcript and is cheap enough to run on every call.

Version 1.2 makes Tyto’s outputs more reliable and decisions made with it easier to trust.

One Risk Score, six dimensions

The Tyto Risk Score (risk_score) is the headline number. It indicates the risk the audio poses to VAD, turn-taking, STT or S2S. We break the Risk Score into three bands - Good, Warn and Bad - based on thresholds.

The six Tyto Dimensions sit alongside the score. Each one measures a single condition in the audio. Use the Risk Score to identify when there is a problem and the dimensions to decide which action to take.

THE SPEAKER

  • Speaker loudness speaker_loudness How loud the caller is.

  • Speaker reverb speaker_reverb Distance from the phone and echo from the room.

THEIR SURROUNDINGS

  • Interfering speech interfering_speech Other people talking, or a TV or radio nearby.

  • Noise noise Traffic, wind, music or a busy room.

THE CONNECTION

  • Packet loss packet_loss Dropouts and gaps in the audio.

  • Codec degradation codec_degradation Artifacts from heavy compression on the line.

Each score covers a 5-second window of the caller's audio. Tyto needs no transcript and no reference signal. It runs on CPU inside the ai-coustics SDK, so the audio stays in your infrastructure.

Tyto reads the audio and your logic leverages the Risk Score and dimensions to make the decision, with your thresholds and your actions.

What's new in version 1.2

Fewer false alarms. We significantly decreased the share of clean calls rated as Warn or Bad, to about 1 in 7. This means your Voice Agent won’t ask a user to repeat a phrase that was clear the first time.

Bad ratings you can act on. The share of Bad ratings that land on calls with real downstream errors rose to 74% in this release, making Bad ratings from Tyto 1.2 a more dependable signal.

More accurate dimensions. The interfering-speech dimension tracks the presence of background speakers more closely and stays near zero on calls where nobody else is talking. We also halved the error present Noise and Packet loss dimensions’ output.

Tyto 1.2 maintains the same compute cost and interface of 1.1.

249 calls · Tyto 1.2 Risk Score against measured riskGoodWarnBad0.00.00.20.20.40.40.60.60.80.81.01.0False alarmsTyto Risk Score →Measured risk →
Clean calls rated Warn or Bad14%17 of 122 calls
Calls rated Bad that are bad74%25 of 34 calls

Each dot is a call. Across, the Tyto Risk Score. Up, how much trouble the call caused downstream. Clean calls sit in the Good band.

System One for audio

A decision model returns a choice, a score or a probability where a language model would return text. Decision models like TypeSafe’s Jev read transcripts to determine user sentiment or intent. Tyto listens to the audio natively to identify problems with it and tell you how far the downstream parts of your Voice AI stack can trust it. This allows you to act on signals that other models lose, for use by Voice Agents while the caller is still on the line or by teams that review the calls afterwards.

Decisions during the call

A live call leaves a few seconds to react, so an in-call decision needs an input that is fast, cheap and already there. These are decisions a Voice Agent can make with Tyto while the caller is on the line.

Decision

Tyto output

What the agent does

How to take turns

interfering_speech is high

Turns off barge-in or waits for a longer utterance before it stops talking

Whether to ask for a quieter spot

interfering_speech or noise stays high

Asks the caller to move or to turn down what is playing nearby

Whether to ask for the handset

speaker_reverb is high and speaker_loudness is low

Asks the caller to take the phone off speaker or hold it closer

Whether to confirm what it heard

risk_score is Warn or Bad during a name, a number or a date

Reads the detail back before acting on it

When to hand off

risk_score stays Bad after the steps above

Transfers the call to a person with a note on what Tyto heard

Decisions after the call

After the call the same scores can be aggregated to give one record per call, for example taking the share of the call falling into the Warn or Bad bands. Teams that run Voice Agents review calls with LLM judges, human reviewers or sometimes both. Tyto helps decide where human effort is best placed and costs a fraction of an LLM judge.

Decision

Tyto output

What your team does

Which calls a person listens to

95th-percentile risk_score is Bad

Moves those calls to the top of the review queue

Whether to trust the transcript

risk_score was Warn or Bad while key details were spoken

Sends the recording for human transcription or a second check

Where a failure started

risk_score on a failed call

Low risk: checks prompts, tools and agent logic. High risk: checks the audio path first

This offline analysis is indispensable for teams whose Voice AI stacks are deployed in clinical, financial or legal contexts.

Take a look at our post-call demo to see how Tyto scores your own recordings (your data stays on your device).

Listen for yourself

Flag this call

Rescheduling a callback, with company

A real test call with a second person talking near the caller. Interfering speech averages 0.64. Risk is Bad in 12 of 24 windows and Warn in 8 more, so the whole call lands in Bad: the agent is likely to mishear or answer the wrong voice.

Caller audio, before any enhancement0:00 / 0:28
window at 0:02
risk 0.18 Good
Risk Score through the call
BadWarnGood0.300.500:000:050:100:150:200:25
Risk, call mean
0.52Bad
Risk, p95
0.78Bad
Call at Warn or worse
83%
Drivers
Interfering speech, speaker reverb

Each dot is one score. Test calls from the ai-coustics public test-call set, CC BY-NC 4.0, scored with tyto-1.2-l-16khz.

In the first test call a second person talks near the caller. Tyto 1.2 rates 83% of the call Warn or worse and interfering_speech and speaker_reverb are the highest dimensions. The second is a clean call on a quiet line, and every window scores below 0.06. Press play and follow the score through each call.

Good to know when you upgrade

  • Update the SDK and the model together. Tyto 1.2 ships in SDK 0.26.0 (Python aic-sdk 3.4.0) as tyto-1.2-l-16khz. SDK 0.26.0 needs the Tyto 1.2 model file, so download it with the update.

  • Recheck your thresholds. The bands are unchanged, but scores have shifted between versions. Thresholds you tuned on 1.1 need a second look.

  • Check your plugin version. The SDK covers Python, Node, Rust, C, C++ and WebAssembly.

  • LiveKit and Pipecat support. The LiveKit plugin runs Tyto 1.2 from version 0.26.0 and Pipecat from version 1.16.0.

Get started

Tyto 1.2 is available now. To make it easy to try, Tyto usage is free for all new and existing customers until Dec 15. It applies automatically, with no promo code needed. Score a few of your own calls and compare the Bad ratings with the calls your team already knows went wrong.

Final logo

Bring real-time audio intelligence into your voice AI stack

Bring real-time audio intelligence into your voice AI stack

Bring real-time audio intelligence into your voice AI stack