mstar.api_server.openai.realtime_vad

mstar.api_server.openai.realtime_vad#

Server turn detection for /v1/realtime (OpenAI’s server_vad).

TurnSegmenter cuts a 16 kHz stream into turns from a per-frame speech probability, and returns what to do in stream order: start a turn, add audio to it, stop it, commit it. A commit covers exactly the frames up to the end of the turn’s silence window; audio after that point, even in the same append, is segmented for the next turn, so a turn end inside one append never drops the start of the next utterance, and each sample is committed exactly once.

The probability comes from Silero VAD (MIT), its 2 MB ONNX export run by onnxruntime on one CPU thread, about 0.15 ms per 32 ms frame. Its weights are read from disk only: MSTAR_SILERO_VAD_PATH, else the copy bundled with the silero-vad package (pip install 'mstar[vad]').

Functions

samples_to_ms(n)

silero_factory()

Make one SileroVAD per session over a model loaded once, or return None (with one warning) when the weights or onnxruntime are missing.

silero_weights()

The Silero ONNX file this server uses, or None.

Classes

ServerVAD([threshold, prefix_padding_ms, ...])

OpenAI's server_vad settings.

SileroVAD(session)

Silero VAD for one stream.

SpeechProbability(*args, **kwargs)

Per-session speech probability of one FRAME of 16 kHz audio; stateful.

TurnSegmenter(vad, cfg, max_turn_samples[, pos])

Splits a 16 kHz stream into turns (OpenAI server_vad semantics).
class mstar.api_server.openai.realtime_vad.ServerVAD(threshold=0.5, prefix_padding_ms=300, silence_duration_ms=500, create_response=True, interrupt_response=True)[source]#

Bases: object

OpenAI’s server_vad settings. create_response and interrupt_response are accepted and echoed; a transcription session has no response to create or interrupt.

Parameters:

  • threshold (float)

  • prefix_padding_ms (int)

  • silence_duration_ms (int)

  • create_response (bool)

  • interrupt_response (bool)

create_response: bool = True#
interrupt_response: bool = True#
prefix_padding_ms: int = 300#
silence_duration_ms: int = 500#
threshold: float = 0.5#
view()[source]#
Return type:

dict

class mstar.api_server.openai.realtime_vad.SileroVAD(session)[source]#

Bases: object

Silero VAD for one stream. The ONNX session is shared and stateless; the stream keeps the model’s recurrent state and the last CONTEXT samples, which the model reads before each frame.

CONTEXT = 64#
reset()[source]#
Return type:

None

class mstar.api_server.openai.realtime_vad.SpeechProbability(*args, **kwargs)[source]#

Bases: Protocol

Per-session speech probability of one FRAME of 16 kHz audio; stateful.

reset()[source]#
Return type:

None

class mstar.api_server.openai.realtime_vad.TurnSegmenter(vad, cfg, max_turn_samples, pos=0)[source]#

Bases: object

Splits a 16 kHz stream into turns (OpenAI server_vad semantics).

feed takes audio of any length and returns operations in stream order: ("speech_started", audio_start_ms), ("audio", samples) for the open turn, ("speech_stopped", audio_end_ms), ("commit",). Times are milliseconds from the start of the session’s audio. A turn starts on a frame at or above threshold and takes up to prefix_padding_ms of the audio before it; it ends after the frame that brings silence_duration_ms of whole frames, counted from a frame below threshold - HYSTERESIS (included) with none at or above threshold since, whatever that last frame’s probability (the frames belong to the turn). A turn holds at most max_turn_samples: it is committed before the frame that would pass that, and in mid-speech the next turn starts with that frame. Audio between turns, beyond the prefix, is in no turn. A partial frame waits for the next feed.

Parameters:
end_turn()[source]#

A client commit: an open turn takes the partial frame and stops there. Between turns the partial frame and the prefix padding are dropped, so audio from before the commit never reaches the next turn.

Return type:

list[tuple]

feed(audio)[source]#
Parameters:

audio (ndarray)

Return type:

list[tuple]

reset(skipped=0)[source]#

A clear: drop buffered audio, keep the timeline (skipped more samples were dropped before they reached the segmenter).

Parameters:

skipped (int)

Return type:

None

shift(n)[source]#

Move the stream position by n samples (the session’s timeline correction after a resampler start-over, with no partial frame held).

Parameters:

n (int)

Return type:

None

take_over(old)[source]#

Go on from old between turns (a settings change): its stream position, its partial frame, not scored yet, and its prefix padding.

Parameters:

old (TurnSegmenter)

Return type:

None

unscored()[source]#

The partial frame, not scored yet.

Return type:

ndarray

mstar.api_server.openai.realtime_vad.samples_to_ms(n)[source]#
Parameters:

n (int)

Return type:

int

mstar.api_server.openai.realtime_vad.silero_factory()[source]#

Make one SileroVAD per session over a model loaded once, or return None (with one warning) when the weights or onnxruntime are missing.

Return type:

Callable[[], SpeechProbability] | None

mstar.api_server.openai.realtime_vad.silero_weights()[source]#

The Silero ONNX file this server uses, or None.

Return type:

Path | None