mstar.model.components.audio_features

mstar.model.components.audio_features#

Whisper-family log-mel front end, in torch.

Whisper, Qwen3-ASR’s AuT and the Higgs audio tower all consume the same features: a 16 kHz waveform, 25 ms Hann-windowed STFT at a 10 ms hop, a Slaney-scaled mel filterbank (80 or 128 bins), log10, an 8-unit dynamic range clamp under the clip’s maximum, then (x + 4) / 4. This module reproduces HF’s WhisperFeatureExtractor torch path to ~1e-5 so a natively served encoder sees the features its HF oracle was tested with, without transformers in the serving path. It runs wherever the waveform is, on the CPU of the API-server data worker or on the GPU inside an encoder’s forward (every op is fixed-shape for a fixed clip length, so it captures into a CUDA graph).

Functions

load_audio_file(path[, sample_rate])

Decode an audio file to a float32 mono waveform at sample_rate.

sinusoid_positions(length, channels[, ...])

Whisper's fixed sinusoidal position table, (length, channels): [sin | cos] halves over geometrically spaced timescales.

slaney_mel_filter_bank(num_frequency_bins, ...)

Area-normalized triangular mel filters, (num_mel_filters, num_frequency_bins) in float64 — the transpose of HF's layout, so the forward is one filters @ power_spectrogram matmul.

Classes

LogMelSpectrogram([num_mel_bins, ...])

(B, samples) or (samples,) float waveform -> (B, n_mels, frames) log-mel features, frames = samples // hop_length.
class mstar.model.components.audio_features.LogMelSpectrogram(num_mel_bins=128, sampling_rate=16_000, n_fft=400, hop_length=160, chunk_length=30)[source]#

Bases: Module

(B, samples) or (samples,) float waveform -> (B, n_mels, frames) log-mel features, frames = samples // hop_length.

Matches HF’s _torch_extract_fbank_features: torch.stft with center=True yields samples // hop + 1 frames and the last is dropped, and the dynamic-range clamp is per clip. Whisper pads or truncates to chunk_length seconds before this transform; use pad_or_trim() for that.

Parameters:

  • num_mel_bins (int)

  • sampling_rate (int)

  • n_fft (int)

  • hop_length (int)

  • chunk_length (int)

extra_repr()[source]#

Return the extra representation of the module.

To print customized extra information, you should re-implement this method in your own modules. Both single-line and multi-line strings are acceptable.

Return type:

str

forward(waveform)[source]#

Define the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

Parameters:

waveform (Tensor)

Return type:

Tensor

num_frames(num_samples)[source]#
Parameters:

num_samples (int)

Return type:

int

pad_or_trim(waveform, length=None)[source]#

Zero-pad or truncate the last dim to length samples (the 30 s window by default), as Whisper’s extractor does before the STFT.

Parameters:
Return type:

Tensor

mstar.model.components.audio_features.load_audio_file(path, sample_rate=16_000)[source]#

Decode an audio file to a float32 mono waveform at sample_rate.

libsndfile (soundfile) first — it needs no FFmpeg and covers WAV, FLAC, OGG and MP3 — then torchcodec for anything else (M4A, WebM, …). Resampling goes through torchaudio.

Parameters:

  • path (str)

  • sample_rate (int)

Return type:

Tensor

mstar.model.components.audio_features.sinusoid_positions(length, channels, max_timescale=10_000.0)[source]#

Whisper’s fixed sinusoidal position table, (length, channels): [sin | cos] halves over geometrically spaced timescales. The checkpoint ships it as embed_positions.weight; this is for models (Qwen3-ASR’s AuT) that compute it instead.

Parameters:
Return type:

Tensor

mstar.model.components.audio_features.slaney_mel_filter_bank(num_frequency_bins, num_mel_filters, sampling_rate, min_frequency=0.0, max_frequency=8000.0)[source]#

Area-normalized triangular mel filters, (num_mel_filters, num_frequency_bins) in float64 — the transpose of HF’s layout, so the forward is one filters @ power_spectrogram matmul.

Parameters:

  • num_frequency_bins (int)

  • num_mel_filters (int)

  • sampling_rate (int)

  • min_frequency (float)

  • max_frequency (float)

Return type:

ndarray