personalitylinmult.inference.audio_features_onnx

Extract WavLM features via ONNX Runtime – no torch/transformers.

The ONNX Runtime-backed counterpart to extract_wavlm(). Decoding still shells out to ffmpeg (a subprocess, not a Python dependency) to produce mono 16 kHz PCM WAV directly – WavLM’s own required sample rate – avoiding torchaudio, which would otherwise pull torch back in just for resampling. Reading the WAV uses only the standard library’s wave module plus numpy, both already required.

Known limitation: ffmpeg’s resampler and torchaudio’s Resample transform (used by the PyTorch path, via exordium.audio.wavlm.WavlmWrapper) use different filter implementations, so they do not produce bit-identical 16 kHz waveforms from the same source file (~3e-4 mean, ~2.5e-2 max absolute sample difference, measured on a real clip). WavLM’s transformer layers can amplify this at a handful of frames, but the final Big Five predictions shift only ~0.001-0.004 per trait end to end – small and bounded, not exact. Accepted deliberately to keep this module genuinely torch-free.

Attributes

logger

Module-level logger.

WAVLM_ONNX_SAMPLE_RATE

WavLM's required input sample rate, matching exordium's

Exceptions

AudioDecodeError

Raised when ffmpeg cannot decode the input into a WAV.

Functions

load_waveform_16k(→ numpy.ndarray)

Load any audio/video file as a mono 16 kHz waveform, via ffmpeg.

extract_wavlm_onnx(→ numpy.ndarray)

Extract WavLM layer features using an ONNX Runtime session.

extract_emotion2vec_onnx(→ numpy.ndarray)

Extract emotion2vec+ features using an ONNX Runtime session.

Module Contents

personalitylinmult.inference.audio_features_onnx.logger

Module-level logger.

personalitylinmult.inference.audio_features_onnx.WAVLM_ONNX_SAMPLE_RATE = 16000

WavLM’s required input sample rate, matching exordium’s WAVLM_SAMPLE_RATE – see onnx_export.py’s WAVLM_MODEL_ID.

exception personalitylinmult.inference.audio_features_onnx.AudioDecodeError[source]

Bases: Exception

Raised when ffmpeg cannot decode the input into a WAV.

Initialize self. See help(type(self)) for accurate signature.

personalitylinmult.inference.audio_features_onnx.load_waveform_16k(input_path: str | pathlib.Path) → numpy.ndarray[source]

Load any audio/video file as a mono 16 kHz waveform, via ffmpeg.

Parameters:

input_path (str | Path) – An audio or video file.

Returns:

(samples,) float32 waveform in [-1, 1], at WAVLM_ONNX_SAMPLE_RATE.

Return type:

np.ndarray

personalitylinmult.inference.audio_features_onnx.extract_wavlm_onnx(input_path: str | pathlib.Path, session: onnxruntime.InferenceSession) → numpy.ndarray[source]

Extract WavLM layer features using an ONNX Runtime session.

Parameters:

  • input_path (str | Path) – An audio or video file.

  • session (onnxruntime.InferenceSession) – A session over the graph export_wavlm_to_onnx() produced.

Returns:

(T, 768) float32 features, matching extract_wavlm()’s output for the same input.

Return type:

np.ndarray

personalitylinmult.inference.audio_features_onnx.extract_emotion2vec_onnx(input_path: str | pathlib.Path, session: onnxruntime.InferenceSession) → numpy.ndarray[source]

Extract emotion2vec+ features using an ONNX Runtime session.

Reuses load_waveform_16k() – emotion2vec+, like WavLM, expects a 16 kHz mono waveform, so the same decode step covers both.

Parameters:

  • input_path (str | Path) – An audio or video file.

  • session (onnxruntime.InferenceSession) – A session over the graph export_emotion2vec_to_onnx() produced.

Returns:

(T, 768) float32 features, matching exordium.audio.emotion2vec.Emotion2vecWrapper’s output for the same input.

Return type:

np.ndarray