personalitylinmult.inference.audio_features

Extract the audio features the A / AT / AVT models consume, from a media file.

The trained models take pre-extracted per-modality features, not raw audio. To predict on a new clip a caller must first run the same extractors used to build fi.h5. This module wraps the two acoustic ones — WavLM and emotion2vec — so a demo (notebook or Space) can go from an uploaded file to a features dict the predictor accepts, without importing the batch preprocessing scripts.

exordium (the extractor stack) is imported lazily inside the functions so this module stays importable where those heavy deps are absent (e.g. a unit-test environment that only exercises the predictor).

Attributes

logger	Module-level logger.
WAVLM_LAYER	WavLM transformer layer used everywhere in this project (matches fi.h5).
AUDIO_SUFFIXES	Extensions treated as audio; anything else is decoded through ffmpeg first.
AUDIO_SAMPLE_RATE	Sample rate the training audio was extracted at.

Exceptions

AudioExtractionError

Raised when audio cannot be decoded from an input file.

Functions

ensure_audio(→ tuple[pathlib.Path, bool])	Return a WAV path for the input, extracting audio from video if needed.
extract_wavlm(→ numpy.ndarray)	Extract the WavLM layer-9 feature sequence for one clip.
extract_emotion2vec(→ numpy.ndarray)	Extract the emotion2vec+ feature sequence for one clip.
extract_audio_features(→ dict[str, numpy.ndarray])	Extract both acoustic features for the audio (A) model.

Module Contents

personalitylinmult.inference.audio_features.logger

Module-level logger.

personalitylinmult.inference.audio_features.WAVLM_LAYER = 9

WavLM transformer layer used everywhere in this project (matches fi.h5).

personalitylinmult.inference.audio_features.AUDIO_SUFFIXES

Extensions treated as audio; anything else is decoded through ffmpeg first.

personalitylinmult.inference.audio_features.AUDIO_SAMPLE_RATE = 44100

Sample rate the training audio was extracted at.

exception personalitylinmult.inference.audio_features.AudioExtractionError

Bases: Exception

Raised when audio cannot be decoded from an input file.

Initialize self. See help(type(self)) for accurate signature.

personalitylinmult.inference.audio_features.ensure_audio(input_path: str | pathlib.Path) → tuple[pathlib.Path, bool]

Return a WAV path for the input, extracting audio from video if needed.

Audio files are passed through unchanged; any other container (video) is decoded to a temporary mono 44.1 kHz WAV with the same ffmpeg settings the training pipeline used.

Parameters:

input_path (str | Path) – An audio or video file.

Returns:

(wav_path, is_temporary) – the caller deletes the WAV when is_temporary is True.

Return type:

tuple[Path, bool]

Raises:

AudioExtractionError – If ffmpeg fails or the input has no audio stream.

personalitylinmult.inference.audio_features.extract_wavlm(audio_path: str | pathlib.Path, device_id: int = -1) → numpy.ndarray

Extract the WavLM layer-9 feature sequence for one clip.

Parameters:

• audio_path (str | Path) – Path to an audio/video file exordium can read.

• device_id (int) – Device index; -1 for CPU (portable default).

Returns:

(T, 768) float32 features.

Return type:

np.ndarray

personalitylinmult.inference.audio_features.extract_emotion2vec(audio_path: str | pathlib.Path, device_id: int = -1) → numpy.ndarray

Extract the emotion2vec+ feature sequence for one clip.

Parameters:

• audio_path (str | Path) – Path to an audio/video file exordium can read.

• device_id (int) – Device index; -1 for CPU.

Returns:

(T, 768) float32 features.

Return type:

np.ndarray

personalitylinmult.inference.audio_features.extract_audio_features(audio_path: str | pathlib.Path, device_id: int = -1) → dict[str, numpy.ndarray]

Extract both acoustic features for the audio (A) model.

Accepts audio or video: a video’s audio track is extracted to a temporary mono 44.1 kHz WAV (matching the training pipeline) before feature extraction, then cleaned up.

Parameters:

• audio_path (str | Path) – Path to an audio or video file.

• device_id (int) – Device index; -1 for CPU.

Returns:

{"wavlm": (T, 768), "emotion2vec": (T, 768)}, the exact input the A model’s predictor expects.

Return type:

dict[str, np.ndarray]

Raises:

AudioExtractionError – If the input is a video with no decodable audio.