omniloader.data.npy

A dataset over a per-sample .npy folder layout.

NpyFolderDataset reads the common feature-extraction layout where each sample is a directory of .npy files, one per named value:

root/
  <sample_key>/
    video.npy
    audio.npy
    valence.npy
    ...

Arrays are memory-mapped (mmap_mode="r") so large features are not copied into RAM until sliced. Every sample directory is expected to contain the same set of files; the first sample defines the key set unless keys is given.

Classes

NpyFolderDataset

Read samples from a directory of per-sample .npy sub-folders.

Module Contents

class omniloader.data.npy.NpyFolderDataset(root: str | pathlib.Path, keys: collections.abc.Sequence[str] | None = None, mmap: bool = True)

Bases: torch.utils.data.Dataset

Read samples from a directory of per-sample .npy sub-folders.

Parameters:

• root – Directory whose immediate sub-directories are samples.

• keys – Optional value names (.npy stems) to load. When None, the files present in the first sample directory define the key set.

• mmap – Memory-map the arrays instead of reading them fully. Defaults to True.

Raises:

FileNotFoundError – If root has no sample sub-directories.

__len__() → int

Number of sample directories.

__getitem__(index: int) → dict[str, Any]

Load the sample at index as a dict of tensors plus its key.

Parameters:

index – Sample position.

Returns:

A dict mapping each key to a tensor, plus a key metadata string (the sample directory name).