personalitylinmult.data.dataset

Random-access reads from the built FI HDF5 file.

This dataset touches the HDF5 file and nothing else — no feature directories, no label CSV, no pickles. That is what makes a training run reproducible from a single published artifact, and what keeps __getitem__ down to one chunk read per feature.

Analysis window. The file stores whole clips; the window is applied here, on read, using each feature’s rate_hz. Changing window_seconds from 15 to 10 to 5 is a config edit, not a rebuild, and the crop is pushed into the HDF5 read itself so windowed-away data never crosses the bus.

Worker safety. h5py handles do not survive fork. The handle is therefore opened lazily inside whichever process first calls __getitem__, and stripped from the pickled state so spawn-based workers construct their own.

Attributes

logger

Module-level logger.

NO_RATE_SENTINEL

rate_hz attribute value meaning "no wall-clock time axis".

Exceptions

DatasetError

Raised when the HDF5 file cannot serve the requested configuration.

Classes

FeatureView

Resolved read plan for one feature within one dataset.

FiHdf5Dataset

Reads FI samples for one split out of the built HDF5 file.

Module Contents

personalitylinmult.data.dataset.logger

Module-level logger.

personalitylinmult.data.dataset.NO_RATE_SENTINEL = -1.0

rate_hz attribute value meaning “no wall-clock time axis”.

exception personalitylinmult.data.dataset.DatasetError[source]

Bases: RuntimeError

Raised when the HDF5 file cannot serve the requested configuration.

Initialize self. See help(type(self)) for accurate signature.

class personalitylinmult.data.dataset.FeatureView(name: str, dim: int, stored_time_dim: int, window_time_dim: int, rate_hz: float | None, n_layers: int)[source]

Resolved read plan for one feature within one dataset.

Parameters:

  • name (str) – Feature name.

  • dim (int) – Feature width.

  • stored_time_dim (int) – Time dimension as stored in the file.

  • window_time_dim (int) – Time dimension after windowing.

  • rate_hz (float | None) – Native rate, or None if not wall-clock.

  • n_layers (int) – Layer count, 0 when there is no layer axis.

property is_multi_layer: bool

Whether reads carry a layer axis.

Returns:

True when the model should see (B, L, T, F).

Return type:

bool

property is_windowed: bool

Whether the analysis window crops this feature.

Returns:

True when the stored clip is longer than the window.

Return type:

bool

class personalitylinmult.data.dataset.FiHdf5Dataset(h5_path: str | pathlib.Path, subset: str, feature_names: collections.abc.Sequence[str], target_names: collections.abc.Sequence[str], window_seconds: float | None = None, crop_mode: str = 'head', seed: int = 42)[source]

Bases: torch.utils.data.Dataset

Reads FI samples for one split out of the built HDF5 file.

Parameters:

  • h5_path (str | Path) – The built dataset.

  • subset (str) – One of "train", "valid", "test".

  • feature_names (Sequence[str]) – Features to return, in model input order.

  • target_names (Sequence[str]) – Label columns to return, in head order.

  • window_seconds (float | None) – Analysis window. None reads whole clips.

  • crop_mode (str) – "head", "center", or "random". "random" is honoured only on the training split; evaluation always uses "center" so it stays deterministic.

  • seed (int) – Base seed for "random" cropping.

Raises:

  • DatasetError – If the file is missing, has the wrong schema version, or lacks a requested feature or target.

  • SchemaError – If subset or crop_mode is not recognised.

property handle: h5py.File

The process-local HDF5 handle, opened on first use.

Opening lazily is what makes the dataset fork-safe: each worker creates its own handle rather than inheriting a parent’s, which HDF5 does not support and which corrupts reads.

Returns:

The open file.

Return type:

h5py.File

__getstate__() → dict[str, Any][source]

Drop the file handle so the dataset can be pickled to a worker.

Returns:

Picklable state.

Return type:

dict

__len__() → int[source]

Number of clips in this split.

Returns:

Clip count.

Return type:

int

__getitem__(index: int) → dict[str, Any][source]

Read one clip.

Parameters:

index (int) – Row within this split.

Returns:

{name} and {name}_mask per feature, plus targets and video_id. Every feature is paired with a mask, True = valid.

Return type:

dict