blinklinmult.data.writer¶

Write samples straight into a corpus’s HDF5 file, as they are produced.

The 1.x pipeline staged every sample as a .npy file plus a manifest, then a second pass read them back and assembled the HDF5. Measured on MRL-Eye, the staging cost 884 MB against an 11 MB h5 — 80x the artifact, for a copy that existed only to be read once and deleted.

This writer removes that layer. A preprocessing run opens one file, appends each sample as it is cut, and closes it: one command, one artifact, and no intermediate that can drift out of step with either end.

The file is complete or it is absent. Samples go to <name>.h5.tmp, renamed only on a clean close. An interrupted run leaves nothing behind rather than a partial file that a later run would mistake for a finished one.

The layout, dtypes, compression, and root attributes are exactly those blinklinmult.data.builder writes — that contract is what OmniLoader reads, and it does not change.

Attributes¶

`logger`	Module-level logger.
`COMPRESSION`	Compression filter, paired with the shuffle filter for float data.
`COMPRESSION_LEVEL`	a middle ground between file size and read CPU.
`IMAGE_DTYPE`	On-disk dtype for eye crops and descriptors.
`LABEL_DTYPE`	On-disk dtype for targets. Labels are tiny; precision costs nothing.

Exceptions¶

WriterError

Raised when a corpus cannot be written as configured.

Classes¶

H5Writer

Streams one corpus's samples into its HDF5 file.

Module Contents¶

blinklinmult.data.writer.logger¶: Module-level logger.

blinklinmult.data.writer.COMPRESSION = 'gzip'¶: Compression filter, paired with the shuffle filter for float data.

blinklinmult.data.writer.COMPRESSION_LEVEL = 4¶

a middle ground between file size and read CPU.

Type:: Compression level

blinklinmult.data.writer.IMAGE_DTYPE¶

On-disk dtype for eye crops and descriptors.

The crops dominate the file — a 15x3x64x64 window is 184k values against 15 labels — and fp16 halves both the file and the read bandwidth at a precision finer than 8-bit pixel data carries in the first place.

blinklinmult.data.writer.LABEL_DTYPE¶: On-disk dtype for targets. Labels are tiny; precision costs nothing.

exception blinklinmult.data.writer.WriterError¶

Bases: RuntimeError

Raised when a corpus cannot be written as configured.

Initialize self. See help(type(self)) for accurate signature.

class blinklinmult.data.writer.H5Writer(spec: blinklinmult.data.schema.DatasetSpec, h5_path: pathlib.Path, config_yaml: str = '', git_sha: str = 'unknown', extra_attrs: dict[str, Any] | None = None)¶

Streams one corpus’s samples into its HDF5 file.

Use as a context manager: the file is renamed into place on a clean exit and discarded on an exception.

Parameters:

spec (DatasetSpec) – The corpus declaration, which fixes the shapes and which targets are written.
h5_path (Path) – Destination file.
config_yaml (str) – The corpus config, embedded for traceability.
git_sha (str) – Commit the build was made from.
extra_attrs (dict | None) – Further root attributes describing how this corpus was sampled — the evaluation protocol and its stride, say. A file read out of context should still be able to say what it is.

Raises:

WriterError – If a sample disagrees with the declaration.

__enter__() → H5Writer¶

Open the temporary file and write the root attributes.

Returns:: This writer.
Return type:: H5Writer

__exit__(exc_type, exc_value, traceback) → None¶

Close the file, renaming it into place only on success.

Parameters:

exc_type – Exception type, if one is propagating.
exc_value – The exception.
traceback – Its traceback.

add(subset: str, video_id: str, frame_group: str, eye_side: str, eye_images: numpy.ndarray, eye_image_mask: numpy.ndarray | None = None, blink_presence: numpy.ndarray | None = None, blink_presence_mask: numpy.ndarray | None = None, eye_state: numpy.ndarray | None = None, eye_state_mask: numpy.ndarray | None = None, eye_features: numpy.ndarray | None = None, eye_feature_mask: numpy.ndarray | None = None, blink_id: int = NO_BLINK, head_pose: numpy.ndarray | None = None, quality_signals: dict[str, numpy.ndarray] | None = None, blink_ids: numpy.ndarray | None = None, quality: dict[str, numpy.ndarray | float | str] | None = None) → str¶

Write one eye-wise sample.

Parameters:

subset (str) – Split this sample belongs to.
video_id (str) – Source recording or subject.
frame_group (str) – Identifier shared by both eyes of one window, which is what lets frame-level scoring recombine them.
eye_side (str) – Which eye.
eye_images (np.ndarray) – (T, C, H, W) normalised crops.
eye_image_mask (np.ndarray | None) – (T,) validity; None means every frame is real.
blink_presence (np.ndarray | None) – (T,) target.
blink_presence_mask (np.ndarray | None) – (T,) validity.
eye_state (np.ndarray | None) – (T,) target.
eye_state_mask (np.ndarray | None) – (T,) validity.
eye_features (np.ndarray | None) – (T, F) descriptors.
eye_feature_mask (np.ndarray | None) – (T,) validity.
blink_id (int) – The blink this window contains, or NO_BLINK.
head_pose (np.ndarray | None) – (T, 3) [yaw, pitch, roll] in degrees. Required when the corpus declares has_head_pose.
quality_signals (dict[str, np.ndarray] | None) – Each of QUALITY_SIGNALS as a (T,) array in [0, 1]. Stored separately rather than combined, so the weighting stays a dataloader decision.
blink_ids (np.ndarray | None) – (T,) which annotated blink each frame belongs to, NO_BLINK outside every event.
quality (dict | None) – Optional diagnostics stored verbatim alongside the sample – per-frame (T,) arrays such as EYE_ON_SCREEN, the FACE_BOX, or scalars like CONFIDENCE. Nothing here is read during training; it exists so bad data can be found and looked at. Per-frame arrays are fitted to the window length exactly as the images are.

Returns:

The sample key.

Return type:

str

Raises:

WriterError – If the writer is closed, the subset is unknown, the key collides, or an array disagrees with the declaration.