blinklinmult.train.config¶

Typed configuration for the blink training pipeline.

Configuration is plain YAML parsed into frozen dataclasses, split across three files per run — --data, --model, --train. Two properties motivate that over a framework:

The same DatasetSpec objects describe a corpus to the HDF5 builder and to the dataloader. A framework deriving its schema from class constructors (Lightning CLI / jsonargparse) could not span both, because the builder is not a Lightning class.
Nothing here changes the working directory or writes hidden output trees, so MLflow artifact paths stay predictable.

Unknown keys are rejected rather than ignored: a silently-dropped key produces a run whose recorded config does not describe what it did, which is worse than a crash. Retired 1.x model keys are rejected with an explicit rename hint.

Every knob a run depends on lives in one of these dataclasses, and ExperimentConfig.to_flat_dict() flattens the lot into MLflow params.

Attributes¶

`logger`	Module-level logger.
`TASKS`	The trainable tasks.
`TASK_TARGETS`	Target keys each task supervises.
`MODEL_FAMILIES`	Architecture families a model config may select.
`BACKBONES`	Image backbones that embed one eye crop.
`OPTIMIZERS`	Supported optimizers.
`SCHEDULERS`	Supported LR schedulers.
`LOSSES`	Supported losses. All operate on masked binary targets.
`PRECISIONS`	Lightning precision settings this project supports.

Exceptions¶

ConfigError

Raised when a configuration is invalid or self-contradictory.

Classes¶

`DataConfig`	Dataloader configuration.
`OptimConfig`	Optimizer and LR-schedule configuration.
`ModelConfig`	Architecture configuration.
`MLflowConfig`	MLflow tracking configuration.
`EarlyStoppingConfig`	Early-stopping configuration.
`TrainConfig`	Training-loop configuration.
`ExperimentConfig`	One run: its data, its model, and its training procedure.

Functions¶

`load_yaml`(→ dict[str, Any])	Parse a YAML file into a dict.
`apply_override`(→ None)	Set a nested config value addressed by a dotted key.
`parse_override`(→ tuple[str, Any])	Parse a `key=value` CLI override, typing the value via YAML.

Module Contents¶

blinklinmult.train.config.logger¶: Module-level logger.

blinklinmult.train.config.TASKS: frozenset[str]¶

The trainable tasks.

blink_presence and eye_state are the paper’s two tasks; joint trains both heads at once over the union of corpora, which is the case OmniLoader’s masking exists to support and the main reason for the v2 rewrite.

blinklinmult.train.config.TASK_TARGETS: dict[str, tuple[str, ...]]¶: Target keys each task supervises.

blinklinmult.train.config.MODEL_FAMILIES: frozenset[str]¶

Architecture families a model config may select.

linmult is the multimodal cross-modal transformer over eye images plus handcrafted features; lint is the single-stream transformer over eye images alone (the BlinkLinT baseline); cnn is the per-frame image backbone with no sequence model (the DenseNet121 baseline). All three are in the paper.

blinklinmult.train.config.BACKBONES: frozenset[str]¶

Image backbones that embed one eye crop.

convnext_femto is the default, chosen on measured CEW eye-state accuracy (96.6% over three seeds, against 94.4% for densenet121) at 4.8M parameters and roughly 4x the throughput. densenet121 is kept because it is the published 1.x model and the paper’s numbers refer to it; densenet121_truncated drops its last dense block, which holds 31% of the parameters and runs on a 2x2 map at this crop size.

blinklinmult.train.config.OPTIMIZERS: frozenset[str]¶: Supported optimizers.

blinklinmult.train.config.SCHEDULERS: frozenset[str]¶: Supported LR schedulers.

blinklinmult.train.config.LOSSES: frozenset[str]¶: Supported losses. All operate on masked binary targets.

blinklinmult.train.config.PRECISIONS: frozenset[str]¶

Lightning precision settings this project supports.

Validated here so a typo fails at config load with a readable message rather than deep inside the Trainer.

exception blinklinmult.train.config.ConfigError¶

Bases: ValueError

Raised when a configuration is invalid or self-contradictory.

Initialize self. See help(type(self)) for accurate signature.

blinklinmult.train.config.load_yaml(path: str | pathlib.Path) → dict[str, Any]¶

Parse a YAML file into a dict.

Parameters:: path (str | Path) – File to read.
Returns:: Parsed mapping.
Return type:: dict
Raises:: ConfigError – If the file is missing or is not a mapping.

class blinklinmult.train.config.DataConfig¶

Dataloader configuration.

Parameters:

eval_datasets (list[str]) –
Extra corpora to include in the test split only, on top of datasets. This is the cross-corpus generalisation protocol: train on what is annotated the way the model needs, then measure transfer to corpora never trained on. Empty means the test split matches datasets.

A corpus listed here that also appears in datasets is not duplicated – it simply contributes its own test split as usual.
datasets (list[str]) – Corpora to include, by name. Each must have a declaration under config/data/<name>.yaml and a built HDF5 file.
window_seconds (float | None) – Analysis window applied to every corpus, which each converts to its own frame count from its rate. None keeps each corpus’s declared window.
time_dim (int | None) – Shared window length in frames, overriding what the corpora derive. None takes the longest any of them needs, so none is truncated.
image_size (int | None) – Shared eye-crop size. None requires the corpora to agree.
strategy (str) – Mixing strategy across corpora.
strategy_kwargs (dict) – Strategy-specific arguments. Empty by default rather than carrying the default strategy’s knobs: a config that changed only strategy would otherwise pass the previous strategy’s arguments to one that does not accept them.
batch_size (int) – Samples per batch.
num_workers (int) – Dataloader worker processes.
pin_memory (bool) – Page-lock host batches for faster device copies.
persistent_workers (bool) – Keep workers (and their HDF5 handles) alive between epochs.
prefetch_factor (int) – Batches prefetched per worker.
cache_size (int) – Per-worker LRU sample cache; 0 disables.
preload (bool) – Read whole splits into RAM. Fast for the small corpora, ruinous for MRL-Eye.
seed (int) – Seed for mixing, shuffling, and augmentation.
stills (bool) – Serve single frames instead of windows, for the frame-wise model. Video corpora are reduced to their individual annotated frames at load time; the still corpora are already single-frame and pass through unchanged.
still_stride (int) – In still mode, keep every Nth open-eye frame of a window. Closed-eye frames are always kept.
still_open_to_closed (float) – In still mode, open-eye frames kept per closed-eye frame in the training split.
fit_fractions (dict[str, float]) –
Per-corpus fraction of the train and validation splits to keep, as {corpus: fraction} in (0, 1]. Corpora not named are kept whole.

Test is never touched. Capping it would score each arm on a different subset and make the arms incomparable, which is the whole reason for running them. That is the difference from train.limit_fit_batches, which caps every corpus at once: this thins one corpus while the rest stay full, which is what a mix dominated by a single large corpus needs. MPEblink is 63 758 of the 70 786 training windows here – 90% – so subsampling it alone changes the cost of an epoch by an order of magnitude while leaving the RN corpora, the headline benchmark, at full size.

The subsample is a deterministic stride over the sorted sample ids, so a run is reproducible and every arm sees the same subset.
embedding_cache (str | None) –
Directory of precomputed eye embeddings, or None to encode live.

A frozen encoder computes the identical vector for a given crop on every epoch, so every epoch after the first recomputes a constant. Caching turns 360 crops per batch into 8 small matrices: measured 0.74 s/batch frozen against milliseconds cached, and 3.4 GB of embeddings replacing 51 GB of crops.

Valid only with a frozen encoder and augmentation off, and both are refused rather than warned about. Augmentation perturbs the crop before the encoder, so a cache would freeze one augmented view per window forever – keeping the distortion and losing the variety. An unfrozen encoder changes weights every step, so the cache is stale after the first one. Either mistake trains on embeddings from a different model and reports a plausible, wrong number.

__post_init__() → None¶

Validate the dataloader configuration.

Raises:: ConfigError – If any value is out of range or unrecognised.

classmethod from_dict(data: dict[str, Any]) → DataConfig¶

Build from a plain dict.

Parameters:: data (dict) – Config values.
Returns:: The validated config.
Return type:: DataConfig

resolve_specs(config_dir: pathlib.Path) → list[blinklinmult.data.schema.DatasetSpec]¶

Load the declaration of every configured corpus, in config order.

Parameters:: config_dir (Path) – Directory of per-dataset YAML declarations.
Returns:: One spec per configured corpus.
Return type:: list[DatasetSpec]
Raises:: ConfigError – If a declaration is missing or invalid.

class blinklinmult.train.config.OptimConfig¶

Optimizer and LR-schedule configuration.

Parameters:

name (str) – Optimizer name.
lr (float) – Peak learning rate.
backbone_lr (float | None) – Separate learning rate for the image backbone. A pretrained CNN fine-tunes at a fraction of the rate the randomly-initialised transformer needs; None uses lr for everything.
weight_decay (float) – L2 / decoupled weight decay.
momentum (float) – SGD momentum; ignored by the Adam family.
scheduler (str) – Schedule name, or "none".
warmup_ratio (float) – Fraction of training spent warming up.
scheduler_patience (int | None) – Epochs without improvement before plateau cuts the learning rate. None uses a tenth of max_epochs, which scales with the run but is usually too patient: at 28 minutes an epoch, waiting five epochs to react costs over two hours. Ignored by the other schedulers.
scheduler_factor (float) – Multiplier applied to the learning rate when plateau fires. Ignored by the other schedulers.
gradient_clip_val (float) – Gradient-norm clip; 0 disables.

__post_init__() → None¶

Validate the optimizer configuration.

Raises:: ConfigError – If any value is out of range or unrecognised.

classmethod from_dict(data: dict[str, Any]) → OptimConfig¶

Build from a plain dict.

Parameters:: data (dict) – Config values.
Returns:: The validated config.
Return type:: OptimConfig

class blinklinmult.train.config.ModelConfig¶

Architecture configuration.

Field names track linmult 2.x exactly. The 1.x names the published code used (input_modality_channels, projected_modality_dim, number_of_layers, add_projection_fusion, aggregation) no longer exist upstream and are rejected with a rename hint rather than silently ignored.

input_feature_dim is absent by design: it is derived from the backbone’s embedding width and the dataset’s feature width, so the model cannot disagree with its data.

Parameters:

name (str) – Model name, recorded in the run.
family (str) – linmult, lint, or cnn.
backbone (str) – Image backbone embedding one eye crop.
backbone_pretrained (bool) – Start the backbone from ImageNet weights.
backbone_freeze (bool) – Freeze the backbone’s parameters.
backbone_wide_stem (bool) – Drop one stride-2 reduction so a 64px eye crop keeps a 4x4 feature map rather than collapsing to 2x2. Free for the backbones in BACKBONE_STEM_IS_FREE, which reduce through a parameter-free max-pool; for the others it re-strides a convolution and perturbs the pretrained kernels.
backbone_output_dim (int) – Width the backbone’s embedding is projected to before the sequence model. The raw backbone embedding is 960-d per eye for MobileNetV4, 1024-d for DenseNet121; projecting first keeps the transformer’s input width independent of which backbone is chosen.
encoder_weights (str | None) –
Path to a frame-wise .ckpt whose eye encoder initialises this model’s. The encoder — backbone and projection — is the only component BlinkCNN and the sequence models share, so it is the only part that transfers; the transformer starts fresh either way. It is a starting point, not frozen: optimizer.backbone_lr fine-tunes it from there.

A path rather than a baked-in default so that retraining the frame-wise model and pointing at the new checkpoint is a one-line config edit. None starts the encoder from ImageNet weights.
encoder_freeze (bool) –
Hold the eye encoder fixed – no gradient, and eval mode so its BatchNorm statistics stop moving too. Turns encoder_weights from a starting point into a fixed feature extractor, which is what separates the frozen arm from the fine-tuned one in the video benchmark.

Worth measuring rather than assuming: the video corpora are small next to the 4.8M backbone parameters they would be fine-tuning, so freezing may well win. Meaningless without encoder_weights – freezing an ImageNet encoder that never saw an eye is not an arm anyone wants – and rejected in that combination.
event_head (str | None) –
Stack a blink-interval head on the ESR signal – dense, attention, or None to keep one shared head.

With one head both annotations train the same parameters, and a blink interval covers 3-4x more frames than actual closure, so a corpus annotating only intervals drags the closure prediction wider. Measured on the video benchmark, where 72% of batches supervised events alone: RN30’s ESR F1 fell to 0.4107 against the frame-wise model’s 0.6614.

Stacking keeps the event prediction a function of the closure signal, so the two cannot contradict each other – the reason a single head was chosen originally – while giving each its own parameters. None reproduces the previous behaviour exactly.
event_head_hidden_dim (int) – Width inside the event head.
event_head_detach (bool) – Cut the gradient from the event loss to the ESR head. True is the point of the split; False is worth measuring, since a little event gradient may sharpen blink boundaries that RN15 under-specifies at 15 fps.
d_model (int) – Transformer internal width.
num_heads (int) – Attention heads.
cmt_num_layers (int) – Cross-modal (LinMulT) / encoder (LinT) depth.
branch_sat_num_layers (int) – Per-branch self-attention depth; LinMulT only.
attention_type (str) – linear (LinMulT) or softmax (MulT); this is the only architectural difference between the two baselines. flash, performer, and bigbird are also accepted.
flash_query_key_dim (int | None) – Flash-attention query/key width.
performer_num_random_features (int | None) – Performer feature count.
bigbird_block_size (int) – BigBird block size.
bigbird_num_global_tokens (int) – BigBird global-token count.
bigbird_num_random_tokens (int) – BigBird random-token count.
dropout_input (float) – Dropout before projection.
dropout_output (float) – Dropout after fusion.
dropout_pe (float) – Dropout after positional encoding.
dropout_ffn (float) – Dropout inside FFN blocks.
dropout_attention (float) – Attention-weight dropout.
add_module_tcn (bool) – Per-modality TCN after projection.
tcn_num_layers (int) – TCN depth, when enabled.
tcn_kernel_size (int) – TCN kernel size.
tcn_dropout (float) – TCN dropout.
add_module_ffn_fusion (bool) – FFN + residual after fusion.
head_dropout (float) – Dropout inside each head.
head_hidden_dim (int) – Head hidden width.
head_norm (str) – Head normalisation (bn or ln).

__post_init__() → None¶

Validate the architecture configuration.

Raises:: ConfigError – If a value is out of range or a name is unknown.

classmethod from_dict(data: dict[str, Any]) → ModelConfig¶

Build from a plain dict.

Parameters:: data (dict) – Config values.
Returns:: The validated config.
Return type:: ModelConfig
Raises:: ConfigError – If a retired linmult 1.x key is present.

class blinklinmult.train.config.MLflowConfig¶

MLflow tracking configuration.

Parameters:

tracking_uri (str) – Backend store. A SQLite database, not a directory: MLflow 3.x puts the filesystem store in maintenance mode and raises rather than using it. Relative by default so no absolute local path leaks into a committed config.
experiment_name (str) – MLflow experiment to log under.
run_name (str | None) – Run name; MLflow generates one when None.
log_model (bool) – Log the best checkpoint as an MLflow artifact.

__post_init__() → None¶

Reject the retired filesystem store.

Raises:: ConfigError – If the tracking URI is a bare directory path.

classmethod from_dict(data: dict[str, Any]) → MLflowConfig¶

Build from a plain dict.

Parameters:: data (dict) – Config values.
Returns:: The validated config.
Return type:: MLflowConfig

class blinklinmult.train.config.EarlyStoppingConfig¶

Early-stopping configuration.

Parameters:

enabled (bool) – Whether to stop early at all.
monitor (str) – Metric to watch.
mode (str) – "max" or "min".
patience (int) – Epochs without improvement before stopping.
min_delta (float) – Smallest change counted as an improvement.

__post_init__() → None¶

Validate the early-stopping configuration.

Raises:: ConfigError – If mode is not "min" or "max".

classmethod from_dict(data: dict[str, Any]) → EarlyStoppingConfig¶

Build from a plain dict.

Parameters:: data (dict) – Config values.
Returns:: The validated config.
Return type:: EarlyStoppingConfig

class blinklinmult.train.config.TrainConfig¶

Training-loop configuration.

Parameters:

task (str) – blink_presence, eye_state, or joint.
targets (list[str]) – Target keys to supervise. Defaults to the task’s.
eval_targets (list[str]) – Targets scored but never trained. The model grows no head for these and they reach no loss; the existing prediction is measured against a second annotation. This is how a frame-wise closure model is benchmarked on blink events, which is what the video corpora annotate.
loss (str) – Loss name.
loss_kwargs (dict) – Loss-specific arguments (e.g. focal gamma).
event_carrier_only (bool) – In the event report, keep only the largest corpus and drop the rest as carriers. Correct when a single corpus is being scored through a carrier that supplies the head; wrong on a joint split, where it would discard real corpora because one happens to be larger. Off by default; the per-corpus eval scripts set it.
occlusion_yaw (float | None) –
Absolute head yaw, in degrees, beyond which the far eye is treated as self-occluded and masked out of both loss and metrics. None disables the rule.

Applied in the dataloader rather than baked in at build time, so the threshold stays sweepable without re-preprocessing. Only MPEblink is materially affected – 35.7% of its frames sit beyond 45 degrees – while RN15, RN30 and TalkingFace are near-frontal and lose almost nothing. The right value is a question for inspection, 30 and 45 both being defensible, which is why there is no default.
smoothness_weight (float) – Weight on temporal_smoothness(), which penalises frame-to-frame chatter in the predicted signal. A blink is continuous motion, and a jagged signal is what makes interval extraction brittle – every spurious crossing of the operating point becomes a spurious event boundary. Needs no annotation, so it applies to every corpus. 0.0 disables it, which is the default: it changes what the model optimises and so must be opted into.
smoothness_margin (float) – Frame-to-frame jumps at or below this are free, so a genuinely fast closure is not penalised.
duration_weight (float) – Weight on duration_prior(), which penalises predicted closures far outside a plausible blink duration (~100-400 ms, about 3-12 frames at 30 fps). Also annotation-free. 0.0 disables it.
task_weights (dict[str, float]) – Per-target loss weight for a joint run. Absent targets default to 1.0.
max_epochs (int) – Training epochs.
accelerator (str) – Lightning accelerator.
devices (list[int] | int | str) – Lightning devices spec.
precision (str) – Lightning precision.
seed (int) – Global seed.
deterministic (bool) – Request deterministic kernels.
output_dir (str) – Root for run outputs.
limit_batches (int | float | None) – Cap the batches per epoch in every stage, for a fast but complete run. An integer is a batch count, a float in (0, 1] a fraction, None the whole split.
limit_test_batches (int | float | None) – Cap the test split only, leaving train and validation whole. The inverse of limit_fit_batches, and what a threshold-fitting pass needs: it wants the full validation split to fit on and no test pass at all. Same units as limit_batches.
limit_fit_batches (int | float | None) –
Cap train and validation only, leaving test at full size. This is the knob for cheap diagnostic sweeps: capping test as well would score each arm on a different subset and make the arms incomparable, which defeats the purpose of running them. Same units as limit_batches.

Unlike --fast-dev-run, this keeps the logger, the callbacks and the checkpointing, so every metric a full run produces is written and reaches MLflow — fast_dev_run replaces the logger with Lightning’s DummyLogger and nothing is recorded at all. Use this when the point is to inspect the metrics quickly rather than to prove the code path runs.
optimizer (OptimConfig) – Optimizer settings.
early_stopping (EarlyStoppingConfig) – Early-stopping settings.
mlflow (MLflowConfig) – Tracking settings.

__post_init__() → None¶

Validate the training configuration.

Raises:: ConfigError – If the task, targets, loss, or weights are invalid.

weight_for(target: str) → float¶

Loss weight of one target.

Parameters:: target (str) – Target key.
Returns:: The configured weight, or 1.0.
Return type:: float

classmethod from_dict(data: dict[str, Any]) → TrainConfig¶

Build from a plain dict, including nested sections.

Parameters:: data (dict) – Config values.
Returns:: The validated config.
Return type:: TrainConfig

class blinklinmult.train.config.ExperimentConfig¶

One run: its data, its model, and its training procedure.

Parameters:

data (DataConfig) – Dataloader settings.
model (ModelConfig) – Architecture settings.
train (TrainConfig) – Training settings.

__post_init__() → None¶

Validate consistency across the three sections.

Also derives the one data setting that depends on the model: whether the handcrafted descriptors are consumed, which decides if geometric augmentation is safe. Derived here rather than configured, so the two cannot drift apart.

Raises:: ConfigError – If the model family and the task disagree.

classmethod from_files(data_path: str | pathlib.Path, model_path: str | pathlib.Path, train_path: str | pathlib.Path, overrides: dict[str, Any] | None = None) → ExperimentConfig¶

Load a run’s three config files, applying dotted overrides.

Parameters:

data_path (str | Path) – Data config.
model_path (str | Path) – Model config.
train_path (str | Path) – Train config.
overrides (dict | None) – Dotted section.key overrides, e.g. {"data.batch_size": 8}.

Returns:

The assembled configuration.

Return type:

ExperimentConfig

to_flat_dict() → dict[str, Any]¶

Flatten every field to dotted keys, for mlflow.log_params.

Returns:: e.g. {"data.batch_size": 32, "train.optimizer.lr": 0.001}.
Return type:: dict

blinklinmult.train.config.apply_override(raw: dict[str, Any], dotted: str, value: Any) → None¶

Set a nested config value addressed by a dotted key.

Parameters:

raw (dict) – Nested config mapping, modified in place.
dotted (str) – Key path, e.g. "train.optimizer.lr".
value (Any) – Value to set.

blinklinmult.train.config.parse_override(text: str) → tuple[str, Any]¶

Parse a key=value CLI override, typing the value via YAML.

Parameters:: text (str) – e.g. "data.batch_size=8".
Returns:: The dotted key and its parsed value.
Return type:: tuple[str, Any]
Raises:: ConfigError – If the text has no =.