blinklinmult.train.mlflow_utils

MLflow wiring for blink training runs.

Logging is explicit rather than via mlflow.pytorch.autolog(): autolog picks its own cadence and parameter names, which would fight the flattened dataclass params and make two runs hard to compare. Everything logged here is chosen.

What makes a run reproducible is not just its hyperparameters but the exact data it saw, and this project’s data is several files — one per corpus. environment_params() therefore records each built dataset’s hash, size, and embedded builder SHA individually, so two runs over “the same six corpora” can be told apart when one of them was rebuilt.

Attributes

logger	Module-level logger.
MAX_PARAM_LENGTH	MLflow rejects parameter values longer than this.
RUN_ID_FILE	Name of the file in a run's output directory holding its MLflow run id.
SKIP_ARTIFACT_DIRS	Directories excluded from the bulk artifact upload.

Functions

git_sha(→ str)	Return the current commit SHA.
file_md5(→ str)	Hash a file's contents.
dataset_params(→ dict[str, Any])	Record the identity of every dataset file the run reads.
environment_params(→ dict[str, Any])	Collect the facts that make a run reproducible.
truncate_params(→ dict[str, Any])	Clip over-long values so MLflow accepts every parameter.
stored_run_id(→ str | None)	The MLflow run id a previous launch recorded, if any.
remember_run_id(→ None)	Record a run's MLflow id so a later resume can continue it.
build_logger(config[, run_name, run_id])	Construct the MLflow logger for a run.
log_artifacts(→ None)	Upload a run's output directory as artifacts.

Module Contents

blinklinmult.train.mlflow_utils.logger

Module-level logger.

blinklinmult.train.mlflow_utils.MAX_PARAM_LENGTH = 500

MLflow rejects parameter values longer than this.

blinklinmult.train.mlflow_utils.git_sha(root: pathlib.Path = PROJECT_ROOT) → str

Return the current commit SHA.

Parameters:

root (Path) – Repository root.

Returns:

The SHA, or "unknown" outside a git checkout.

Return type:

str

blinklinmult.train.mlflow_utils.file_md5(path: pathlib.Path, chunk_size: int = 1 << 20) → str

Hash a file’s contents.

Parameters:

• path (Path) – File to hash.

• chunk_size (int) – Read size in bytes.

Returns:

Hex digest, or "missing" if the file does not exist.

Return type:

str

blinklinmult.train.mlflow_utils.dataset_params(specs: list[blinklinmult.data.schema.DatasetSpec], root: pathlib.Path = PROJECT_ROOT) → dict[str, Any]

Record the identity of every dataset file the run reads.

Two runs with identical hyperparameters over differently-built corpora are not comparable, and nothing else in the config would reveal that.

Parameters:

• specs (list[DatasetSpec]) – Corpora the run uses.

• root (Path) – Repository root.

Returns:

Per-corpus hash, size, and the builder SHA embedded in the file.

Return type:

dict

blinklinmult.train.mlflow_utils.environment_params(config: blinklinmult.train.config.ExperimentConfig, specs: list[blinklinmult.data.schema.DatasetSpec], root: pathlib.Path = PROJECT_ROOT) → dict[str, Any]

Collect the facts that make a run reproducible.

Parameters:

• config (ExperimentConfig) – The run’s configuration.

• specs (list[DatasetSpec]) – Corpora the run uses.

• root (Path) – Repository root.

Returns:

Library versions, the commit, and every dataset’s identity.

Return type:

dict

blinklinmult.train.mlflow_utils.truncate_params(params: dict[str, Any]) → dict[str, Any]

Clip over-long values so MLflow accepts every parameter.

Parameters:

params (dict) – Parameters to log.

Returns:

The same parameters, with long values truncated.

Return type:

dict

blinklinmult.train.mlflow_utils.RUN_ID_FILE = 'mlflow_run_id.txt'

Name of the file in a run’s output directory holding its MLflow run id.

Written on first launch and read back on resume, so a continued run appends to the same MLflow run instead of minting a second, disjoint one. Without it a resumed run’s metric curve starts over at the resumed epoch and the two halves cannot be read as one training history.

blinklinmult.train.mlflow_utils.stored_run_id(output_dir: pathlib.Path) → str | None

The MLflow run id a previous launch recorded, if any.

Parameters:

output_dir (Path) – The run’s output directory.

Returns:

The id, or None when this is a first launch.

Return type:

str | None

blinklinmult.train.mlflow_utils.remember_run_id(output_dir: pathlib.Path, mlflow_logger) → None

Record a run’s MLflow id so a later resume can continue it.

Reading mlflow_logger.run_id also materialises the run, which is why this is called once at launch rather than lazily.

Parameters:

• output_dir (Path) – The run’s output directory.

• mlflow_logger – The run’s MLFlowLogger.

blinklinmult.train.mlflow_utils.build_logger(config: blinklinmult.train.config.ExperimentConfig, run_name: str | None = None, run_id: str | None = None)

Construct the MLflow logger for a run.

Parameters:

• config (ExperimentConfig) – The run’s configuration.

• run_name (str | None) – Overrides the configured run name.

• run_id (str | None) – An existing run to append to, for a resumed run. None starts a new one.

Returns:

The configured logger.

Return type:

MLFlowLogger

blinklinmult.train.mlflow_utils.SKIP_ARTIFACT_DIRS: frozenset[str]

Directories excluded from the bulk artifact upload.

Checkpoints are hundreds of MB and the chosen one is logged deliberately at the end of a run; copying every one of them into the tracking store would bloat it for no benefit.

blinklinmult.train.mlflow_utils.log_artifacts(mlflow_logger, output_dir: pathlib.Path) → None

Upload a run’s output directory as artifacts.

Parameters:

• mlflow_logger – The run’s MLFlowLogger.

• output_dir (Path) – Directory of run outputs.