omniloader.introspection

Introspection helpers: describe a dataset mix and validate it against specs.

describe() summarises a collection of datasets — the union schema, which dataset provides which key (the coverage matrix), the fraction of valid (non- placeholder) positions per key, and class histograms for integer targets. validate() dry-runs each dataset, checking that the raw samples it emits match the shape and rank their TensorSpec declares — catching schema/data mismatches before training starts.

Classes

Report

A human-readable summary of a dataset mix.

Functions

describe(→ Report)	Summarise a dataset mix into a Report.
validate(→ list[str])	Check that each dataset's raw samples match its declared specs.

Module Contents

class omniloader.introspection.Report

A human-readable summary of a dataset mix.

names

Per-dataset names.

sizes

Per-dataset sample counts.

union_keys

Every feature/target key in the union schema.

coverage

{dataset_name: {key: provided?}}.

valid_fraction

{key: mean fraction of valid positions} (sampled).

class_distributions

{target_key: {class_id: count}} (sampled).

__str__() → str

Render the report as an aligned text table.

omniloader.introspection.describe(datasets: collections.abc.Sequence[omniloader.loader.SizedDataset], schemas: collections.abc.Sequence[omniloader.schema.spec.DatasetSchema], max_samples: int = 64) → Report

Summarise a dataset mix into a Report.

Parameters:

• datasets – The source datasets.

• schemas – One schema per dataset.

• max_samples – Cap on samples read per dataset for the valid-fraction and class-distribution statistics.

Returns:

The populated Report.

omniloader.introspection.validate(datasets: collections.abc.Sequence[omniloader.loader.SizedDataset], schemas: collections.abc.Sequence[omniloader.schema.spec.DatasetSchema], num_samples: int = 4, strict: bool = False) → list[str]

Check that each dataset’s raw samples match its declared specs.

For every declared key it verifies the value’s rank and trailing feature size against the TensorSpec (the sequence length is free — unification pads it). Dtype is not checked because unification casts it.

Parameters:

• datasets – The source datasets.

• schemas – One schema per dataset.

• num_samples – Number of samples to probe per dataset.

• strict – When True, raise ValueError if any issue is found.

Returns:

A list of human-readable issue strings (empty when everything matches).

Raises:

ValueError – If strict and issues were found.