annie.parsers.csvmeta

Generic table metadata parser for label / protagonist sources (domain).

Unlike the fixed 17-column detection schema (annie.parsers.base), a label table has an arbitrary shape: the user picks which column joins to the video id (the key column) and which columns to surface as tags/filters (the value columns). This module reads headers, suggests a key column by matching values against the known video stems, and builds a key -> {column: value} map.

A table is a CSV file or one worksheet of an Excel workbook: every reader takes an optional sheet and hands workbooks to annie.parsers.workbook, so callers never care which format they read.

When a key repeats, an explicit order column decides which row counts (e.g. the newest timestamp); see order_rows() and duplicate_report().

Classes

DuplicateReport	How a table's key column repeats, for the Dataset tab's warning.
JoinSuggestion	A proposed way to join a table to the videos.

Functions

read_header(→ list[str])	Return the column names of a table (its header row).
read_rows(→ list[dict[str, str]])	Read a table into column-keyed dict rows (CSV: BOM- and CRLF-tolerant).
count_rows(→ int)	Count data rows in a table (excludes the header), cheaply.
suggest_key_column(→ str | None)	Guess which column joins the CSV to videos.
load_value_map(→ dict[str, dict[str, str]])	Build a key -> {value_column: value} mapping from a table.
order_rows(→ list[dict[str, str]])	Return rows sorted by column, stably, with empty values always last.
duplicate_report(→ DuplicateReport)	Report the repeated keys of a table and which of them the order can't separate.
suggest_join(→ JoinSuggestion | None)	Propose the (id column, target) pair whose values overlap the most.
short_name(→ str)	Prefill for a tag name: the header on one line, cut at a word within limit.
distinct_column_values(→ list[str])	Return the distinct non-empty values of one column, in first-seen order.
distinct_values(→ list[str])	Return the sorted distinct non-empty values of column across a map.

Module Contents

annie.parsers.csvmeta.read_header(path: str | pathlib.Path, sheet: str | None = None) → list[str]

Return the column names of a table (its header row).

Parameters:

• path – Path to the CSV file or workbook.

• sheet – The worksheet of a workbook (None: the first); ignored for a CSV.

Returns:

The header column names in order, or an empty list if the file is empty or missing.

annie.parsers.csvmeta.read_rows(path: str | pathlib.Path, sheet: str | None = None) → list[dict[str, str]]

Read a table into column-keyed dict rows (CSV: BOM- and CRLF-tolerant).

Parameters:

• path – Path to the CSV file or workbook.

• sheet – The worksheet of a workbook (None: the first); ignored for a CSV.

Returns:

One dict per data row, keyed by column name. Empty if missing.

annie.parsers.csvmeta.count_rows(path: str | pathlib.Path, sheet: str | None = None) → int

Count data rows in a table (excludes the header), cheaply.

Parameters:

• path – Path to the CSV file or workbook.

• sheet – The worksheet of a workbook (None: the first); ignored for a CSV.

Returns:

The number of data rows, or 0 if missing/empty.

annie.parsers.csvmeta.suggest_key_column(path: str | pathlib.Path, video_stems: set[str] | None = None, *, sample: int = 500, sheet: str | None = None) → str | None

Guess which column joins the CSV to videos.

Scores each column by the fraction of its sampled values found in video_stems; the best non-zero column wins. With no stems (or no match), falls back to the first header that looks like an id (see _KEY_HINTS), else the first column.

Parameters:

• path – Path to the CSV file.

• video_stems – Known video stems to match against, or None.

• sample – Maximum number of rows to inspect.

• sheet – The worksheet of a workbook (None: the first); ignored for a CSV.

Returns:

The suggested key-column name, or None if the file has no header.

annie.parsers.csvmeta.load_value_map(path: str | pathlib.Path, key_column: str, value_columns: tuple[str, ...] | list[str], *, sheet: str | None = None, order_column: str | None = None, descending: bool = True) → dict[str, dict[str, str]]

Build a key -> {value_column: value} mapping from a table.

Parameters:

• path – Path to the CSV file or workbook.

• key_column – The column whose value identifies the video (the join key).

• value_columns – The columns to carry into each entry.

• sheet – The worksheet of a workbook (None: the first); ignored for a CSV.

• order_column – Decides which row counts when a key repeats: rows are ordered by it (see order_rows()) and the first row per key wins.

• descending – Order direction; True puts the largest / newest value first.

Returns:

Mapping from the (stripped) key value to a dict of the selected column values, in winning-row order. Rows with an empty key are skipped. Without an order_column a repeated key keeps its last row (the historical rule; the Dataset tab requires an order column whenever a key repeats).

annie.parsers.csvmeta.order_rows(rows: list[dict[str, str]], column: str, *, descending: bool) → list[dict[str, str]]

Return rows sorted by column, stably, with empty values always last.

The column compares numerically when every non-empty value is a number (9 < 10), otherwise as text — which also orders ISO dates and timestamps chronologically, the form annie.parsers.workbook gives Excel dates. Rows with equal values keep their file order, so the result is always deterministic.

Parameters:

• rows – Dict rows as returned by read_rows().

• column – The column to order by.

• descending – True for largest / newest first.

Returns:

A new, ordered list (the input is not modified).

class annie.parsers.csvmeta.DuplicateReport

How a table’s key column repeats, for the Dataset tab’s warning.

duplicated

Keys that appear on more than one row, in first-seen order.

ties

Duplicated keys whose two leading rows (after ordering) share the same order value, so only file order separates them. Every duplicated key is a tie when no order column is given.

annie.parsers.csvmeta.duplicate_report(rows: list[dict[str, str]], key_column: str, order_column: str | None = None, *, descending: bool = True) → DuplicateReport

Report the repeated keys of a table and which of them the order can’t separate.

Parameters:

• rows – Dict rows as returned by read_rows().

• key_column – The join-key column.

• order_column – The column chosen to resolve repeats, or None.

• descending – The chosen order direction.

Returns:

A DuplicateReport.

class annie.parsers.csvmeta.JoinSuggestion

A proposed way to join a table to the videos.

column

The table column holding the ids.

target

What the ids match: a target name passed to suggest_join().

matched

How many distinct ids of that column the target contains.

annie.parsers.csvmeta.suggest_join(rows: list[dict[str, str]], header: list[str], targets: dict[str, set[str]], *, sample: int = 500) → JoinSuggestion | None

Propose the (id column, target) pair whose values overlap the most.

A target is anything ids can match: all video ids, or the values of one nested- layout path field such as {subject}. Ties go to the earlier column, then the earlier target, so the first target (the video id) wins when both fit equally.

Parameters:

• rows – Dict rows as returned by read_rows().

• header – The table’s column names, in order.

• targets – Target name to the set of values it offers.

• sample – Maximum number of rows to inspect.

Returns:

The best JoinSuggestion, or None when no column matches anything.

annie.parsers.csvmeta.short_name(header: str, limit: int = _SHORT_NAME_LIMIT) → str

Prefill for a tag name: the header on one line, cut at a word within limit.

Parameters:

• header – The column header (may be a long, multi-line question).

• limit – Maximum length of the result.

Returns:

The header with whitespace collapsed; when longer than limit, cut back to the last whole word and marked with ….

annie.parsers.csvmeta.distinct_column_values(rows: list[dict[str, str]], column: str) → list[str]

Return the distinct non-empty values of one column, in first-seen order.

Used by the Browse id-list filter: a CSV supplies the set of video ids to keep, and the order is preserved so the preview shows them as the file lists them.

Parameters:

• rows – Dict rows as returned by read_rows().

• column – The column to collect.

Returns:

The distinct stripped values of column, empties excluded.

annie.parsers.csvmeta.distinct_values(value_map: dict[str, dict[str, str]], column: str) → list[str]

Return the sorted distinct non-empty values of column across a map.

Parameters:

• value_map – A map from load_value_map().

• column – The value column to collect.

Returns:

Sorted distinct values, empties excluded.