Fidelity dashboard

A cell moved when its verdict changed between two runs. The direction is not a matter of taste: every lane declares an order over its verdicts in src/verdicts.py, and a cell that drops rank is a regression. Thresholds live in thresholds.json and the exit code of make delta is the gate, so a regression stops a build rather than scrolling past in a diffstat.

Counting probes treats a dropped footnote body and a dropped column width as one event each. They are not. Each probe is classified by what is missing from the page when it does not survive: content means text or a referent is gone with nothing left saying so, structure means every character survived but the document's shape did not, and presentation means styling or a measurement is gone while the words and the shape are intact. The rubric, and the calls that are arguable, are in src/severity.py.

The weighted score below forgives the canonical verdict, because the question it asks is whether the content came home, not whether the writer restyled it on the way. The three per-class columns are the point: two formats reach the same percentage by losing very different things.

Every lane above starts from a pandoc AST, which answers how much of a pandoc document a format can hold. It cannot answer the question a Carve author has, because the pandoc AST has no node for an admonition, a task list, a highlight or an editorial mark, so those never appear in the probe set at all.

This lane starts from Carve instead: 42 source files, one construct each, written out through the pandoc-carve bridge and read straight back, and again through each of 13 export formats. exact is character for character, equivalent is a formatter restyling, respelled means the source is spelled differently and renders identically (a [^1] reference footnote coming back as an inline one), and lossy means a reader would see the difference. The three left-hand verdicts preserve rendered content; source-only information such as comments can still be lost and reported in warnings.

One line per recorded run, appended by src/delta.py --record. Medians rather than means: a handful of formats cannot be measured on a runner without a TeX engine, and a mean would report the machine instead of pandoc.

Two profiles, because the two comparisons are not the same question. Against the pinned pandoc nothing may move without someone deciding it should. Against pandoc's nightly, upstream restyles writers routinely, so presentation-class movement is expected and content-class movement is not.