Read a ledger row

A ledger row answers one question for one surface and one period: how much better or worse did treatment do than control on the primary metric? Read it in this order: is it readable, is the SRM p-value clear, then the difference and its interval.

An example row#

{
  "period_start": "2026-07-20",
  "period_end": "2026-10-11",
  "surface": "home_articles",
  "metric": "tap_rate",
  "control_rate": 0.042,
  "treatment_rate": 0.052,
  "difference": 0.010,
  "ci_low": 0.004,
  "ci_high": 0.016,
  "n_control": 14800,
  "n_treatment": 15020,
  "n_users_control": 1480,
  "n_users_treatment": 1502,
  "srm_p": 0.61,
  "readable": true,
  "revenue_cents_control": null,
  "revenue_cents_treatment": null,
  "export_uri": null
}

The numbers are an example, not a result.

Field by field#

How to read it#

The interval decides, not the point. If the whole interval is above zero, treatment did better. If it crosses zero, the data cannot tell the arms apart yet. Here +0.4 to +1.6 is above zero: Patterna's slate got more tapped slates than the editorial list.

The metric is per slate, the interval is per unit. One user sees many slates, and their taps are not independent. So the interval is clustered on the randomised unit with the delta method. That is why the row carries both slates (n_*) and units (n_users_*). See Proof and the ledger.

Every served slate counts. The ledger is intention to treat: slates the user never scrolled to, and slates the app gave up on, count as served with no tap, and so do slates answered with shown: false. Only slates requested with context.measured: false are left out, in both arms. See Timeouts and what counts in the ledger.

readable: false means wait until each arm reaches the experiment's readable_sample; the panel shows the interval dashed amber with the projected "readable on" date until then.

An SRM flag means stop. If the arm sizes differ from the split you asked for more than chance allows, do not read the row. Find the cause first. See Run a holdout.

Revenue is not computed yet. When it is, it will be a last-touch diagnostic, never proof. Retention and conversion are reported, not proven, until the user base is large enough.

Synthetic rows never count. Example data from the twin is flagged synthetic and excluded from the ledger. The "Real / Example" switch never mixes them.

Check it yourself#

A row is computed from the logged impressions and the interactions reported on them. The export below gives you both, so you can check the slate and tap counts. It does not yet carry the clustering unit, measured, not_shown, tester, fallback, blocks or the experiment window, so it cannot reproduce a row's interval on its own; that is an open item on our side.

curl -sS -X POST "https://patterna-api.orven.am/v1/proof/export" \
  -H "Authorization: Bearer $PATTERNA_ADMIN_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "from": "2026-07-20",
    "to": "2026-10-11",
    "surface": "home_articles"
  }'

The answer is 202 with a job_id. Poll GET https://patterna-api.orven.am/v1/proof/exports/{job_id} until it is ready, then download it. In the panel, use Proof → Export: pick the period, wait for the file, download it. The note there spells out the arithmetic.

Fetch the rows themselves with:

curl -sS "https://patterna-api.orven.am/v1/proof/ledger?surface=home_articles&from=2026-07-20&to=2026-10-11" \
  -H "Authorization: Bearer $PATTERNA_ADMIN_KEY"

Without from and to the ledger covers the last 28 days.

Weekly rows#

GET https://patterna-api.orven.am/v1/proof/ledger?surface=home_articles&bucket=week returns one row per ISO week and arm: week_start, arm, impressions, taps, rate and a Wilson interval (lo, hi). The weeks are disjoint, not cumulative. They show the trend; the summary row above is the result.

Rows per block#

When one slate is shown in two blocks of a page and the interactions carry block (for example top and end), ask the ledger with by=block (every block) or block=<name>: it returns one row per surface and block, over the slates reported seen in that block and the taps in it. Those rows are descriptive; the whole-slate row decides.