Capture coverage — honest, not magical#

BitExact's central honesty invariant is that integrity is not completeness: a passing bundle proves the captured trajectory is intact, honestly redacted, and carries what a replay would need — never that it is the complete set of calls the agent made, and never, on its own, that the run is sealed: an unsealed bundle passes unless the auditor asks for --require-sealed. A call made entirely outside a wrapped surface (a proxied/patched model client, a record_tool-wrapped tool) leaves no trace and no gap to detect.

Capture coverage makes the health and consistency of what was recorded operationally visible at scale, without ever crossing into a completeness claim it cannot support.

What it reports#

bitexact coverage report [<workflow>] (and the Capture coverage panel on a workflow's page) aggregate a corpus of runs into:

• Recorded surfaces — the tool names and model ids that appear across the corpus, each with the number of runs it shows up in.
• Inconsistent surfaces — surfaces recorded in some runs of the corpus but not others. A tool a workflow records in most runs but not a few is a lead worth a human look; it is not asserted to be a gap (it may simply be conditional logic).
• Capture-health signals, aggregated from what the per-run inspector already computes: recording failures, orphan stream journals, declared gap markers, and unsealed runs.
• Executions refused at open (health.refused, listed under lacked_evidence) — runs that never opened, so they left no evidence at all. See below.
• Runs stored under a tampered name (health.tampered, listed under tampered) — a run file whose name is not a run id, which no recorder writes. bitexact health alerts on the same files by the same rule, and unfiltered bitexact coverage report reads the same corpus health does, so the two never disagree about them.
• A verdict — degraded when there are recording failures, orphan journals (real signs an engaged recorder dropped data), executions refused at open, or runs under a tampered name, otherwise clean. Declared gap markers are honest and do not degrade the verdict; inconsistent surfaces are leads, not failures.

bitexact coverage report claims-triage --version v4
# exit 0 = capture healthy, 1 = degraded (a CI capture-health gate)

Runs refused at open#

An SDK run that cannot open — the license lapsed, the recorder key is missing or not one the license names, the signer or configuration is faulty, the store cannot be opened, the run id is rejected — runs the agent's code unrecorded, and BitExact records the refusal itself. So do the model proxy and the MCP proxy, for a call naming a run that does not exist. Strict mode (BITEXACT_FAILURE_MODE=strict) and the test modes (replay, regress, fork) raise instead, and record nothing.

Each refusal is a refused event on a hash-chained refusal ledger kept per requested run id (ledger kind refusals): the cause, where it was refused (sdk, proxy or mcp-proxy), and the workflow, version and environment the run asserted. When a recorder key loads, the event is signed with it; a signer that fails leaves the event unsigned, never unwritten. The existing run with the same id, if there is one, is never touched. A run id the store cannot use is not stored; its refusal is counted without one.

bitexact coverage report counts these executions as lacking evidence, even when no run was recorded at all, and names each with its causes. A refusal is marked unsigned unless a recorder key the license names signed every event, and record tampered when its ledger fails verification; either way it still counts. A refusal record can only add executions without evidence; it never excuses missing evidence for a run that was recorded.

bitexact coverage report
# capture coverage - all runs: 1200 runs, verdict degraded
#   ...
# WARNING: 3 executions refused at open, with no evidence:
#   job-0031 (license_lapsed x2)
#   job-0032 (recorder_unavailable x1, unsigned)

The viewer's Capture coverage card and its Overview show the same count. A refusal is shown to a scoped viewer only when its asserted workflow and environment are in scope and match the report's filter; one that asserted neither appears only in the unrestricted, unfiltered report.

What a refusal record does and does not prove: an edited event fails its hash (and its signature), a removed event breaks the chain, and a removed tail fails the count witness on the embedded store; on PostgreSQL the ledger table refuses updates and deletes. Every refusal event is also a leaf in the transparency log (Transparency log), so a refusal deleted from the store - its whole ledger file included - is named by bitexact log audit and raised by bitexact health as log-refusal-gone. When the store itself is what failed, the refusal cannot be written: it is counted in refusals_unwritten_total at /metricsz and logged as refusal_unwritten at error level, and those are its only record.

A malformed identity argument to bitexact.run() (a workflow that is not a string, a scope that is not a list of strings) is left out of the run's identity and declared on the run as an invalid_argument gap; strict mode and the test modes raise it.

In evidence packs#

A selection or workflow evidence pack carries the refusal ledgers its filters select under data/refusals/, whole, and its workpaper lists them under Executions refused at open. Refusals ride along only when the selection is by fields a refusal records - workflow, version, environment and the time window - and names no runs; otherwise the workpaper says they are not included. A pack from the viewer carries only the refusals in the caller's scope.

A case pack carries the refusals recorded under the case's pinned run ids, and no others: a case record pins run ids, never a workflow or a period, so no wider span is part of it. A single-run pack carries none - a refused execution has no run of its own - and its workpaper says how many refusals the store records under that run id. Both workpapers say plainly that other refusals are not part of the pack and point to a workflow or selection pack.

The standalone verifier checks each refusal ledger as a hash chain, requires every signature an event carries to verify, and fails a ledger whose events are not refusals or name another run. It counts the refusals as declared capture gaps - exit 3 unless --allow-gaps - and says how many a recorder key signed (the pinned one, with --expect-recorder-key). An unsigned refusal is counted, never failed: it can only add an execution that left no evidence.

Expected against captured#

Completeness needs a second witness: something outside BitExact that knows what ran. Your orchestrator or job system is that witness. Export the run ids of the executions that should have evidence, one per line, and:

bitexact coverage report --expected executions.txt
# capture coverage - 1200 expected, 1197 with evidence (99.8%)
#   2 with declared gaps (evidence, capture incomplete): job-0412, job-0977
# WARNING: 3 executions lacked evidence:
#   job-0031
#   ...
# exit 1 when any execution lacked evidence

An execution with no evidence is one BitExact never recorded at all: the recorder or collector was unreachable, the license had lapsed, or a run was refused at open. Those cases never stop the agent, and this is where they become visible; a refusal BitExact recorded is listed with its cause (not recorded: license_lapsed x1). A run recorded under a suffix because its id was taken (<id>-<8 hex>, with the id it asked for in its run_meta) counts as its execution's evidence; a lookalike id without that does not.

Sources and silence#

The corpus report says what was recorded. The coverage scorecard answers a narrower, operational question: is evidence still arriving from every place you say it should? It is read with bitexact coverage sources, at GET /api/coverage/sources in the inspector, and the recorder's /metrics carries it as bitexact_source_last_evidence_seconds, bitexact_source_state (one series per state, 1 for the state the expectation is in) and bitexact_collector_last_heartbeat_seconds, so your own "no data" monitors can sit on top. The scorecard is operational state, never evidence, and is never signed as if it were. Like every coverage report it carries the completeness boundary, in its note field:

Coverage reports when declared sources last produced evidence. It cannot see a call made outside a wrapped surface, so it is never a measure of completeness.

Reading the scorecard#

bitexact coverage sources [--workflow WORKFLOW] [--environment ENVIRONMENT] [--json]
bitexact coverage sources --rebuild [--json]

bitexact coverage sources reads the scorecard from the run index, the capture-source heartbeats and the expectations ledger alone. It prints each declared expectation with its state, its match, its last evidence and last heartbeat, its allowed silence and severity, the state of each capture path and each authenticated collector host it names, the undeclared sources, each inferred expectation with the coverage expect add command that declares it, the collectors' last heartbeats, the collector and removal alerts, a warning when a read bound left sources out, then what it can and cannot vouch for (a host is authenticated by its collector's client certificate; a workflow, environment, agent and SDK label are asserted by the writer that sent them), then the note above. --json prints the whole scorecard instead. --workflow and --environment narrow it to one workflow's or one environment's expectations and sources, as a viewer's access scope does.

The run index is a cache, so a row edited to move a source's last evidence forward would keep a silent source live. --rebuild reads the index's per-source rows, rebuilds the whole index from the manifests, reads it again and names every source whose indexed row differed, with the fields that did. It exits 1 when a row differed, the index could not be read before the rebuild, a manifest could not be read, or the index's read bound left sources uncompared, so a match is claimed only when every source was compared; 0 otherwise.

Sources#

A source is identified by up to five parts: workflow, environment, agent_id, path (the capture path) and host (the collector's authenticated peer). An expectation names any subset in its match; a part it leaves out matches anything. The capture path is one of proxy, sdk-python, sdk-ts, sdk-go, sdk-java, otel, langchain, openai-agents, claude-agent-sdk, adk, auditant, llamaindex, vapi, mcp-proxy and governance for a writer that writes into the store itself, or relay: and one of them for an event the recorder took from a collector. governance is every SDK's governance-decision API (a policy engine's decision, a person's approval), whichever engine the entry names as reported_by, so an expectation with "paths": ["governance"] (or ["relay:governance"] for a host collector) watches decision recording on its own: plain SDK traffic never keeps it live.

The two clocks#

• Last evidence is the recorder's own ts on the newest chain entry from the source. The recorder writes that time, never the source, so a source cannot make itself look recent by sending a future time.
• Last heartbeat is when the recorder last heard from the host's collector, or when a proxy or SDK process that writes into the store itself last wrote its liveness row. Both send one every BITEXACT_HEARTBEAT_S seconds (default 60). A process's liveness row names, as serves, the workflow, environment and agent of each run it wrote for, and speaks only for those sources: another process on the same path is no channel for this one, which then reads silent.
• In a run the recorder took from collectors, the recorder's own entries (the run's run_meta, its seal, a source's end) are evidence of no source and move no source's last evidence. A run the proxy or the Python SDK writes in process counts its run_meta as that path's evidence, since that path wrote it. A loss the recorder or a collector declares counts against the source it names.

Only evidence makes a source live. A heartbeat proves the link, never the capture: it decides only between quiet and dark.

States#

Each expectation is in one state, computed when the scorecard is read:

A source that is observed but matched by no expectation is listed as undeclared: a lead, which raises an alert only when the customer has asked for one. bitexact coverage expect undeclared-alerts on|off --reason R [--severity S] asks, or stops asking: a signed setting on the capture-expectations ledger, attributed to its principal (--by, default the OS user) with its reason, since turning the alerts off is how a new source would go unnoticed. It is off until asked; --severity (default warning) goes only with on, and coverage expect list shows the setting in force.

Alerts#

Only a declared expectation raises a state alert. Both collector alerts are raised however recent the collector's heartbeat: a link that heartbeats while it drops events is still raised.

Asserted and authenticated#

The capture label a source sends is asserted: any process holding the host secret could send any label, and the workflow, environment and agent a run names are its own claim. The collector's peer is authenticated by the recorder's mutual TLS, from the certificate it presented. The scorecard shows both and says which is which; it never claims more than the authenticated part proves.

The expectations ledger#

The sources you expect are declared on a signed, hash-chained ledger (bitexact coverage expect add|remove|reject|withdraw|list|import|verify), so every change is attributed to the principal who made it, timestamped and tamper-evident. Each change is signed by a recorder key this deployment's license names, and bitexact coverage expect verify exits 2 when the chain or a signature does not verify. A window (--max-silence, --heartbeat) is an ISO-8601 duration of days, hours, minutes and seconds, such as PT2H or P1DT30M.

bitexact coverage expect add uw-prod --workflow loan-underwriting --environment prod --max-silence PT2H --severity critical --reason "go-live"
bitexact coverage expect list
bitexact coverage expect verify

Removing an expectation is how a silent source would be hidden, so a removal needs a reason, stays on the ledger, and raises expectation-removed for 7 days. Removing a critical expectation needs two principals, as a redaction does: the first one's removal is recorded as a request, and only a different principal's removal takes the expectation out. Lowering a critical expectation's severity is refused for the same reason. A pending request is closed without a removal by reject, from a principal who did not request it, or by withdraw, from the one who did; each takes a reason and stays on the ledger, and list names the outcome. import takes JSON lines, each {"op": "expect", ...} or {"op": "remove", "id", "reason"}, checks every line's form and then runs the whole file against this rule, each line against the state the lines before it leave, before it writes any, so an orchestrator or an infrastructure-as-code pipeline can own the inventory and a file the rule refuses changes nothing. The one exception is a change another principal makes while the file is being written: a line it makes refused stops the import there, the lines before it stay applied, and the error names the line and how many changes were applied.

On the command line --by (and an import line's by) is typed, not authenticated. With BITEXACT_REQUIRE_DUAL_CONTROL set, the command line refuses to remove or request the removal of a critical expectation, in remove and in import: that is done in the inspector (Settings, Coverage), where each principal is the signed-in one. Rejecting or withdrawing a request keeps the expectation, so the command line still does both.

Inferred expectations#

For every (workflow, environment, path) that no declared expectation covers and that has at least 20 runs whose start times span at least 7 days, the scorecard proposes an expectation with max_silence = max(24 hours, 3 x the 95th percentile gap between consecutive run starts). A proposal is labeled "inferred from history" and never alerts: declaring it is one step. Runs in the sample environment are left out, so sample and demo runs cannot make a silent production workflow look covered.

What it deliberately does not do#

It does not enumerate "the callsites you forgot to wrap." Doing so would require knowing about calls BitExact never saw — exactly the thing the honesty invariant says it cannot know. Every report carries that boundary as a field so no downstream consumer can restate it as a completeness guarantee.

Two honest, existing tools sit either side of it:

• The per-run Capture scope tab shows recording failures, orphan journals, and declared gaps for a single run.
• bitexact lint flags, at the source level, nondeterminism that replay cannot capture — the code-level complement to this store-level report.

Together they make incompleteness visible and investigable — never invisible, and never dressed up as complete.