Verifying a BitExact bundle — a procedure for auditors#

This is a self-contained procedure for an auditor, regulator, or counterparty who has received a signed BitExact evidence bundle and wants to establish, independently, what it does and does not prove. Steps 1 to 4 require no BitExact license, no access to the producer's systems, and no trust in BitExact the company; they run offline, on an air-gapped machine if you choose. Step 5 is different and says so: re-executing the run needs the BitExact SDK under a license with the replay entitlement, so it is an option for a party that has one, not part of the independent procedure.

Working inside the producer's inspector instead - finding a decision, reading a run, opening a case? See Inspector guide.

What you need#

The bundle file (*.bundle.json, a streaming *.bundle.jsonl, or a *.dsse.json DSSE envelope).
The open-source verifier, bitexact_verifier.py. Its only dependencies are cryptography and asn1crypto; both are widely audited, and you can read the whole verifier — one file — yourself.

Where to get it. The public repository github.com/bitexact/bitexact-verifier carries each release under a verifier-v<version> tag, with CHECKSUMS.txt beside it. Every evidence pack and provenance bag also vendors the exact copy that produced it, under data/verification/. Or install the release yourself: pip install bitexact-verifier==<version>.
Optionally, the public keys or certificates you intend to trust: the producer's bundle-signing key, its recorder key, and/or the timestamp authority's certificate. Obtain these through your own channel, not from the bundle.

Is the verifier itself genuine?#

The verifier is the root of trust: a tampered verifier would report OK on tampered evidence. You do not have to trust BitExact to establish that the copy you hold is authentic — a vendor signature would only move the question to "do I trust BitExact's key?". Rely instead on two properties that need no vendor trust:

Read it. It is one stdlib-only Python file (plus cryptography and asn1crypto for signatures and RFC 3161). Because it is interpreted, the file you read is the file that runs — there is no build step hiding behind it. You, or an expert you choose, can audit the whole thing.
Confirm its digest, out-of-band. verifier/CHECKSUMS.txt in a product release records the sha256 of bitexact_verifier.py under its version, and bitexact-verifier --version tells you which release you hold. The point of the check is to compare that digest against a channel independent of wherever you got the file, because the case that matters most is a verifier handed to you inside a producer's evidence pack: the producer is the party you are auditing, and its self-declared hash proves nothing.

The independent channel is the public repository: fetch CHECKSUMS.txt at the verifier-v<version> tag of github.com/bitexact/bitexact-verifier and compare it with the sha256 PyPI records for bitexact_verifier.py inside the bitexact-verifier==<version> wheel. Both channels are BitExact's; an auditor who needs one outside BitExact's control should also obtain the digest from the producer through a separate contractual channel and record that they did so.

The honest limit: confirming the file does not extend to the interpreter and libraries it runs on — CPython, cryptography/OpenSSL, and asn1crypto are part of the trusted base and lie outside this one file. No signing or provenance scheme closes that gap; a signature would only re-root the same trust in a vendor key.

The procedure#

Verify integrity, trusting nothing yet.

bitexact-verifier run.bundle.json

Exit code 0 means every hash and signature in the bundle re-derives from first principles: the entry chain is intact from genesis to the recorded head, each field commitment matches, any redaction actually removed the value it claims to, a sealed run's terminal marker is final and attests the right step count, and any signature and checkpoints are internally consistent. Exit 2 prints FAIL: with the exact step or check that failed, on stdout, and is disqualifying — stop here. Exit 1 prints error: and is no verdict at all: the verifier could not use an input you gave it (a path that does not exist, a trust file or pin that makes no sense, a mistyped flag); fix the command and run it again. Exit 3 means all of that held but the run declares capture gaps: the record is intact and incomplete, the OK: line counts the gaps, and each gap names its cause - a gap marker in the run, or a field an entry's capture declaration says was withheld or can never be opened. Accept them deliberately with --allow-gaps. Observed entries that a remote host captured and relayed are counted on the OK: line too: that host saw them, not the recorder. So is what the capture policy kept of the entries - how many fields were scrubbed, kept hash-only or left unscanned (docs/capture-controls.md). A scrubbed field proves only that the scrubbed value is what was committed; a hash-only field is a keyed hash of its value, which is not de-identification.

The codes are one contract, shared word for word by the verifier and the bitexact command line:

Exit	Meaning
`0`	OK - the evidence verifies, or the command did what it was asked
`1`	operational error - a usage error, a missing run or file, input that cannot be read, or a license refusal: no verdict was reached
`2`	FAIL - the evidence does not verify, or a check it requires cannot run on this install (a library is missing)
`3`	verified, with declared capture gaps - an intact record of an incomplete capture

The success line also says what the chain alone can say about the run's end: sealed means a terminal marker attests it and, if the run was recorder-signed, a checkpoint covers that head; unsealed - end not attested means the record is intact up to its last entry but nothing attests that it is the last. A live run and a truncated one look the same to the chain. Add --require-sealed when your procedure must not accept an unfinished run.

Pin the identities you trust. Integrity alone does not tell you who produced the bundle. Require the specific keys you obtained independently:
```
bitexact-verifier run.bundle.json \
  --expect-key <bundle-signing key, 64 hex> \
  --expect-recorder-key <recorder key, 64 hex>
```
or supply several accepted keys (supporting rotation) in a trust file with --trust-file trust.json. --expect-recorder-key pins both the signed checkpoints and any WORM anchor to that recorder key. A verifier told to require a bundle key fails an unsigned bundle. Told to require a recorder key, it fails any bundle whose head is not covered by a checkpoint or WORM anchor signed with that key, and any bundle with a withheld field that no recorder-attested on-chain claim accounts for — whatever else signed the bundle. The exporter's own signature never stands in for the recorder's: the exporter is the party with store access, which is exactly whom the recorder tier lets you distrust. The success line then reads recorder-attested.

Pin the post-quantum half too. A producer's keys are hybrid: an Ed25519 key and an ML-DSA key beside it, and every checkpoint, anchor and signature carries both. Ed25519 alone can be forged by anyone with a quantum computer, who recovers the key from the public half every checkpoint carries. Pin the ML-DSA half and that forgery fails:
```
bitexact-verifier run.bundle.json \
  --expect-recorder-key <recorder key, 64 hex> \
  --expect-recorder-pq-key <its ML-DSA public key, hex or a file> \
  --expect-key <bundle key> --expect-pq-key <its ML-DSA public key>
```
or give each trust-file entry its pq_public_key (bitexact keygen prints the entry). A pinned PQ half is required on everything that key signed, and a present one that does not verify always fails. --require-pq demands a post-quantum signature everywhere without pinning one, and fails, in bitexact verify as in the standalone verifier, where the installed cryptography has no ML-DSA to check it with. The success line names each RFC 3161 timestamp's own signature (timestamp: classical TSA signature (RSA/ECDSA) - ...), and says signatures: Ed25519 + ML-DSA-65 (post-quantum), post-quantum key pinned, or Ed25519 only - not resistant to a quantum forger for evidence signed without the half.
Check the external time anchor, if present. A bundle may carry anchors that pin its head outside the producer's own storage — the defense against a producer who could otherwise rewind their store before signing. Require the timestamp authority you trust:
```
bitexact-verifier run.bundle.json --expect-tsa tsa-root.pem
```
The verifier checks that the RFC 3161 token's imprint is the bundle head, that the authority's signature is valid, and that the anchor still matches the head the bundle carries — so a bundle truncated below an anchored point fails. Without --expect-tsa the token's time is unverified (the run still binds, but the success line says so). A WORM anchor is validated against the recorder key and bound to the head — external, immutable evidence the head existed and was not rolled back, though (unlike an RFC 3161 token) its own timestamp is recorder-asserted.
Read the trajectory. The bundle's entries are the ordered record of the run: each model call's request and response, each recorded tool call, each captured source of nondeterminism, and any explicitly synthetic injected step. Redacted fields are marked and counted; a value can be hidden while its commitment still proves it existed. What you are reading has been proven, step by step, to be the bytes that were recorded.
Reconstruct or replay, if you need to. With the BitExact SDK and the same bundle, the run can be re-executed deterministically against the recorded responses; any divergence halts at the exact step. A counterfactual can be forked; it is recorded as a synthetic step that cannot be presented as part of the production record.

Checking a file in the viewer#

The viewer's Verify → Verify a file you were handed runs this same verifier, the copy the BitExact release ships, which is byte-identical to the published one. It runs as an isolated process over the file you drop: a bundle (.json), a streamed bundle (.jsonl), a DSSE envelope, or an evidence pack's .zip. The same drop zone takes a signed regress-suite or model-qualification report, and shows its signature and the report's own verdict as two separate statuses. The page reads which kind of file it is from the file, and asks when the file does not say. Under Options you can pin the same keys: --expect-key, --expect-recorder-key, --expect-witness and --require-sealed. Unless you name a recorder key or untick the box, it pins the recorder keys the deployment's license names. Each capture key the deployment's capture policy revokes is passed as --revoked-capture-key, so the verdict names the fields committed under it as guessable from the date it was revoked, as bitexact verify does. The result is the verifier's own output and exit code, alongside the exact command that repeats the check. Run that command on your own machine: the viewer is part of the system you are auditing, and a check you run yourself does not depend on it.

A run's own Verify drawer gives the same route as numbered steps: download the evidence pack; from the folder holding bagit.txt, run python data/verification/bitexact_verifier.py ., the command the pack's runbook (data/verification/verify-procedure.md) gives; then run it again with --expect-recorder-key and the producer's key, confirmed with the producer by phone or signed email.

In the viewer, a run still being written that carries no checkpoint yet is shown as Recording, not as failing: its chain so far is intact and its writer signs at its first checkpoint or its seal. Its recorded content stays hidden until a licensed key has signed it, because whoever can write the store can also hold a run open. That is the viewer's verdict only. bitexact verify, an export and this verifier still refuse such a run under a recorder-key pin, and a sealed run with no checkpoint signed by a pinned key fails everywhere.

What a passing bundle proves — and what it does not#

A passing bundle proves the captured trajectory is intact (un-tampered), honestly redacted (a hidden field's commitment still stands), and complete enough to reconstruct — every value a replay would be served is present and covered by the chain. Note the limit: this procedure never runs a replay, and the verifier contains no replay engine. It proves the inputs to one are intact, not that one succeeded. Step 5 is where a party with a license can actually re-execute. When the success line says sealed it is also un-truncated: the seal attests the step count, and a recorder-signed run's seal must itself be covered by that recorder's last checkpoint, so nothing was appended or removed after the recorder finished. An unsealed verdict makes no claim about the end of the run. Under --expect-recorder-key a hidden field is additionally accountable: every redaction must be backed by a recorder-attested on-chain claim, so a keyless strip (a field withheld with no such claim) is refused rather than certified. A WORM anchor adds that the head was recorder-signed and placed in a store outside the run's own files, so a rollback of that store is detectable — its anchored_at time is asserted by the recorder, not independently established. An RFC 3161 anchor adds an independent authority's attestation that the head existed no later than the token's time.

It does not prove the trajectory is the complete set of calls the agent made. BitExact wraps the model and tool interfaces it is given, so a call the producer's code makes entirely out of band never enters the record and leaves no gap to detect. The determinism linter and replay-divergence detection surface many such gaps, but they do not prove completeness. A passing bundle is verifiable evidence that what was captured is intact and honest — not an attestation that nothing else occurred. The verifier checks what the format specifies and nothing more.

A worked example#

verifier/testdata/adverse-decision.bundle.json is a complete, signed sample you can run this procedure against: an automated loan decline — a decision about a person — that carries bound identity (the deciding principal, agent, model version, and the content hashes of the policy and prompt pack), the retrieval context the model was given, a credit-score tool call, the model's decision, a human loan officer's decision upholding it, and a marker for the downstream adverse-action notice BitExact does not capture. Verifying it with --expect-key and the key from its signature block returns exit 0. Its identity, context, human-decision, and marker entries are marked asserted and its tool and model calls observed, so what was attested is never confused with what was captured. examples/adverse_decision.py shows how it was produced — offline, with no API key.

The evidence pack (a workpaper you can hand on)#

For an examination file rather than a single bundle, bitexact evidence-pack <run> assembles a workpaper-shaped, self-verifying package: the signed bundles and fork graph of the provenance bag, plus a human-readable workpaper.md (cover, scope, exhibits, an assertions matrix with an honest limit on every row, and a blank conclusion the examiner writes themselves), a per-run verify-procedure.md runbook, an out-of-band trust index, and — with --verifier <path> — the standalone verifier pinned by its own SHA-256 so an air-gapped auditor needs no network. Everything is sealed by the bag's SHA-256 manifest; the pack verifies with the same procedure above, run against the whole directory. The pack states its scope boundary and never concludes compliance — it supplies the verifiable facts and leaves the judgment to you.

Every pack and provenance bag also carries the capture policy its runs opened under, as data/capture-policy-<digest>.json (the digest's : written as -): its detectors, field rules and budget, with each capture key named by its key id, never the key or where it is kept. The document's canonical digest (JCS, BLAKE2b-256) is the capture_policy the run's run_meta names, so you can check the one against the other. Only the policy in force when the pack is made can be carried: a run recorded under a policy since replaced, or with none in force (off), brings no document, and the workpaper and the reading view say so. The workpaper of every pack (a case dossier's runbook included) and the reading view show, for each run, every field a capture control touched with its declared outcome, the counts and the scope sentence a verdict prints, and quote each policy document the pack carries once. The verifier does not check the document against the runs; it names it after the OK line with the producer's other statements.

Every pack and provenance bag also carries data/view.html, a reading view: its runs as readable steps, opened in any browser you already have, offline. It is a reading aid and proves nothing - its first line says it is not a verification result, and it shows no verdict. What binds it to the evidence is the verifier: after the bag's checksums and every run verify, it renders the page again from the bundles and compares the bytes. A page that differs fails the pack (exit 2, data/view.html does not match the runs it describes), so a producer cannot bag a page that tells a different story; one that matches adds view matches the bundles to the OK line. If the page and data/runs/*.bundle.json ever disagree, the bundles are the record. The page carries no script, no external reference, no form and no link out of itself, so it reads the same with JavaScript off and under the strictest browser policy, and it is plain text you can read in a text editor. If your organization forbids opening HTML, read the bundles and workpaper.md instead, or ask the producer for a pack made with --no-view, which verifies without it. A pack handed to you without its page gets one from bitexact-verifier <pack> --render-view view.html: your own rendering, written only once the bag verifies.

The OK line on a pack is followed by the files the verdict does not stand behind: not checked against the runs (the producer's own statements, sealed against later change only), then every payload file it did not check against the runs - data/workpaper.md, data/trust.json, data/verification/verify-procedure.md and the like. Read those as the producer's account, not as evidence.

For a whole workflow rather than a single run, bitexact evidence-pack --workflow <name> assembles the model-file: every recorded run of the workflow as an independently-verifiable bundle, the workflow's append-only governance ledger (ownership, registered versions, promotions — a hash-chained record you re-walk with bitexact workflow verify <name>), and an inventory workpaper with the version × environment breakdown, any composition drift (a version label carrying more than one composition digest), and the model-inventory / change-control assertions — each with an honest limit and a blank conclusion. bitexact drift <name> reports drift on its own as a CI pre-flight; bitexact regress --workflow <name> --version <v> --entry <module:function> proves the behavioral delta across a version boundary.

A workflow or selection pack also carries, under data/refusals/, the refusals its filters select: executions BitExact refused to record (a lapsed license, a missing or unlicensed recorder key, a faulty signer or store), which ran with no run of their own. The verifier checks each as a hash chain whose signatures must verify, and counts them as declared capture gaps - an OK line with recorded refusals, exit 3 unless --allow-gaps - saying how many your pinned recorder key signed. A case pack carries only the refusals recorded under its pinned run ids, and a single-run pack none; each workpaper says so, because a refusal missing from such a pack does not mean none occurred.

For an investigation rather than a run or a workflow, bitexact evidence-pack --case <id> assembles the case file: every run pinned into the case as an independently-verifiable bundle, the case's append-only chain-of-custody ledger (who pinned which run, who recorded which finding, when — a hash-chained record you re-walk with bitexact case verify <id>), and a workpaper with the pinned-run inventory, the custody history, the recorded findings, and the issue-tracking / record-integrity assertions, each with an honest limit and a blank conclusion. A run pinned by id but not recorded in this store is documented in the inventory, never silently dropped.

A note or finding in a case can be about one run the case has pinned, and one step of it: bitexact case note <id> --run <run> --step 3 --text …. A finding about a whole run, with no step, is the reviewer's sign-off on it. The viewer gathers every review of a run, from every case that pinned it, on that run's Reviews tab (GET /api/runs/<run>/reviews). Reviews live in the cases' own ledgers, beside the evidence: they never change a run's chain and are never an input to its verdict. A review of a run the case has not pinned, or of a step the run does not have, is refused.

A selection an examination asks for again - "the Q3 claims sample" - can be kept in the pack library: bitexact evidence-pack --workflow claims --since … --save q3-claims, then bitexact evidence-pack --pack q3-claims (or the viewer's Pack library) each time it is due. The library keeps the selection, never the evidence: each issue resolves it against the store at that moment, so a run purged or redacted since is reflected, and each issue is recorded in every included run's custody chain as pack:q3-claims.