BitExact guide

Self-hosted evidence instrument for AI agents. Reconstruct a decision, replay it deterministically, and let anyone verify it without trusting you.

See it in action: watch a lending decision get proven, then improved: one real run recorded, replayed bit-for-bit when the decision came under review, then forked to test a corrected policy. Anyone can re-derive the whole record offline with the open verifier.

Overview

BitExact records every model call, tool call, and captured source of nondeterminism that runs through the proxy or a wrapped client into a hash-chained manifest, then replays every value the record kept bit-exactly with zero network and zero side effects (a value the capture policy scrubbed or a redaction removed is refused, never guessed). Any divergence between the record and a re-execution halts with the exact step and a diff. Runs export as signed evidence bundles that anyone can check with the open-source verifier — no BitExact license required. Everything runs inside your VPC; nothing phones home. BitExact proves the integrity of what was captured, not that the capture is complete: a call made entirely out of band is outside the record. The determinism linter flags code that would bypass capture - direct HTTP calls, unwrapped model clients - before it runs; a call made entirely out of band at run time leaves no gap to detect.

Install

# from your platform's release bundle (Python 3.15+), no index needed:
pip install --no-index --find-links wheels --require-hashes -r requirements.lock
pip install --no-index --no-deps bitexact_sdk-<version>-*.whl
# or load the image we deliver and run the recording proxy:
IMAGE=$(docker load -q -i bitexact-<version>-amd64.tar | sed -n 's/^Loaded image: //p')
docker run -p 8787:8787 -v bitexact-data:/data -v $PWD/secrets:/secrets:ro \
  -e BITEXACT_LICENSE=/secrets/bitexact.license \
  -e BITEXACT_SIGNING_KEY=/secrets/recorder.key \
  $(docker image inspect --format '{{.Id}}' "$IMAGE")

Both are delivered to you with each release, with a SHA256SUMS file and the image's cosign signature. The vendor key licenses are checked against is built into both.

Record a run

import bitexact
from bitexact.adapters import wrap_openai   # or wrap_anthropic

@bitexact.record_tool
def fetch_balance(account_id): ...

with bitexact.run("run-42") as r:
    client = wrap_openai(openai.OpenAI())
    balance = fetch_balance("acct-104")
    answer = client.chat.completions.create(...)

Model traffic flows through a loopback recording proxy bound to the run; tool results and time.time / random.random / random.randint / uuid.uuid4 values are captured into the same manifest. API keys are forwarded upstream and never recorded. The proxy speaks OpenAI, Anthropic, Gemini, Vertex AI, Azure OpenAI and Bedrock, streaming included (providers).

Replay — bit-exactly, offline

with bitexact.run("run-42", mode="replay"):
    ...  # identical code; zero network, zero tool side effects

Every call is matched by identity and content against the recording: a changed prompt, different tool arguments, an extra call, an early exit, or any recorded step left unserved raises DivergenceError naming the exact step with a recorded-vs-actual diff (unserved steps are listed by identity). Pure thread-scheduling reorder is not a divergence — replay verifies calls and content, not the scheduler. A tampered manifest refuses to replay at all. Set the mode without code changes via BITEXACT_MODE.

Counterfactual fork & fork science

with bitexact.fork("run-42", at_step="http_call:2") as f:
    ...  # everything before the 2nd model call served, the rest runs live

stats = bitexact.fork_matrix("run-42", 7, 20, agent_fn)
#  -> outcome counts across 20 forks: is this outcome typical, or a fluke?
#     agent_fn may take the run or take nothing and use bitexact.current()

mutated = {"status": 200, "media_type": "application/json",
           "text": '{"choices": [{"message": {"content": "APPROVE"}}]}'}
bitexact.fork("run-42", at_step=9, inject={4: mutated})
#  -> "what if the model had said X at step 4?" - recorded as `injected`;
#     a replacement is the recorded-response shape: status, media_type, text

The fork writes its own manifest, opened with a provenance entry hash-linked to the source prefix — a counterfactual is itself verifiable evidence, and bitexact verify RUN --provenance walks the entire fork ancestry, hash-checking every hop against its verified source and surfacing synthetic responses anywhere in the chain. Fork points address by step or by kind ("tool_call:2"). A fork matrix runs N forks from one step and aggregates outcomes, so "was this failure statistically typical?" is answerable from counts. A response-mutation fork replaces a recorded response and continues — recorded as an explicitly synthetic injected entry, so an experiment can never be laundered into run evidence. bitexact forks prints the provenance tree.

Determinism linter

bitexact lint src/ --format sarif   # stable BX rule ids, JSON/SARIF, inline suppressions

The capture matrix and the linter derive from one table, so they can never disagree about what replays. Captured during a run: time.time/time_ns/monotonic, random.random/randint/ choice/uniform/shuffle/getrandbits/randbytes, uuid.uuid4, os.urandom, and the secrets token functions. The linter flags the rest — datetime.now, perf_counter, uuid1, unknown random.*/secrets.* calls — and from time import time-style imports that bind functions directly and defeat interception. Exit code 1 when findings exist.

Frameworks & concurrency

@bitexact.record_tool          # sync or async: both record and replay
async def fetch(...): ...

pool.submit(bitexact.propagate(tool), args)   # thread pools, explicitly

from bitexact.adapters import temporal_run
with temporal_run():                          # inside a Temporal activity
    ...                        # one evidence trail per execution, per retry

Replay matches each call by identity and content, so parallel graphs (a LangGraph fan-out with an async tool and an async model client is in the test suite) replay correctly even when scheduling differs from the recording. Capture follows the run context: asyncio tasks, executor calls and propagated pool tasks are captured. An unpropagated pool tool call, or one through a propagate() wrapper created outside a run, still runs but is not recorded, and every run recording at that moment declares a run_context_lost gap naming the tool, so verify exits 3 rather than calling the capture whole; a task that outlives its run is noted beside that run and never touches its sealed chain. Strict mode raises instead. Agent code that itself lives under site-packages must be named in BITEXACT_AGENT_PACKAGES to be captured. Runs are isolated per context: one run per thread or task, and any number of them at once, so a Temporal worker records concurrent activities with no limit on max_concurrent_activities; each activity execution and each retry is its own evidence trail. bitexact otel-export renders any verified run (with the otel license entitlement) as OTel GenAI span events — zero-duration markers derived from the evidence, one shared trace per run — for the observability stack you already have.

Recording secrets.* or os.urandom writes the generated values into the manifest — that is what makes them replayable, and it means evidence can contain live key material. The recorder warns once per run; redact before sharing (bitexact redact --fields STEP:value) and rotate any secret that replay or fork feeds into live traffic.

Determinism qualification

bitexact qualify --model gpt-4o --prompt "..." -n 20
bitexact qualify --wire anthropic --model claude-sonnet-5 --prompt "..." -n 20

Sends the identical request N times and compares the model-controlled bytes exactly: exact-match rate, distinct outputs, first divergence with context. The report is self-hashed.

Regression across a run corpus

bitexact regress run-1 run-2 --entry myagent:main \
    --intended http_call:request.text.messages \
    --git-sha $GIT_SHA --sign --key signing.key
# nonzero exit on proven unintended divergence — a drop-in CI gate

Regress replays recorded runs against your changed prompt or code, offline and with zero model spend. Every place the new code behaves differently becomes a finding: flipped prompts and tool decisions (with recorded-vs-actual diffs), brand-new calls (a typed UnrecordedCallError, never a silent None), and recorded steps no longer taken, reported by identity. Declare the change you meant to make with --intended, and first_divergence lands on the first unintended one. Mind the grain: an intended path marks everything beneath it, list contents included, so declare the narrowest path that covers your change and review the flip diffs the report keeps. Reports carry source head hashes, SDK version, timestamps, the git sha and label as supplied by the caller, and an optional signature a recipient checks with bitexact verify-report; gate is the machine-readable CI verdict. Reports never overwrite history: they accumulate beside the manifests, so prune on your own schedule. A sweep is crash-honest: an aborted run reports the exception and the step it reached, and one crashing run never stops the sweep. Offline regress shows the calls your change makes, not how the model would answer them. That ceiling is by design; confirm leads with a live fork at the divergence step.

Local viewer

bitexact view   # http://127.0.0.1:8788 — the inspector

Optionally token-gated (BITEXACT_VIEWER_TOKENS), paginated, with on-demand payloads, head-hash-cached verification, fork provenance, and regress reports surfaced. Browse recorded runs, per-step detail, and chain-verification status in the browser. The viewer never edits or deletes a recorded entry. It does act, and each act is recorded as evidence of its own: a legal hold, a fork, a two-person redaction (which destroys the named values in place and keeps their commitments), a regress, a suite run, a case. Every such route needs the X-BitExact-Action header and a role that may perform it; the route list is in the API reference.

For the people who read the evidence there: the inspector guide for compliance and audit staff. Signing in, roles, SSO, the read-only token compliance staff should be given, and the sign-in throttle: identity and access. How an auditor checks a bundle independently: verifying a bundle.

Concurrent agents

with bitexact.run("run-42", mode="replay"):
    ...  # asyncio.gather-style calls match by identity and content

Identical concurrent calls to one source consume their recorded values first-to-last — which recorded value lands in which racing task is scheduler-dependent, the documented ceiling for exact duplicates. A tool served from the record absorbs the nondeterminism it recorded internally, so tools with internal clocks or jitter replay clean. http_matching="unordered" remains accepted and now describes regress mode's content matching.

PII redaction

bitexact redact run-42 --fields 3:request.text     # the stored manifest
bitexact export run-42 --redact-steps 7 --sign --key signing.key

Every committed field carries its own salted commitment, so redaction is per-field: remove the prompt, keep the proven status, model, and usage. Either way the chain and any signature still verify, and the destroyed salt defeats dictionary attacks. bitexact redact rewrites the stored manifest itself, write-ahead: a chained audit entry naming the fields and the operator lands on the chain before any value is destroyed, and the CLI warns if quarantine or stream-journal files may still hold copies so the operator can complete the erasure. Export-time redaction (--redact-steps) instead produces a redacted copy for a recipient and leaves the store untouched. A redacted bundle proves what it omits was committed to, not what it said.

Evidence bundles & the verifier

bitexact keygen -o signing.key
bitexact export run-42 --sign --key signing.key -o run-42.bundle.json
bitexact-verifier run-42.bundle.json --expect-key <public key>

Bundles follow the bundle spec (aer-bundle): a BLAKE2b-256 hash chain (SHA-256 in FIPS mode) over every step, recorder-signed checkpoints when a recorder key is configured, plus an optional detached signature at export. Signatures are hybrid Ed25519 + ML-DSA (FIPS 204), the key bitexact keygen makes: both halves sign the same bytes, so a relying party who pins the ML-DSA key (--expect-pq-key) refuses a forger who recovers the Ed25519 key with a quantum computer, and every verdict names the suite it checked, Ed25519 only included. A recorder key held in an HSM keeps both halves in the token; the ML-DSA half is exercised against a real kryoptic PKCS#11 3.2 token (key custody). The verifier is a standalone Apache-2.0 tool an auditor runs independently: github.com/bitexact/bitexact-verifier, the standard library plus cryptography and asn1crypto, no product code. Its real verdict on a signed four-step run:

$ python bitexact_verifier.py run-42.bundle.json --expect-key <public key> --expect-pq-key <its post-quantum half>
OK: run-42 - 4 steps (observed 4 / asserted 0 / synthetic 0), chain verified, sealed, signed, signatures: Ed25519 + ML-DSA-65 (post-quantum), post-quantum key pinned
standing when recorded: ok (stated by the producer, not proof)

Licensing

License keys are files signed with the vendor's hybrid Ed25519 + ML-DSA key and validated entirely offline, with no code path that could phone home. bitexact license shows status. BitExact is licensed use only: without a valid key nothing records or replays (agents' calls still pass through the proxy, unrecorded, unless strict mode is on), and a license bound to a recorder key records only under that key, one deployment per key. When a license expires, recording continues through a grace period (14 days unless the license names another); after it the deployment becomes read-only: nothing new is produced, and everything already recorded stays listable, verifiable and exportable. The commercial shape is on the licensing terms page.

Without a valid license only bitexact capture keygen, bitexact keygen, bitexact license, bitexact pki, bitexact recorder-key, bitexact version and bitexact witness answer. bitexact collector, bitexact mcp-proxy, bitexact recorder and bitexact serve start in any license state and check the license on each call: they pass the agents' traffic through, or declare a gap, rather than record.

Environment reference

Every variable the product reads, generated from the code (a test fails when one is missing). The full command reference is the CLI page; which acts are done from the command line rather than the inspector, and why, is administration from the command line, and monitoring and alerting is ops health.

CLI reference

Every command. License names the entitlement a command requires — a license that does not grant it refuses the command. Commands marked — need no entitlement; the ones that need no license at all are listed under Licensing above, and say so below.

Evidence is portable and standards-shaped: hashes are computed over canonical JSON (aer-jcs, a restriction of RFC 8785 to values that canonicalize identically in any language), signatures carry key ids and signed timestamps with trust-file rotation, exports can be wrapped as in-toto/DSSE envelopes for cosign-style tooling, and BITEXACT_HASH_ALG=sha256 selects SHA-256 chain hashes and HMAC-SHA256 commitments for FIPS-constrained environments (key ids remain BLAKE2b-derived labels; they identify keys and carry no security weight). Key procedures: rotation runbook.

Recording never stands between an agent and its provider. When the recorder cannot write, the call is delivered, its response is marked x-bitexact-recording: degraded, and the run records a declared capture gap naming the cause, which bitexact verify reports (exit 3) rather than passing the run as complete. With BITEXACT_FAILURE_MODE=strict recording is fail-closed instead: non-streaming calls return an error rather than an unrecorded response, and a streamed response, passed through a write-ahead journal, is truncated rather than delivered unrecorded. Every read cross-checks a per-run head record, catching crash truncation, and when BITEXACT_SIGNING_KEY names a key file, signed, hash-linked checkpoints attest the chain head every BITEXACT_CHECKPOINT_EVERY steps. The precise guarantee: the local store detects in-place tampering; anti-rollback against an actor with data-dir write access comes from holding an exported signed bundle — whose signature pins the head and the full checkpoint set — or an externally anchored head hash. One writer per run, enforced with an OS-level lock; the data dir must be a local filesystem.

Commercial terms: pricing. See also the compliance mapping and the replay taxonomy.