Capture controls#

BitExact records evidence; it never changes what an agent sends or receives. The capture policy changes only what the record keeps of it. It runs before anything is written - where every entry is built, and where every remote event enters a host's collector - so a value the policy removes never reaches a spool, the wire, a store, an index, a log line or a backup.

What the record keeps of a field#

Every outcome but kept is declared in the entry's capture block, inside the entry hash and every signature over it (bundle-spec.md, Capture controls). A verdict counts them, from the declarations only, in a clause it adds to its OK: line. For a run with two scrubbed fields (two card numbers found in one, an IBAN in the other) and one hash-only field committed under capture key 3a9f…, the clause is:

, 2 fields scrubbed before capture (iban 1, pan 2), 1 field hash-only (never held; 1 can be opened with key 3a9f..)

and the verdict states its scope on a line of its own:

Scrubbed and hash-only fields prove that a value was presented and which rule removed it. They do not prove what the value was, they cannot be replayed, and hash-only custody is not HIPAA de-identification.

What a scrub proves is narrow: that the scrubbed value is what was committed. That the original held what the detector found is the recorder's word.

A run recorded with the policy off says so from its first entry (run_meta.capture_policy is off), and the verdict on it says so first in the clause:

, no capture policy was in force

followed by a scope line of its own, since such a run removed nothing:

A run recorded with no capture policy in force kept every value as captured, card numbers, IBANs and credentials included.

A bag or pack of more than one run counts its runs recorded that way instead: , 2 runs recorded with no capture policy in force. When it also holds fields a policy scrubbed or kept hash-only, its scope line states both sentences, the one above first.

What each detector finds, and what it cannot#

A detector reads text: every string at any depth, every number as its JSON text, every object key, and a JSON object or array of any size inside a string as the document, every value a repeated key shadows included (so neither an escaped digit nor a duplicate key can hide a card number). A space or hyphen as a URL encodes it (%20, %2D, and a query's +) reads as the separator it encodes, so a number spaced in a request path or query is found as it was written. It finds what has the shape and passes the check it names. Nothing else.

A value split across stream chunks is found: the policy scans a streamed response once it is reassembled, never chunk by chunk. A value split across separate fields is not found by any detector.

Why a plain hash of a card number protects nothing#

A 16-digit card number whose issuer prefix (6 to 8 digits) and last four are known has 4 to 6 unknown digits (16 - 6 - 4 = 6 at most), and the Luhn check fixes one of them: at most 10^5 candidates, and 10^3 with an 8-digit prefix. Against an unkeyed hash - SHA-256 of the number, or a commitment whose salt is stored beside it - that is a fraction of a second on one core. A 4-digit PIN is 10^4 candidates, and a date of birth within a century about 4 x 10^4 (36,525 days). Any digest an attacker can recompute without a secret is no protection for these values.

A hash-only field's salt is the first 128 bits of an HMAC-SHA256 of its nonce and path under the 256-bit capture key K, which the evidence never holds. Without K, each of those 10^5 candidates must be tested against a salt the attacker cannot compute: the search is over the salt (2^128 for each field) or the key (2^256), not the card number. The per-field nonce means equal values never give equal commitments (unless a rule makes the field linkable, below), so not even a key holder learns anything without testing field by field. That is why a scrub keeps the last four in its token but never hashes the number: the PCI correlation problem of a hashed and a truncated copy of one PAN does not arise.

The policy#

With no configuration, the shipped default is in force: card numbers (issuer prefix and Luhn, in any script's digits), IBANs (country length and mod-97) and credentials (by key, key=value, Bearer/Basic) are scrubbed everywhere, within 1000 ms for each MiB a step holds. BITEXACT_CAPTURE_POLICY names a JSON file to use instead, a shipped preset (pci), or is off:

{
  "detectors": {"pan": {"keep_last4": true}, "iban": {}, "credentials": {},
                "us_ssn": {"require_context": true}, "email": {},
                "custom": [{"name": "account", "pattern": "ACCT-\\d{8}"}]},
  "fields": [
    {"kind": "identity", "path": "on_behalf_of", "mode": "hash-only"},
    {"kind": "tool_call", "where": {"tool": "kyc_lookup"}, "path": "result",
     "mode": "hash-only"},
    {"kind": "*", "path": "*", "mode": "scrub"}
  ],
  "budget_ms": 1000,
  "hash_only_key": {"file": "/run/secrets/bitexact-capture.key"},
  "retired_keys": [{"file": "/run/secrets/bitexact-capture-2026-q3.key"}],
  "revoked_keys": {"3a9f0c1d2e4b5a6978c0d1e2f3a4b5c6": "2026-10-01"}
}

• fields are rules, first match wins: a step kind (or *), an optional where on the step's own fields, a path (or *; a rule on a key covers what is under it; *.NAME covers every path whose last key is NAME) and a mode - keep, scrub or hash-only. A path no rule names is kept. A hash-only rule may add "linkable": true (below). A rule that could never match is refused when the policy is loaded, since the value it was written for would be stored as captured: a kind that is not an entry kind the policy governs, or a path that is not KEY, KEY.SUBKEY, * or *.NAME.

• budget_ms is how long the detectors may take for each MiB a step holds as JSON (default 1000, a whole number from 1 to 10000): a step's deadline is budget_ms times its size in MiB, and at least one MiB's worth for a small step. It is never a fixed time per step, so a large payload is given time in proportion and is never withheld for its size alone. The detectors are linear - about 60-110 ms per MiB of text or JSON, and 330 ms for a MiB of digits, on a development machine - so the default bounds a runaway scan with room to spare on slower hardware. Each detector checks the deadline at every match, run of digits or group it considers, so a scan stops at the deadline rather than finishing the text; the one thing it cannot interrupt is a single regular-expression search between two matches, which is linear in the text. A step that runs out at that rate has its unscanned fields withheld and declared, never written raw. A detector that raises is treated the same way. A key the detectors fail on is not kept either: it is stored as [withheld key], and every field under it is withheld. The agent's call is never affected.

  No agent call waits for more than one budget of scanning. An entry no larger than one MiB is scanned and written in the call. A larger one is scanned and written after the call has returned, in the order the agent recorded it:

  • the recording proxy hands the agent its response at once and keeps the response in a journal sealed under the host key until it is recorded - if the process dies first, the journal is left as an orphan journal that bitexact repair names, as a stream's is;
  • the Python SDK's Run.record returns a RecordHandle, a read-only mapping that becomes the written entry: reading it (handle["step"]) waits until it is written, at most BITEXACT_DRAIN_TIMEOUT seconds (default 30), and reads as empty while it is still unwritten; and handle.result(timeout) waits at most timeout seconds (that bound when none is given). Before the call returns, the entry as it stood at the call is synced to a journal sealed under the host key, and the journal is removed once the entry is written. If the process dies first, bitexact repair writes every such journal into the chain, in the order the agent recorded them, and adds a repair entry naming them, so an entry whose call returned is never lost. An entry that cannot be written is declared as a gap and the handle becomes empty; its journal stays as an orphan journal. The run's exit writes every queued entry before it seals, waiting at most BITEXACT_DRAIN_TIMEOUT seconds: past it the agent's exit returns, each entry still unwritten is declared a storage_unavailable gap in the run's failures record, naming its journal, and the run is left unsealed - its journals stay for bitexact repair to fold.

  Strict mode always writes in the call, since it answers only once the evidence is written. One entry's scan time is bounded by budget_ms times its size in MiB, and its size by BITEXACT_MAX_BODY_BYTES.

• custom detectors take a pattern built only from literals (letters, digits, space and _ : / # @ -), classes [..], the escapes \b \d \w \s \D \W \S, the escaped literals \. \- \_ \: \/ \# \@ \+ (a bare . is no literal: \. is the one way to write a dot) and bounded quantifiers up to 64 - no groups, alternation or unbounded repetition, and at most one quantifier that varies in length (? or {n,m} with n < m; {n} is fixed) - so every pattern matches in time linear in the text and none can stall a write. \d{4}-?\d{4} is accepted; \d{1,8}-\d{1,8} is refused.

• hash_only_key is the capture key, from the same sources as the recorder key (recorder-key-custody.md): 32 bytes as 64 hex characters in a file ({"file": PATH}; bitexact capture keygen -o FILE makes one), the same KMS-encrypted ({"kms_ciphertext": PATH}), an AWS KMS HMAC key of key spec HMAC_256 ({"kms_hmac": KEY_ID}, its id, ARN or alias), or a secret key in a PKCS#11 token ({"pkcs11": {"lib": PATH, "slot": N, "label": L, "pin_env": VAR}}, the PIN read from $VAR, never written in the policy). A file or KMS-encrypted key is held in the process that uses it. A KMS HMAC key or a token's key never leaves KMS or the token: every use of it is an HMAC-SHA256 that KMS (GenerateMac, HMAC_SHA_256) or the token (CKM_SHA256_HMAC) computes. A token that stops answering mid-run never stops the call: each hash-only field is committed under a random salt and declared capture_key_unavailable, and a digest the recorder would have keyed - a gap's withheld digest, a relayed event's aer_source.sent - is taken out and declared withheld with the cause capture_key_unavailable; replay treats a field it cannot open as withheld. With no hash_only_key, BITEXACT_CAPTURE_KEY names a key file. The key never appears in evidence, and must be kept apart from it.

• retired_keys lists the keys of earlier rotations, from the same sources. A retired key commits nothing new; every field committed under it still opens.

• revoked_keys maps the id of a key that may be in other hands to the date from which it may be. A verdict names the fields committed under a revoked key: whoever holds that key can test guesses at them.

Presets#

BITEXACT_CAPTURE_POLICY=pci selects the shipped PCI preset: the default's detectors on every path, and every base64 value (*.b64, the bytes the detectors cannot read) kept hash-only instead of unscanned, so no card number can be stored inside bytes nobody scanned. Give it a capture key (BITEXACT_CAPTURE_KEY): with none, each such field is committed, never stored, never openable, and declared a capture_key_unavailable gap. record_context_from_bytes records its bytes as content.b64, so they follow the same rule: kept and declared unscanned under the default policy, hash-only under pci, and hash-only under any policy when the call passes custody={"content.b64": "hash-only"}.

Rotating a key#

Make the new key, name it hash_only_key, and move the old one to retired_keys. New fields are committed under the new key; fields under the old one open as before, and a policy that no longer holds a field's key says so by the key's id. Each opening's custody record names the key the candidate was tested under, the retired one for a field committed before the rotation. A key that may have been taken goes into revoked_keys with the date: rotate away from it, keep it in retired_keys while its fields must still open, and every verdict over those fields says they are guessable by its holder from that date.

Linkable hash-only fields#

A hash-only rule with "linkable": true commits its field without a nonce: the salt is the key's alone, so an equal value gives an equal commitment wherever it is committed under that key, and records can be joined on it without the value. That is also what anyone holding the evidence can see: which fields, steps and runs hold the same value. A linkable field is exactly the correlation that GDPR's pseudonymization analysis (EDPB Guidelines 01/2025) and the PCI SSC's guidance on hashed and truncated PANs warn about, and it is still not HIPAA de-identification. Off unless a rule asks for it; never set by an agent's hint; with no capture key it is an ordinary unopenable hash-only field. The verdict counts the linkable fields.

A policy that cannot be used refuses the process at start: the recorder and the collector do not start, and an SDK run records nothing (the agent runs on). The recording proxy keeps forwarding every call and declares each one it cannot record, as it does for any recording failure. Every run records in its first entry which policy was in force (its digest), or off.

A policy's digest names its capture key by key id, never by where the key is kept: moving the key file is the same policy, a new key is not.

Hints from the agent#

An agent can ask for a field to be kept more strictly than the policy says, never less strictly:

r.record("tool_call", {"tool": "kyc", "result": result},
         custody={"result": "hash-only"})
r.record_tool_call("kyc", args=[name], result=result,
                   custody={"args": "scrubbed-upstream:presidio",
                            "result": "hash-only"})

@bitexact.record_tool(custody={"result": "hash-only"})
def kyc_lookup(name): ...

The TypeScript, Go and Java SDKs take the same on their tool wrappers: recordTool(name, fn, { custody }), RecordTool(name, fn, WithCustody(...)), recordTool(name, custody, fn). A hint on a key covers what is under it, as a rule's does. A wrapper given a custody that is not one refuses it where the tool is wrapped, never in a call. So does record_tool given a custody for a tool already wrapped with another: the first wrapping records its calls, and would keep them without the custody asked for.

scrub and hash-only make a field stricter, with the policy off too (BITEXACT_CAPTURE_POLICY=off): a scrub hint then runs the shipped default's detectors (card numbers, IBANs, credentials) on its field and declares the scrub, and a hash-only hint commits the field with no capture key and declares capture_key_unavailable. Turning the policy off never keeps a hinted field as sent, and never withholds it as a scrub that failed. scrubbed-upstream records the agent's claim that it cleaned a value before BitExact saw it, as its claim: the verdict counts it as "claimed cleaned upstream (asserted, not checked)", never as a scrub. scrubbed-upstream:TOOL names the tool that cleaned it (scrubbed-upstream:presidio), declared as "by": "presidio". Every SDK takes the same hints (protocol vectors, custody_hints, with an encoded tool call). A hint that is not one withholds the entry and declares an invalid_argument gap: kept without the custody it asked for, the value could be stored where it must not be.

Collectors#

A collector applies its host's policy before it spools an event and sends what it did with it; the recorder declares it under the collector's name (applied_by). A collector cannot claim another's name, and a local process cannot claim a collector's work: both are refused. The recorder applies its own policy again, and applying it twice is safe: a field the collector already declared, and that the recorder leaves as it arrived, is not declared again, and the entry stays the collector's. A field the recorder does change is declared with its own applied_by (recorder) and policy beside the collector's entry-level ones. An event that arrives holding a value the recorder's own policy removes is caught there, and bitexact health names the collector (collector-unscrubbed): the values crossed the wire and sat in its spool. A field the recorder only declares unscanned removes nothing and names no one.

Opening a hash-only field#

A holder of the capture key can test whether a value is the one a hash-only field committed:

echo '"user:7f3a9c"' > candidate.json
bitexact verify RUN --open 4:on_behalf_of --candidate-file candidate.json --by alice

The candidate file holds the value as JSON. Once the run verifies, the command prints whether it matches, and exits 2 when it does not. Testing a candidate is the same act as guessing it, so every test is recorded on the run's custody chain first, under --by. A field committed with no capture key can never be opened, and is a declared capture_key_unavailable gap. bitexact verify RUN --open-withheld 9 --candidate-file FILE --by alice tests the file's exact bytes against the content the gap at step 9 withheld, recorded the same way.

A hash-only field is a keyed hash of its value. It is pseudonymous, not de-identified: under HIPAA a code derived from an identifier does not de-identify it (45 CFR 164.514(c)), and under GDPR the key is the "additional information" that must be kept apart for the data to be pseudonymous (Art. 4(5)). Treat evidence holding hash-only fields as personal data.

Backfilling a recorded run#

A run recorded with the policy off, or under an earlier rule set, can hold what the policy in force removes. Backfill never rewrites it. It files one dual-control redaction request for every such field, and the existing, accountable redaction destroys them once a different principal approves:

bitexact capture backfill --run RUN --by alice
bitexact redaction approve RUN --by bob

The request names, by STEP:PATH, every stored field the policy in force keeps hash-only, and every field whose value or path a detector finds something in (or runs out of time on, which the scrub would have withheld). It skips fields already redacted, the product's own blocks (aer_source, a gap's declaration) and b64 bytes no detector reads. Its reason names the policy's digest. Until the approval, nothing on the run changes; bitexact redaction status RUN shows the pending request and bitexact redaction reject RUN --by bob --reason TEXT rejects it. After it, the run verifies with those fields redacted. A run that holds nothing the policy removes prints nothing to request and exits 0.

Backfill exits 1 when the policy is off (there is nothing to backfill against), when the run does not exist, when a redaction request is already pending for the run, and under BITEXACT_REQUIRE_DUAL_CONTROL: there, --by is typed, not authenticated, so request the redaction in the viewer.

What else the policy reaches#

• The record's own digests. Under a policy, a gap's digest of content it did not store is keyed with the capture key (or dropped with no key), and the recorder drops its digest of a relayed event its own policy changed. Neither lets a guess at unstored content be tested.
• Logs go through the detectors; a log holds no commitments to declare a scrub in, so it is scrubbed, never declared.
• The stream journal - the proxy's write-ahead copy of a streamed response, before the policy has run on it - is always sealed, whatever the policy and with or without a capture key, in the envelope the collector's spool uses: each chunk sealed with ChaCha20-Poly1305 under the journal's own random key, which is stored wrapped by the host key. The host key is the file BITEXACT_HOST_KEY names, or <data directory>.host.key beside the data directory, made owner-only on first use; never inside the data directory, so a copy of the store carries nothing that opens it. A journal a failed finalization leaves behind opens only on that host (bitexact capture journal FILE). The policy scans the reassembled body when the journal is folded in, so a card number split across two chunks is still found.
• The index holds only what the entry stored.

Replay, regress and fork#

A scrubbed, hash-only or withheld value is not what was observed.

• Replay refuses to serve one, by step and path. With accept_scrubbed=True (--accept-scrubbed) it serves a scrubbed value as scrubbed; a hash-only or withheld one is never served. The live call is put through the policy the entry was recorded under before it is compared, so a scrub that reproduces is a match, and a hash-only field is compared by opening it with the capture key.
• Regress calls a step whose difference lies where the record holds no value withheld, never diverged - the same rule a redaction follows - and a run with such a step leaves a suite's divergence rate. With the capture key, a hash-only difference is proven, and is a divergence.
• A fork past a step it cannot serve is refused before the agent runs, and a fork's copy of a matched call keeps what its source kept.
• Every result says when scrubbed values were accepted. A regress report, a regress suite file (inside its signature), a suite result and comparison, and a fork matrix carry accept_scrubbed; a fork carries it in its chain. verify-report, the command line and the inspector say "scrubbed values served as scrubbed" beside the verdict, because the code was compared against the scrubbed record, not the observed values.

When the policy in force is not the one an entry was recorded under, none of that entry's altered fields can be compared. Replay refuses such a step with ReplayRefusedError ("replay refused at step N: the capture policy changed since this run was recorded", naming the digest it was recorded under, the one in force and the fields), whether or not the live call also differs elsewhere, and never reports it as a divergence. A redacted step, or one whose served value the policy kept as less than it was, is refused the same way. ReplayRefusedError is a DivergenceError, so code that stops on a divergence stops on a refusal too.

Opening a hash-only field to compare it is testing a candidate, the same act as verify --open, so replay and regress record each opening on the source run's custody chain first: an open-hash-only event whose role is replay or regress and whose principal is the agent the run names with actor (or the role, when it names none), naming the step, the path and the capture key's id, never the live value. A field whose opening cannot be recorded is not opened: it is withheld, never compared unrecorded (strict mode raises instead).

Settings and health#

The inspector's Settings › Capture policy shows the policy in force, its detectors, its rules (a linkable rule says so) and its key id; each step's Integrity tab names what was kept of each field, and a run's Capture scope tab counts it. bitexact health warns on capture-key-missing (hash-only rules and no key), capture-key-unavailable (per run: the fields a rule or an agent's hash-only hint kept hash-only with no capture key, so never openable, with their paths) and scrub-failed (fields the policy could not scrub in time, withheld as a declared gap), and is critical on capture-policy-invalid and collector-unscrubbed. capture-policy-off warns, and is critical when BITEXACT_ENV=prod. capture-key-revoked names each revoked capture key id and the date it was revoked: it warns while the active key is not among them, and is critical when it is, since new hash-only fields are then committed under a key in other hands.

bitexact capture policy show prints the policy in force, bitexact capture policy check FILE and bitexact capture policy digest FILE try one before it is deployed, and bitexact capture test TEXT shows what the policy in force scrubs from a text.