Model providers#

The recording proxy (bitexact serve, or the endpoint of an SDK run at run.base_url) speaks each provider's own wire. Point the client's base URL at it and keep the client's own credentials: the proxy forwards them to the provider and never records them. It routes each call by its shape, and a call is only ever sent to its own provider's host.

Name the run each call belongs to with the X-BitExact-Run-Id header. An SDK run's own endpoint needs no header: everything sent to it is that run's. A wrapped client called inside a recorded tool also sends X-BitExact-Ctx, naming that tool. The proxy records it as the entry's ctx and never forwards it upstream, so replay retires the call together with the tool it served.

Provider	Point the client at	Upstream setting	Default
OpenAI	`http://<proxy>/v1`	`BITEXACT_UPSTREAM`	`https://api.openai.com`
OpenAI-compatible (vLLM, Ollama, TGI, LiteLLM)	`http://<proxy>/v1`	`BITEXACT_UPSTREAM`	set it to the server
Anthropic	`http://<proxy>`	`BITEXACT_UPSTREAM_ANTHROPIC`	`https://api.anthropic.com`
Google Gemini API	`http://<proxy>`	`BITEXACT_UPSTREAM_GEMINI`	`https://generativelanguage.googleapis.com`
Vertex AI	`http://<proxy>`	`BITEXACT_UPSTREAM_VERTEX`	`https://aiplatform.googleapis.com`
Azure OpenAI	`http://<proxy>`	`BITEXACT_UPSTREAM_AZURE`	none: required
Amazon Bedrock	`http://<proxy>`	`BITEXACT_UPSTREAM_BEDROCK`	none: required

Azure and Bedrock hosts are your own resource and region, so they have no default. Until one is set, a call of that shape is refused with a 502 naming the setting, rather than sent anywhere with its key.

Clients#

# OpenAI
OpenAI(base_url="http://localhost:8787/v1",
       default_headers={"X-BitExact-Run-Id": run_id})

# Anthropic
Anthropic(base_url="http://localhost:8787",
          default_headers={"X-BitExact-Run-Id": run_id})

# Gemini API (google-genai)
genai.Client(api_key=key, http_options=types.HttpOptions(
    base_url="http://localhost:8787",
    headers={"X-BitExact-Run-Id": run_id}))

# Vertex AI (google-genai), with BITEXACT_UPSTREAM_VERTEX set to your region
genai.Client(vertexai=True, project="acme", location="us-central1",
             http_options=types.HttpOptions(
                 base_url="http://localhost:8787",
                 headers={"X-BitExact-Run-Id": run_id}))

# Azure OpenAI, with BITEXACT_UPSTREAM_AZURE=https://<resource>.openai.azure.com
AzureOpenAI(azure_endpoint="http://localhost:8787", api_key=key,
            api_version="2024-10-21",
            default_headers={"X-BitExact-Run-Id": run_id})

TypeScript and Go clients take the same base URL and header.

Amazon Bedrock#

Bedrock goes through the proxy with a Bedrock API key, which the client sends as a bearer token. With boto3, set AWS_BEARER_TOKEN_BEDROCK and point the client at the proxy:

client = boto3.client("bedrock-runtime", region_name="us-east-1",
                      endpoint_url="http://localhost:8787")
client.meta.events.register(
    "before-send.bedrock-runtime.*",
    lambda request, **_: request.headers.__setitem__(
        "X-BitExact-Run-Id", run_id))

Converse, ConverseStream, InvokeModel and InvokeModelWithResponseStream are all recorded; streamed responses (AWS event-stream frames) are relayed as they arrive and kept whole in the run. Model ids and ARNs travel percent-encoded exactly as the client sent them.

A client that signs with IAM credentials (SigV4) cannot go through any proxy: the signature covers the host it was made for, and Bedrock refuses it from another. Use a Bedrock API key for the calls you record, or add bitexact.otel_ingest.BitExactSpanProcessor to the TracerProvider of an agent whose AWS SDK emits OpenTelemetry GenAI spans: each model call is then recorded as asserted (reported by the host) rather than observed.

What is recorded#

Each call is one http_call entry: the method, the path as sent, the query with every credential parameter's value replaced, the wire-behavior headers (anthropic-version, anthropic-beta, openai-beta), the body, and the response. Authorization, api-key, x-goog-api-key and every other header are forwarded and never recorded. The model is taken from the body, or from the path where the wire puts it there (Gemini, Vertex, Azure deployments, Bedrock), and indexed for search.