Offline cassettes and recording

Ordinary sakura test denies network access. Http.recording and Llm.replay let tests exercise provider-shaped exchanges offline using JSON Lines cassettes under tests/fixtures/.

HTTP replayCopy Anchor Link

Test.fixtures() returns a Dir rooted at the package's tests/fixtures directory. It is read-only during ordinary tests. Install it with Fs.rooted, then put Http.recording inside an outer HTTP handler. The first handler in a with list is outermost.

This test belongs in the repository's compiler/examples/secrets package, alongside its committed billing.jsonl fixture:

import std/decimal.{Decimal}
import std/fs.{Fs}
import std/http.{Http}
import std/path.{Path}
import std/request.{Request}
import std/url.{Url}

test "billing cassette replays offline" {
  with Fs.rooted(Test.fixtures()), Http.scripted([]),
      Http.recording(Path.of("billing.jsonl")) {
    let amount = Decimal.parse("12.34").unwrap_or(Decimal.zero())
    let req = Request.post_json(Url.from("https://billing/charge"), amount)
    expect_eq(Http.send(req).status, 200)
  }
}

A request matches on method, URL, ordinary headers and body. Request.secret_headers are neither recorded nor matched. Keep credentials in that secret channel; ordinary headers and bodies are cassette data. Identical requests consume matching responses in order; the last response repeats. Without recording mode, a miss fails with HttpError and a message pointing to sakura test --record.

Cassette format and redactionCopy Anchor Link

A cassette is a JSON Lines file (design §17.122). Three rules apply:

1. Header line. The first line is {"schema_version":"1.0","kind":"cassette"}; each later line is one exchange, {method, url, headers, body, status, response_headers, response_body}. A missing or empty file has no exchanges and gets the header on its first append. A file whose first non-empty line has no schema_version, or a major version other than 1, fails to open with HttpError naming the file and sakura test --record.
2. Credential headers are refused. A request whose ordinary headers contain authorization, proxy-authorization, x-api-key, api-key, x-auth-token or cookie (in any case) fails under Http.recording with HttpError reading header <name> belongs in secret_headers (Http.bearer); not recorded. This happens before any replay or forward, recording or not, so a plain-string token is never written. Use Http.bearer or another secret_headers entry instead.
3. set-cookie is redacted. A set-cookie response header is stored as ["set-cookie", "<redacted>"] and replays as that literal. Other response headers are stored as the server sent them.

The URL and the request body are stored verbatim: a replay must match them. A Secret cannot reach them without an audited Secret.expose (V2031 in url"…", V2032 in Json.encode), but a credential that was never a Secret and is put in a query string or body is outside this guarantee.

LLM replayCopy Anchor Link

Llm.replay(path) installs the Anthropic adapter over Http.recording, an empty scripted outer HTTP handler and Budget.unlimited(), using a placeholder secret. It needs Fs, not live provider credentials. It replays generation, free text, partial callbacks and tool turns; missing exchanges fail as ProviderError.

The LLM extraction example uses Fs.rooted(Test.fixtures()), an outer Budget.unlimited() for its generated @llm function, and Llm.replay(Path.of("extract_ticket.jsonl")). The agent-loop example also scripts its knowledge-base tool.

Llm.replay and Llm.recording keep one cassette open per handler installation. Repeated identical requests consume matching responses in recorded order, then repeat the last response.

From the repository root, after building the CLI:

cd compiler/examples/llm_extract
../../target/debug/sakura check
../../target/debug/sakura test

Recording is opt-inCopy Anchor Link

--record is present in sakura test --help. It sets recording mode, makes the fixture directory writable and grants only the manifest's network hosts. With no net entries it refuses to start (exit 75). Run from the prepared project directory:

sakura test --record

A cassette miss can then forward through the outer Http handler and append the response. Existing hits still replay.

For provider recording, install Llm.recording(path, Llm.anthropic(key, model)) inside an outer Http and Fs handler. It replays cassette hits and records misses through the supplied provider under --record; a miss without recording mode fails as ProviderError. Use Http.host() and a real secret key for live calls. Llm.replay keeps its empty scripted outer handler even with --record, so it cannot fetch a missing provider exchange itself. Review cassette diffs before committing them. The examples below were executed with synthetic responses, without a live provider or real credentials.

A synthetic record-then-replay exampleCopy Anchor Link

In a scratch package, set [permissions] in sakura.toml to include net = ["api.anthropic.com:443"], then put this in src/main.sak. The outer HTTP handler supplies an invented response; it makes no network request:

import std/fs.{Fs}
import std/http.{Http, Matcher}
import std/llm.{Llm, Model, Request}
import std/path.{Path}
import std/prompt.{Prompt}
import std/response.{Response}

test "record a synthetic model response and replay it offline" {
  let response = Response.status(200,
    "\{\"content\":[\{\"type\":\"text\",\"text\":\"hello\"\}],\"stop_reason\":\"end_turn\",\"usage\":\{\"input_tokens\":1,\"output_tokens\":1\}\}")
  with Http.scripted([(Matcher.Any, response)]), Fs.rooted(Test.fixtures()),
      Llm.recording(Path.of("hello.jsonl"),
        Llm.anthropic(Secret.of("synthetic-key"), Model.of("demo-model"))) {
    let req: Request<String> = Llm.request("Be brief.", Prompt.text("Say hello."))
    expect_eq(Untrusted.check(Llm.generate_text(req), "greeting", fn(s) { s == "hello" }), Ok("hello"))
  }
}

Run sakura test --record once to create tests/fixtures/hello.jsonl, then sakura test to replay it offline. Replacing the outer handler with Http.scripted([]) still passes after recording, demonstrating that replay does not need the synthetic response. The placeholder key is not stored in the cassette.

Synthetic cassette conventionCopy Anchor Link

The committed fixtures in llm_extract, agent_loop and secrets are synthetic, not captured live responses. Each fixture directory has a README saying that bodies, IDs and token counts were invented, naming the exchanges and describing regeneration. Their request keys were generated by executing the program with a scripted outer HTTP response under recording mode, so they match the real adapter's serialization. A changed request fails replay and requires deliberate regeneration and review.

All twelve examples now execute in run or test mode; these three replay committed fixtures. This is evidence for the example gate, not evidence that all M3 exit gates have passed.

Streaming responsesCopy Anchor Link

Http.recording also implements Http.stream (design §17.88): a replayed exchange delivers its recorded body to the callback. Under --record the handler reads the whole streamed body from the outer Http before replaying it to the callback, so recording does not stream through. It records at most 16 MiB (16,777,216 bytes) of a response body, streamed or sent: past that, Http.stream or Http.send fails with HttpError reading response body over 16 MiB: sakura test --record stores at most 16,777,216 bytes of a response body, and nothing is appended to the cassette.

Http.read_chunk returns at most the requested number of bytes and an empty buffer at EOF. Consume the body inside the callback; it is closed when that scope exits:

import std/http.{Http, Matcher}
import std/response.{Response}
import std/request.{Method, Request}
import std/url.{Url}

test "stream reads bounded chunks" {
  with Http.scripted([(Matcher.Any, Response.status(200, "hello"))]) {
    Http.stream(Request.of(Method.Get, Url.from("https://example.com/")), fn(i) {
      expect_eq(Http.read_chunk(i.body, 3), String.to_bytes("hel"))
      expect_eq(Http.read_chunk(i.body, 3), String.to_bytes("lo"))
      expect_eq(Bytes.len(Http.read_chunk(i.body, 3)), 0)
    })
  }
}