Batch processing

Upload a JSONL file of requests, create a batch, walk away. Saxeo runs the lines asynchronously within a 24-hour window and hands you back a results file. Because each line runs through the same handlers a live call uses, every line is metered, capped, and gets its own signed receipt — and any batch discount the deployment sets is credited straight back to your balance.

Batch work is the natural fit for evaluation runs, backfills, bulk classification, and embedding a corpus: work with no user waiting on it.

The shape

# 1. Upload the requests
curl -X POST https://www.saxeonetwork.tech/__api/v1/files \
  -H "authorization: Bearer $SAXEO_KEY" \
  -F purpose=batch \
  -F file=@requests.jsonl

# 2. Create the batch
curl -X POST https://www.saxeonetwork.tech/__api/v1/batches \
  -H "authorization: Bearer $SAXEO_KEY" \
  -H 'content-type: application/json' \
  -d '{"input_file_id":"file-...","endpoint":"/v1/chat/completions","completion_window":"24h"}'

# 3. Poll, then download
curl https://www.saxeonetwork.tech/__api/v1/batches/batch_... \
  -H "authorization: Bearer $SAXEO_KEY"

curl https://www.saxeonetwork.tech/__api/v1/files/file-.../content \
  -H "authorization: Bearer $SAXEO_KEY"

The OpenAI SDKs work unchanged:

f = client.files.create(file=open("requests.jsonl", "rb"), purpose="batch")
b = client.batches.create(
    input_file_id=f.id,
    endpoint="/v1/chat/completions",
    completion_window="24h",
)

The input file

One JSON object per line:

{"custom_id":"row-1","method":"POST","url":"/v1/chat/completions","body":{"model":"saxeo","messages":[{"role":"user","content":"Classify: great product"}]}}
{"custom_id":"row-2","method":"POST","url":"/v1/chat/completions","body":{"model":"saxeo","messages":[{"role":"user","content":"Classify: broke on day two"}]}}

• custom_id — yours, unique within the file, ≤256 chars. It is how you match results back to inputs.
• method — "POST".
• url — /v1/chat/completions, /v1/embeddings, or /v1/responses. Every line must match the batch's endpoint.
• body — exactly the body you would post to that endpoint, Saxeo fields included.

Limits: 32 MiB per file, 50,000 requests per file, 20 batches in flight per account. The file is validated line by line at upload, so a malformed file fails while you can still fix it rather than twenty minutes into a run. Errors name a line number and a reason, never the line's content.

What it costs

Each line is billed the ordinary metered price for the work it did. If the deployment sets a batch discount, it is credited back to your balance as a batch_discount row in the ledger, idempotent per usage event so a worker restart can never credit the same line twice.

The default rate is 0. Batching here buys you scheduling and durability, not a lower price: a batch line runs through the same path as a live call and costs the same to serve, because nothing about it is deferred. Vendors who discount their own batch APIs fund that from real deferral, and Saxeo does not yet route lines through those. Check discount_bps on the batch object for what your deployment actually applies, rather than assuming a rate.

Two things stay true, because they are what a receipt is worth:

• Every line still gets its own signed receipt. A batch is a scheduling convenience, not a lower tier of proof. The receipt is fetchable by its request_id through GET /v1/receipts/:id, and it also rides in the output file under response.sable_receipt.
• Every line is still bounded. Holds, spend caps and windows, the circuit breaker, policy-as-code, and model allowlists all apply per line, under the API key that created the batch. A batch cannot spend past what that key was allowed to spend.

The discount applies to inference and embedding compute. It is not a discount on a different, cheaper backend — it is the same models, the same routing, the same proof, run when it suits us.

Status, cancelling, expiry

status moves validating → in_progress → finalizing → completed, and request_counts tracks total / completed / failed as it goes.

• Cancel with POST /v1/batches/:id/cancel. A batch that has not started cancels immediately; a running one stops between chunks. Lines that already ran were executed and billed, so their results are still written to the output file rather than thrown away — you paid for them.
• Expiry. The completion window is 24 hours from creation. Anything left unrun at that point ends as expired, with whatever finished still collected.
• Resume. Progress is persisted per chunk, so a gateway restart mid-batch picks up at the next unfinished line and never re-runs (or re-bills) work that already completed.
• Failure. A batch that cannot start at all ends failed with a fixed class in errors — for example input_file_unavailable, endpoint_mismatch, or key_revoked.

Subscribe to the batch_completed and batch_failed webhooks instead of polling. Their payloads carry ids, status, and counts only — batch content lives in the sealed output file, never in a webhook body sent to a third-party URL.

The output file

One JSON object per line, in input order:

{
  "id": "batch_req_...",
  "custom_id": "row-1",
  "response": {
    "status_code": 200,
    "request_id": "...",
    "body": { "id": "chatcmpl-...", "choices": [ ... ] },
    "sable_receipt": { "receipt": "...", "signature": "0x...", "signer": "0x..." }
  },
  "error": null
}

Successes land in output_file_id; lines that returned an error land in error_file_id, so a partial failure is easy to retry without filtering.

Files

purpose accepts batch only. General-purpose object storage is not something Saxeo offers, and accepting arbitrary purposes would turn a bounded exception into an unbounded one. Output and error files are produced by the batch worker and cannot be uploaded.

Privacy: the deliberate exception

Everywhere else on Saxeo, prompts, completions, and submitted code are never persisted. Batching cannot work that way and be honest about it: a worker cannot run a file it does not hold, and you cannot collect results tomorrow that were not kept. So batch files are the disclosed exception, and they are built the way hosted agents and Relay are:

• Sealed at rest. Input, output, and error files are AES-GCM-encrypted with the gateway's master key. The database stores ciphertext; there is no column holding your plaintext.
• Opened in-frame only. A file is decrypted to run a line or to answer your own download, and never written to a log. Log lines carry ids, byte counts, and fixed error classes.
• Destroyed on a schedule you can predict. A batch's input file has its ciphertext destroyed the moment the batch reaches a terminal state — success, failure, cancel, or expiry. Every file, input and output alike, is purged at its TTL (7 days by default; the deployment sets it, and expires_at on the file object is authoritative). DELETE destroys it immediately. The row survives as your own history; the content does not.

Download your results before the TTL. After it, the metadata remains and the content is gone — including to us. That is the point.

Nothing else changes: individual lines are sealed on ingress and run through the ordinary path, so what reaches the model, and what the receipt records, is identical to a live call.