[Marketen]

Why

Metering held every request's prompt as the model's full context window: about €2 of input hold per request on a 1M-context model, even for a short chat. With this PR the gateway tells Metering an upper bound of the real prompt. Metering side and full rules: dappnode/dappnode-nexus#151 (docs/METERING_HOLDS.md).

The estimate, exactly

apps/gateway/internal/application/services/input_estimate.go, computed on the body after PII masking (what goes upstream):

estimate = body bytes
         + 1,024                                  // provider additions: tool instructions, default system prompt, upstream model name
         + Σ over image parts (16,384 − part bytes) // an image counts 16,384 tokens instead of its bytes

• Text parts (text, input_text, output_text, refusal) and string/null content count as bytes.
• Image parts: image_url, image, input_image. Inline base64 counts as one image, not megabytes.
• No estimate: audio, video, file, any unknown or missing part type, a part that isn't an object, invalid JSON, or /v1/systemone (its provider builds its own prompt). Metering then holds the full context window, as today.

Why it's an upper bound: byte-level BPE and SentencePiece-with-byte-fallback tokenizers never produce more tokens than UTF-8 bytes, and the JSON body contains every string at least once, plus syntax. Measured on adversarial text with cl100k, o200k and the Kimi K2 tokenizer, the worst case was 0.95 tokens/byte (random Unicode). English is about 0.22, code 0.44, CJK 0.13–0.33.

A low estimate can't cost Nexus money: Metering charges real usage, and anything uncovered becomes account debt (see #151).

Changes

• domain.GenerateRequest.EstimatedInputTokens, set in Proxy before Reserve.
• The metering client sends estimated_input_tokens (omitted when unknown).
• No change to what is sent to providers.

Testing

• Table tests for every part type, including inline vs linked images, multiple images, tool messages, and each no-bound case.
• A 2 MB inline image holds about one image allowance, not 2M tokens.
• Proxy passes the estimate to Reserve, and passes none for audio.
• The metering client puts estimated_input_tokens and max_output_tokens on the wire, and omits the estimate when nil.
• go test ./... passes; GOOS=linux go build -tags nitro_enclave ./... builds.

Deploy

After dappnode/dappnode-nexus#151 (its migration must run first), though either order is API-safe. Old metering ignores the field. TEE enclave builds pick this up on their next release.

🤖 Generated with Claude Code