docs(specs): the token request reveals its headers, and the verifier checks them by xgreenx · Pull Request #31 · libid-org/libID

xgreenx · 2026-09-09T13:11:11Z

Based on main, deliberately — see the note at the end.

The problem

The token request's headers were hidden, and for GitHub that was normative rather than ambiguous:

REQ-PLAT-43D: The GitHub Token Service MUST reveal no range outside the seven rows marked yes above.

…the transcript holds the authority only in a Host header this table hides…

That leaves a region of the request the Platform Verifier cannot read, in the one direction it has to reason about. It reads grant_type, client_id and code_verifier out of the body with a form-encoding reading — while common REQ-COMMON-21B fixes the media type precisely because it "selects the platform's request parser".

A media type the verifier cannot see is a value the profile pins and nothing checks. The platform may have parsed those bytes into fields other than the ones read, and every revealed byte would still look correct.

The change

The request line and every request header are revealed, as one range with the body (REQ-PLAT-56A, both profiles). The verifier does not compare the head against a list. It holds a required subset — exactly one host naming the pinned authority and exactly one content-type naming the pinned media type, names normalized (lowercase, spaces and tabs removed, _ read as -) and values compared once the optional whitespace around them is removed — and refuses a forbidden set of names under any spelling: authorization, cookie, content-encoding, transfer-encoding and the method-override names. Every other header is ignored: it changes only what the platform answers, and a wrong answer is a response the verifier cannot read rather than one it can be fooled by. content-length is the one exception — REQ-PLAT-56B holds it to the body it frames (one header, decimal, no leading zero, one empty line ending the head) — and REQ-PLAT-56C fixes the line grammar (no bare LF or CR, no folds, no line without a colon). The identity requests get the same forbidden names and line grammar (REQ-COMMON-39A, 39B) and keep the one-authorization-line count under any scheme (REQ-COMMON-39).

The first draft compared a fixed list in a fixed order. A second look found no security difference between that and required-plus-forbidden, and GitHub answers 403 to a /user request without a user-agent, which the fixed list omitted; §6.5 now names it.

JSON whitespace (REQ-COMMON-19F, TEST-COMMON-10A)

GitHub pretty-prints /user for the pinned media type. The Platform Verifier removes each maximal run of JSON whitespace bounded by a structural byte, reads a member inside one revealed range, and counts the delimiter over the concatenation of the direction's revealed ranges (what libid-org/libID-contracts#37 does); the Implementation reveals a member as the wire carried it, whitespace inside, and never commits it with a bearer; the compact spellings in this specification name the member after removal. REQ-PLAT-51 judges the GitHub terminator after that removal; REQ-PLAT-60 keeps the whitespace in the reveal. Carries the vectors of #32.

Why this discloses nothing

The request is composed byte for byte by the implementation and driven over a raw MPC-TLS session — not the browser's HTTP stack, so there is no cookie jar and no ambient credential attachment. What the head carries is what the implementation put there; the two names that matter are held to pinned values, and the names that would change what the platform does invisibly are refused. Concealment would not have protected a user in any case: the implementation is what would have put anything private there, so hiding it hides it from review, not from the party that inserted it.

The response direction is unchanged and still hides its status line and headers: those are the platform's own bytes and nothing reads them. That asymmetry is now stated rather than inferred.

Two things deliberately preserved

The Host argument. A revealed host header is still not the authority — it is prover-composed text, held to the pinned authority by REQ-PLAT-56A — so the authority still reaches the verifier as the authenticated TLS server identity. The prose keeps that reasoning.
client_secret: still committed, still ordered last by REQ-COMMON-22 so the revealed run stays contiguous; the content-length count spans the revealed prefix and that commitment.

Verification

lint_spec.py over the two files: 3 errors, 13 warnings, all pre-existing on main (E6 on REQ-PLAT-37/38/43); no E10 — every requirement this PR adds (REQ-COMMON-19F, 39A, 39B; REQ-PLAT-56A, 56B, 56C) is cited by a TEST entry. The verifier side is merged and released: libid-contracts v0.9.0 (headers as profile data) and v0.10.0 (required + forbidden, whitespace removal, real X and GitHub fixtures); the prover side is libid-rs v0.4.0.

Review round, 2026-09-11 (Wondertan)

All four points applied: REQ-COMMON-19F states maximal-run removal, reading inside one range and counting over the concatenation, with the three range negatives in TEST-COMMON-10A; TEST-COMMON-18, TEST-PLAT-09C and TEST-PLAT-14 carry the header vectors for 39A/39B and 56A/56B/56C; REQ-PLAT-56A exempts content-length; the X Host sentence and the GitHub rationale no longer describe a hidden header or a fixed list.

Base branch

Not stacked on #19 (docs/ceremony-terminology), even though it rewrites these exact table rows to rename Platform Verifier → LOPPV. That PR is a draft, last updated 2026-09-03, already CONFLICTING with main, and part of a sweeping rename that will have to redo its diff regardless. When #19 is refreshed, its rename sweep absorbs these rows like every other.

…checks them The token request's headers were hidden. §5.2 left the request direction's headers out of its table, and §6.4 said so outright -- REQ-PLAT-43D bounded the disclosure to "the seven rows marked `yes`", and the prose named the `Host` header as one "this table hides". That leaves a region of the request the Platform Verifier cannot read, in the one direction it has to reason about. It reads `grant_type`, `client_id` and `code_verifier` out of the body with a form-encoding reading -- and common REQ-COMMON-21B fixes the media type precisely because it "selects the platform's request parser". A media type the verifier cannot see is a value the profile pins and nothing checks, and the platform may have parsed those bytes into fields other than the ones read. So the request line and every request header are revealed, and REQ-PLAT-56A has the verifier compare them against a fixed list, byte for byte and in order. Both halves are needed: revealing without comparing would leave the bytes public AND unconstrained, which is neither private nor checked. Both profiles' requests are now enumerated the way §5.3 and §6.5 already enumerate the identity request's headers, which is the precedent this follows. Nothing here is user data. The request is composed byte for byte by the implementation and driven over a raw MPC-TLS session, not the browser's HTTP stack, so there is no cookie jar and no ambient credential: the four headers are profile constants. Revealing them discloses nothing and leaves the sent direction with no region a verifier cannot read -- and concealment would not have protected a user in any case, since the implementation is what would have put anything private there. The response direction is unchanged and still hides its status line and headers: those are the platform's own bytes and nothing reads them. That asymmetry is now stated rather than left to be inferred. The `Host` argument survives and had to. A revealed `Host` header is still not the authority -- it is prover-composed text -- so the authority still reaches the verifier as the authenticated TLS server identity, and the header is compared against the profile like every other. `client_secret` is untouched: still committed, still ordered last by REQ-COMMON-22 so the revealed run stays contiguous. The linter reports what it reported before this change: 3 errors, 9 warnings, all pre-existing. Assisted-by: Claude Opus 5 Signed-off-by: xgreenx <xgreenx9999@gmail.com>

… around it The header list in the previous commit was wrong, and wrong in the way that would have been found by a failing launch rather than by review: it enumerated four headers and required the verifier to reject any other set, but a request carrying a body carries a `Content-Length`. hyper emits one for every known-size body -- `set_length` calls `set_content_length` -- and libid sets it nowhere, so the GitHub Token Service's own exchange would have produced a head of five headers and been refused by its own profile. As written the rule rejected every genuine attestation. `Content-Length` cannot be a profile constant: its value is the body's byte count. So REQ-PLAT-56B pins the value against something the verifying side can derive rather than against a literal -- the signed transcript length of the sent direction less the head is the body, whether the body is revealed whole as in X's request or revealed up to a committed suffix as in GitHub's, because common REQ-COMMON-35 makes the direction tile exactly. That is not bookkeeping. The verifier takes the body to be everything after the sole CRLFCRLF; the platform takes it to be `Content-Length` bytes. Where those disagree the fields the verifier reads are not the fields the platform parsed, which is the same divergence REQ-PLAT-56's `grant_type` check exists to stop, reached by a different route. `Transfer-Encoding` overrides `Content-Length` outright and so is refused rather than described. REQ-PLAT-56C carries the line-ending discipline over from the identity request. Common REQ-COMMON-39 already refuses a bare line feed and an obsolete line fold there, because a parser that accepts either ends the head somewhere the verifier does not -- and the token head, which no requirement had ever covered, is the same head with the same parsers reading it. REQ-PLAT-56A now also says the match is exhaustive rather than a presence test. "These five appear" is satisfied by a request carrying a sixth. Found by an audit of the previous commit, which was written from a reading of the request that no HTTP client produces. Assisted-by: Claude Opus 5 Signed-off-by: xgreenx <xgreenx9999@gmail.com>

Order carries no meaning here. Field order is insignificant in HTTP except for repeated names, which REQ-PLAT-56A forbids outright, and the `transfer-encoding`/`content-length` precedence REQ-PLAT-56B settles is by presence rather than position. Nothing a reordering does is not already refused by requiring the exact set. What fixing it would cost is real. It binds every prover to the header order its HTTP library happens to emit: the browser reaches the wire through tlsn's wasm prover, whose `HttpRequest` carries headers in a `HashMap`, so the order is whatever that iteration gives on the day. Pinning it also pins `content-length` last, which is not a promise anyone made -- it is where hyper's `set_length` inserts it, and a patch release could move it. So the set is fixed and the order is not, and the prose says so once rather than three times. Assisted-by: Claude Opus 5 Signed-off-by: xgreenx <xgreenx9999@gmail.com>

Sections 5.3 and 6.5 said "in this order" of headers no verifier orders: the identity request is held to coverage, one line-anchored authorization and the bearer framing, and common section 6 already says header order carries no proof semantics unless a profile commits it. The token request's headers were freed of order in this branch; these two now say the same. Assisted-by: Claude Opus 5 Signed-off-by: xgreenx <xgreenx9999@gmail.com>

…rows The tables listed each body field as its own revealed range. The verifiers, the prover and the attested record carry the request as one range -- the record cannot hold adjacent ranges apart, it merges them before signing -- so a browser planning one range per field would see its count change under it. Say so, once per platform. Assisted-by: Claude Opus 5 Signed-off-by: xgreenx <xgreenx9999@gmail.com>

api.github.com refuses any request without one: 403, with a body that says so, where the same request with one gets the 401 an absent bearer earns. Section 6.5 listed five headers and left it out, so a prover built from the text alone could never reach the identity read. The browser draft already sends one. X's endpoint and both token endpoints do not care, checked the same way, so the token request's pinned set is unchanged. The value is the runtime's to choose: nothing verifies it, and GitHub requires only that it exist. Assisted-by: Claude Opus 5 Signed-off-by: xgreenx <xgreenx9999@gmail.com>

Both identity requests said "exactly N headers". No verifier holds them to that: the contract compares the request line and the authorization line, and the prover's layout finds the same line and nothing else. A count nothing checks is a rule nobody can rely on and a prover can only fail to meet, as GitHub's user-agent just showed. Say which lines are compared and leave the rest to the runtime. The token request is unchanged: there the verifier does hold the set. Assisted-by: Claude Opus 5 Signed-off-by: xgreenx <xgreenx9999@gmail.com>

REQ-PLAT-56A had the verifier reject any header outside the profile's list. A header outside it changes only what the platform answers, and a wrong answer is a response the verifier cannot read, not one it can be fooled by; the rule bound every prover to one HTTP library's habits for nothing. It now requires `host` and the media type, forbids the five names that change what the platform does with the request in a way no revealed byte shows, and ignores the rest. `transfer-encoding` moves from 56B to that list, beside `content-encoding`. Assisted-by: Claude Opus 5 Signed-off-by: xgreenx <xgreenx9999@gmail.com>

The linter says so, and it is right: two MUSTs in one sentence read as one rule with a clause. Assisted-by: Claude Opus 5 Signed-off-by: xgreenx <xgreenx9999@gmail.com>

REQ-PLAT-56A said values are compared exactly; the verifier removes the optional whitespace around them first, and lowercases names with their whitespace removed, which is the normalization REQ-COMMON-39 already gives the identity request. A verifier built from the text alone would have refused `content-type:application/...`, which the contract accepts. 56C now also names a line with no colon, which the contract refuses. Assisted-by: Claude Opus 5 Signed-off-by: xgreenx <xgreenx9999@gmail.com>

…uests REQ-COMMON-39 counted `authorization:bearer` lines, so a second header under Basic or a platform's token scheme was never counted and the Identity Platform answered for whichever credential it honoured; the committed bearer is the one thing the cross-bind fixes. The needle is now `authorization:` under any scheme. REQ-COMMON-39A states the line-ending rule the verifier already applies to the identity request -- no bare line feed, no fold -- which nothing in this file said, and adds the bare carriage return. REQ-COMMON-39B forbids on the identity request the names that change what the platform does with it, `cookie` above all, with `_` read as `-`; the token request's REQ-PLAT-56A now refers to that one list plus `authorization`, and 56C names the bare carriage return and points at 39A. Assisted-by: Claude Opus 5 Signed-off-by: xgreenx <xgreenx9999@gmail.com>

The verifier requires exactly one empty line in the revealed token request and a declared length with no leading zero; REQ-PLAT-56B said neither. A second empty line is a second place a parser could end the head, and a second spelling of the count is a second thing to compare one spelling of. Assisted-by: Claude Opus 5 Signed-off-by: xgreenx <xgreenx9999@gmail.com>

GitHub pretty-prints `/user` for the media type the profile pins, so the compact delimiters this specification spells matched nothing it serves. REQ-COMMON-19F fixes what libid-org/libID-contracts#37 does: the Platform Verifier removes the JSON whitespace touching a structural byte and matches, counts and reads over what is left; the Implementation reveals a member as the wire carried it, whitespace inside, and never commits that whitespace with a bearer; the compact spellings name the member after removal. TEST-COMMON-10A lists the vectors. REQ-PLAT-51 judges GitHub's terminator after that removal and REQ-PLAT-60 keeps the whitespace in the reveal. Carries the vectors of #32 in the form the merged verifier implements. Co-authored-by: Wondertan <hlibwondertan@gmail.com> Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Signed-off-by: xgreenx <xgreenx9999@gmail.com>

Wondertan

I compared this revision with the ceremony implementation and current verifier. The main implementation findings are covered. These details remain:

Make the JSON extraction boundary explicit in REQ-COMMON-19F.

The verifier deliberately distinguishes reading from counting: it extracts a complete field from one contiguous revealed range, but counts duplicate delimiters across all revealed ranges, including delimiters split across ranges.

The updated requirement specifies normalization without retaining that distinction explicitly. Please state that normalization applies within a range for extraction, and that concatenation used for duplicate detection cannot construct a field value. This matches the existing verifier.

Restore the corresponding TEST-COMMON-10A negatives: disjoint fragments assembled into a field, a duplicate delimiter split across reveals, and hidden bearer-prefix whitespace. The existing HTTP-chunk-boundary case does not cover these attestation-range cases. Relevant requirements.
Describe whitespace removal in terms of complete runs.

“Every whitespace byte that touches a structural byte” is ambiguous for several consecutive whitespace bytes. The verifier removes an entire contiguous run when either adjacent byte is structural.

Suggested wording: “Remove each maximal run of JSON whitespace when the byte immediately before or after that run is structural.” This agrees with the mixed-whitespace vectors already listed.
Add conformance coverage for the new header requirements.

REQ-COMMON-39A/39B and REQ-PLAT-56A/56B/56C have no linked tests. Whole-suite lint introduces five corresponding E10 findings.

Extend the existing test entries, retaining their IDs, to cover:
- Additional permitted headers and required/forbidden header normalization.
- A second Authorization header using a different scheme.
- Bare CR/LF, folded headers and malformed header lines.
- Missing, duplicate, padded or mismatched Content-Length.
- Request/body boundaries, including GitHub’s committed secret suffix.
Resolve the remaining header prose contradictions.
- REQ-PLAT-56A says to ignore every other header, but 56B requires checking Content-Length. Explicitly exempt that check.
- The X token explanation still says its Host header is hidden, contradicting the updated disclosure table.
- The PR description and GitHub rationale still describe a fixed header list, although the final requirements permit additional headers. Relevant section.

Assisted-by: GPT-5 Signed-off-by: Wondertan <hlibwondertan@gmail.com>

… every header rule Answers Wondertan's review of 2026-09-11. REQ-COMMON-19F now says what the verifier does at a range boundary: a member is read inside one revealed range, the delimiter is counted over the concatenation of the direction's revealed ranges after the same removal, and no value is ever read from that concatenation. Removal is stated per maximal run of JSON whitespace bounded by a structural byte, which is what `normalizeJsonBytes` removes; "every byte that touches" read as the ends of a run. TEST-COMMON-10A gains the three range negatives: a field assembled from two ranges, a delimiter a boundary cuts in two, and prefix whitespace pushed into the committed range. REQ-COMMON-39A/39B and REQ-PLAT-56A/56B/56C had no test entry, five E10 findings: TEST-COMMON-18 carries the identity-request vectors, TEST-PLAT-09C the token-head vectors, and TEST-PLAT-14 runs them on the exchange with the count held to the revealed prefix plus the committed secret. The lint's E10 count goes from five to none; the rest is main's. Prose that still described the first draft: REQ-PLAT-56A ignores every other header except the `content-length` REQ-PLAT-56B holds; the X section's `Host` is revealed and held to the authority, not hidden; the GitHub rationale no longer says "fixed list"; both identity-request sections say the verifier also refuses the names REQ-COMMON-39B forbids. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Signed-off-by: xgreenx <xgreenx9999@gmail.com>

xgreenx · 2026-09-14T09:18:36Z

@Wondertan — all four applied in 5afdf08:

Boundary. REQ-COMMON-19F now says the verifier reads a member inside one revealed range, counts the delimiter over the concatenation of the direction's revealed ranges after the same removal, and never reads a value from the concatenation. TEST-COMMON-10A gains the three negatives: a field assembled from two ranges, a delimiter a range boundary cuts in two, and prefix whitespace pushed into the committed range.
Runs. Removal is stated per maximal run of JSON whitespace whose immediately preceding or following byte is structural — your wording.
Coverage. TEST-COMMON-18 cites 39A/39B with the identity-request vectors (second authorization under another scheme, an unlisted header passes, forbidden names in any spelling, bare LF/CR and folds). TEST-PLAT-09C cites 56A/56B/56C with the token-head vectors (order, unlisted headers, required-name spellings, missing or duplicate required names, forbidden names, empty lines, content-length missing / duplicate / mismatched / non-decimal / leading zero / anywhere in the head, line grammar). TEST-PLAT-14 runs them on the exchange and holds the count to the revealed prefix plus the committed client_secret. Lint: the five E10 are gone; the remaining 3 errors and 13 warnings are main's.
Prose. 56A exempts content-length for 56B; the X sentence says the host header is revealed and held to the pinned authority rather than hidden; the GitHub rationale no longer says "fixed list"; both identity-request sections say the verifier also refuses the 39B names. The PR description is rewritten to match.

Incorporate PR #31 commit 5afdf08: maximal JSON whitespace runs, single-range extraction with cross-range delimiter counting, header conformance vectors, and matching rationale. Preserve the reconciled browser validation, request-selected notary, and origin admission contracts. Correct the stale code_verifier circuit attribution and GitHub token-layout reference. Assisted-by: GPT-5 Signed-off-by: Wondertan <hlibwondertan@gmail.com>

Exclude the separately checked request line and terminating empty line, and cover a valid colon-free request line in conformance. Assisted-by: GPT-5 Signed-off-by: Wondertan <hlibwondertan@gmail.com>

cloudflare-workers-and-pages · 2026-09-18T16:48:28Z

Deploying with Cloudflare Workers

The latest updates on your project. Learn more about integrating Git with Workers.

Status	Name	Latest Commit	Updated (UTC)
❌ Deployment failed View logs	libid	`fd9b9c5`	Sep 21 2026, 11:05 PM

xgreenx added 4 commits September 9, 2026 14:10

xgreenx marked this pull request as ready for review September 9, 2026 17:20

xgreenx mentioned this pull request Sep 9, 2026

feat: the ceremony wire constructions — digest, PKCE, attested data, reveal layouts libid-org/libID-rs#2

Merged

Wondertan mentioned this pull request Sep 10, 2026

feat(ceremony): implement browser identity ceremonies #28

Open

xgreenx added 2 commits September 10, 2026 10:04

xgreenx mentioned this pull request Sep 10, 2026

feat(ceremony): hold the token request's head to required and forbidden headers, not a fixed set libid-org/libID-contracts#35

Merged

xgreenx added 5 commits September 10, 2026 14:01

docs(specs): one keyword per sentence in REQ-PLAT-56A

770b2fa

The linter says so, and it is right: two MUSTs in one sentence read as one rule with a clause. Assisted-by: Claude Opus 5 Signed-off-by: xgreenx <xgreenx9999@gmail.com>

Wondertan mentioned this pull request Sep 10, 2026

specs: accept JSON whitespace in token and identity responses #32

Closed

Wondertan reviewed Sep 11, 2026

View reviewed changes

Wondertan added a commit that referenced this pull request Sep 12, 2026

Merge token framing specification prerequisite (#31)

a85d428

Assisted-by: GPT-5 Signed-off-by: Wondertan <hlibwondertan@gmail.com>

Wondertan mentioned this pull request Sep 12, 2026

docs(specs): define CCDP and ceremony service contracts #13

Open

Wondertan added a commit that referenced this pull request Sep 13, 2026

Merge token framing specification prerequisite (#31)

44e9574

Assisted-by: GPT-5 Signed-off-by: Wondertan <hlibwondertan@gmail.com>

xgreenx requested a review from Wondertan September 14, 2026 09:18

xgreenx self-assigned this Sep 14, 2026

Wondertan mentioned this pull request Sep 14, 2026

docs(specs): move GitHub token exchange into the browser #35

Open

docs(specs): limit colon checks to header field lines

607322a

Exclude the separately checked request line and terminating empty line, and cover a valid colon-free request line in conformance. Assisted-by: GPT-5 Signed-off-by: Wondertan <hlibwondertan@gmail.com>

Wondertan approved these changes Sep 18, 2026

View reviewed changes

Merge remote-tracking branch 'origin/main' into HEAD

fd9b9c5

xgreenx merged commit 002c201 into main Sep 21, 2026
4 of 6 checks passed

Navigation Menu

docs(specs): the token request reveals its headers, and the verifier checks them - #31

docs(specs): the token request reveals its headers, and the verifier checks them#31
xgreenx merged 16 commits into
mainfrom
specs/token-request-reveals-its-headers

xgreenx commented Sep 9, 2026 •

edited

Loading

Uh oh!

Wondertan left a comment •

edited

Loading

Uh oh!

xgreenx commented Sep 14, 2026

Uh oh!

cloudflare-workers-and-pages Bot commented Sep 18, 2026 •

edited

Loading

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

xgreenx commented Sep 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

The problem

The change

JSON whitespace (REQ-COMMON-19F, TEST-COMMON-10A)

Why this discloses nothing

Two things deliberately preserved

Verification

Review round, 2026-09-11 (Wondertan)

Base branch

Uh oh!

Wondertan left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

xgreenx commented Sep 14, 2026

Uh oh!

cloudflare-workers-and-pages Bot commented Sep 18, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Deploying with Cloudflare Workers

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

xgreenx commented Sep 9, 2026 •

edited

Loading

Wondertan left a comment •

edited

Loading

cloudflare-workers-and-pages Bot commented Sep 18, 2026 •

edited

Loading