Repository navigation
RFC 0001: metadata resolution layer (FORGE-1) - #69
Swimburger wants to merge 21 commits into
Conversation
…k (FORGE-1) Add the metadata resolution layer of RFC 0001. - src/metadata: the native x-forge-* vocabulary, per kind of object, as pure key tables, primitives and value-format descriptors. - src/compat: the compatibility layer, behind the default-on `compat` feature. The Fern and Stainless adapters map their keys onto the same primitives; every vendor name lives here. - src/link/resolve.rs: native first, compat second. A native key's value is kept over a compat one's, whatever order they are written in. - Errors name the key the document wrote. - Docs and the WIT speak the native keys; the WIT copies are regenerated (doc comments only, so forge:plugin stays at 0.3.0). Output is unchanged: the unmodified Fern-keyed suite passes against it, and the real aggregate builds the same files and the same model dump.
x-fern-* and x-stainless-skip become their x-forge-* equivalents; a dotted group string becomes an array. What only the compat layer does is covered by tests/compat.
tests/boundary.rs fails, naming the file and line, when an x-fern or x-stainless name appears in src/, tests/ or packages/forge/ outside src/compat/ and tests/compat/. `just boundary` runs it.
- Each table checks its prefix (x-forge-, x-fern-, x-stainless-) before matching, and only x- keys are resolved, so a skipped key costs one compare instead of a match per table. - A Hit holds only its field and slot; a key's name comes from the slot, in the per-kind name tables, when an error needs it. The reader's per-key result is back to 16 bytes, which fits in registers. Output and the model dump of the aggregate are unchanged.
Builds REV in a throwaway worktree and runs the parse bench of both in ABBA order, so drift in the machine's speed cancels, leaving out pairs in which it changed speed. `just bench-compare [rev] [pairs]` runs it.
…tive - x-forge-method-name, in line with the other -name keys. x-forge-method never shipped, so it gets no alias. - x-forge-sensitive replaces x-sensitive, so every native key begins with x-forge-. x-sensitive stays readable as a deprecated native alias: a new Tier (native, deprecated, compat) decides which key's value is kept, and the alias works with compat off. - tests/boundary.rs also catches vendor names spelled as identifiers (x_fern_...), and the five test names that still had them are renamed. Output and the model dump of the aggregate are unchanged.
…ecision number - Rule 4 now says per key what an empty or punctuation-only value does: names fall back to the wire name; an empty method name or group path is no method, and a punctuation-only one is an error. - The npm release adds no compat module: the engine binary it ships is built with the default compat feature. - "the D4 rule" is resolution rule 2; the D-numbers were never defined.
Today x-forge-group: [] makes no method. The intended meaning is a method at the root, which needs root methods in the model, the WIT and the generators, so it is documented as a known gap rather than fixed here.
… checked - In an alias, the native method name, group, availability and ignore win over Fern's, in either order. - A Fern group path or method name that is an error on its own is not checked when a native key wins. Breaking the precedence rule fails all three.
- The boundary goal covers the engine; the docs packages read vendor keys themselves and are a non-goal. - resolve.rs is the only place link calls compat, not the only cfg point. - Deleting the layer lists every step, and what it leaves behind. - With compat off, vendor keys become ordinary extensions for plugins. - The name lookups are a second way vendor key names leave compat, as data, for errors. - Test counts, the mutation result, and performance and memory are re-measured on the final code.
c6fd809 to
e84e0e9
Compare
Swimburger
left a comment
There was a problem hiding this comment.
- I think we should rename x-forge-group to x-forge-group-name for now, keep it more consistent with Fern until we rename it to x-forge-namespace.
- Are we unnecessarily slowing down CI at all?
- Rename x-forge-ignore to x-forge-skip, internal names included. - Rename x-forge-group to x-forge-group-name, to match the other -name keys and stay apart from the legacy x-forge-group-info. - Remove benches/compare.sh and its just recipe; the RFC describes the method inline. - RFC: record why the key is model-name, and that CI compiles but does not test the build without compat, on purpose.
|
|
|
dimitropoulos
left a comment
There was a problem hiding this comment.
love it! one thing I'd love to keep up-to-date is the AGENTS.md file for this work. I'd love to keep it clear that the intention is that any compat flags are intended to be sectioned off into their own little area of the codebase
AGENTS.md gets a short section: only src/compat/ names vendor extensions, it sits behind the compat feature, native keys win, and tests/boundary.rs enforces it. src/compat/README.md holds the longer how-to, next to the code it describes and deleted along with it.
Each level (operation or alias, parameter, schema, document) now has its own module holding its key constants, its table of names and its reader, in the native vocabulary and in each adapter. A key constant is no longer in scope at a level it does not belong to. Tests check that every key is read at exactly the levels that list it. Also drop the key tables from the adapters' module docs: the code is the source of truth for the keys.
srikrsna
left a comment
There was a problem hiding this comment.
I would like the compatibility layer to be exposed as a separate command so that the spec we produce itself can be transformed. It will also reduce the complexity we have introduced to account for metadata resolution. That way the core will always parse just the forge extensions. And compat can truly be a separate layer altogether.
Drop the status header, the known gaps and follow-ups sections, the test plan and the release shape, and shorten the rest. The vocabulary, the resolution rules, the architecture and the alternatives considered stay.
Implements RFC 0001 for FORGE-1: a metadata resolution layer between the vendor extensions in a spec and the engine.
What changes
x-forge-*vocabulary (src/metadata/), listed in full below. Every native key begins withx-forge-;x-sensitivekeeps working as a deprecated name ofx-forge-sensitive.src/compat/, behind the default-oncompatfeature): the only code that knowsx-fern-*andx-stainless-skip. It maps them onto the native keys, and is designed to be deleted as a whole.x-forge-group-nametakes a dotted string ("accounts.members") or an array, read exactly as groups always have been. The dotted string is Cloudflare's convention rather than Fern's; FORGE-12 tracks moving specs off it and then making the Fern adapter match Fern.tests/boundary.rs) fails ifx-fernorx-stainlessappears insrc/,tests/orpackages/forge/outside the compatibility layer, doc comments included.The native vocabulary
All decided native names, with the vendor key the compatibility layer reads in its place:
x-forge-method-namex-fern-sdk-method-namex-forge-group-name"accounts.members") or array of strings, top level firstx-fern-sdk-group-namex-forge-skipx-fern-ignore(operation) ·x-stainless-skip(property; a list of targets counts asfalse)x-forge-availabilityalpha|beta|preview|generally-available|deprecatedx-fern-availabilityx-forge-model-namecomponents.schemas/$defsx-fern-type-namex-forge-property-namex-fern-property-namex-forge-parameter-namex-fern-parameter-namex-forge-sensitivex-sensitive)x-sensitivex-forge-sensitive, read only where that is not writtenExisting native keys, unchanged:
x-forge-hidden,x-forge-require-confirmation,x-forge-aliases(operation) andx-forge-commands(document root).Not read, and unchanged:
x-fern-enum,x-fern-audiences,x-stabilityand the other vendor keys without a Forge primitive (FORGE-5). Later renames (group→namespace, the availability word) are FORGE-3.Compatibility
Not breaking. Existing specs build the same
Forge, the WIT changes only in doc comments (forge:plugin@0.3.0is unchanged), and generated output is byte-identical. Built withoutcompat, the native keys behave the same, and vendor keys become ordinary extensions passed to plugins.Verification
Checked against the unmodified engine on the released aggregate (
openapi.forge.json, 27.6 MB), as detailed in the RFC's Verification section:forge --target tswrites the same 3,381 files, byte for byte, and the full model dump is identical;wit_is_currentholds;cargo testpasses with compat on and, run locally, with compat off (CI compiles but deliberately does not test the compat-off build; see the RFC); clippy is clean in the library-only, compat-off and all-features configurations;main(paired runs of theparsebench), with the same peak memory and allocation count.Reviewing
The RFC is the place to start, then
src/metadata/mod.rs,src/compat/andsrc/link/resolve.rs. Most of the diff undertests/parse/is fixtures moving to native keys.Open questions for reviewers, also in the RFC:
x-fern-audiencesunread, which differs from the ticket text;""and[]should mean the root, but the model has no root methods yet (Known gaps).Follow-ups: FORGE-3 (the breaking vocabulary renames), FORGE-4 (the versioning RFC), FORGE-5 (a Forge CI lint for vendor keys forge doesn't read) and FORGE-12.