Repository navigation
Fix OG Viewer failing to load pages over the 6 MiB fetch cap - #1918
Open
David Pine (IEvangelist) wants to merge 4 commits into
Open
David Pine (IEvangelist) wants to merge 4 commits into
David Pine (IEvangelist) wants to merge 4 commits into
Conversation
fetchUrl rejected any response larger than its 6 MiB safety cap, so https://aspire.dev/reference/api/csharp/ (6.4 MB of HTML) never loaded in the canvas, even though its title and og:image sit in the first ~44 KB. The agent-readiness probes had the same problem with their 256 KiB cap and silently reported oversized responses as missing. Add an opt-in onLimit: "truncate" mode to fetchUrl that keeps exactly the first maxBytes bytes, aborts the connection, and resolves with truncated: true. The default still rejects. Use it where a partial document is useful: metadata loading, the Browse tab's main document (non-HTML bodies are still rejected because a cut-off file is corrupt), and the agent-readiness probes (a cut-off size is shown as "256 KB+"). Subresources, /api/img and /api/raw keep their current behavior. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 65ef8eed-2fb4-4090-baf5-0c8664d156e6
The server's /api/fetch and /api/agent-readiness errors already begin with "Couldn't ...", and the UI prepended its own "Couldn't load metadata:" / "Couldn't run agent-readiness checks:" context, so the banner read "Couldn't load metadata: Couldn't load metadata. Check the URL and selected-origin access." Add a small failureMessage() helper that only prepends the context when the message doesn't already start with "Couldn't". Server strings are unchanged (tests assert on them and they are the fixed, no-leak messages); other errors, such as network failures, still get the prefix. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 65ef8eed-2fb4-4090-baf5-0c8664d156e6
Contributor
There was a problem hiding this comment.
🟢 Approval recommended
Truncation is narrowly scoped, strict resource handling is preserved, and focused regression coverage addresses the changed behavior.
No open issues 🎉
What changed in this PR
Fixes OG Viewer loading failures for oversized pages while preserving strict size limits for resources that cannot safely be truncated.
Changes:
- Adds opt-in truncation for metadata, Browse documents, and agent-readiness probes.
- Removes duplicated error-message prefixes.
- Documents size-limit behavior and adds regression coverage.
| File | Description |
|---|---|
.github/extensions/og-preview/ui/app.js |
Avoids duplicated error prefixes. |
.github/extensions/og-preview/tests/server.test.mjs |
Covers oversized documents and strict resource rejection. |
.github/extensions/og-preview/tests/http-fetch.test.mjs |
Covers truncation, byte boundaries, and connection closure. |
.github/extensions/og-preview/tests/agent-readiness.test.mjs |
Covers oversized probe responses. |
.github/extensions/og-preview/README.md |
Documents fetch limits and truncation. |
.github/extensions/og-preview/lib/http-fetch.mjs |
Adds opt-in truncation and a result flag. |
.github/extensions/og-preview/lib/agent-readiness.mjs |
Enables partial probes and marks truncated sizes. |
.github/extensions/og-preview/extension.mjs |
Enables document truncation while rejecting incomplete non-HTML resources. |
Review effort: Balanced
The Browse tab runs the live document, but the proxy cut it at the 6 MiB fetch cap. The C# API reference page is 6.4 MB and inlines a 3.7 MB search index ahead of its interactive content, so Browse rendered everything up to the cut and dropped the search UI, footer, and closing tags. Give the main browse document a 16 MiB cap, still truncating rather than rejecting. Metadata reads, sub-resources, and non-HTML files keep their existing limits. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: ed12c02d-40f7-44b1-b708-2f2517aa89e7
## Summary Adds a **Prompt** tab to the OG Viewer canvas and an agent-callable `get_prompt` action. Both turn a URL into one ready-to-use prompt for an AI assistant: a short context block, composable task modules (audit, fix head tags, rewrite copy, share image, structured data, agent readiness, implement, verify), the user's own notes, and a versioned JSON snapshot of the page's SEO and social metadata (URL and redirects, HTTP headers, title, description, canonical, robots, hreflang, JSON-LD, Open Graph, Twitter, diagnostics, agent readiness). **Stacked on #1918.** This branch is based on #1918's branch, so the diff shows only the composer commit. I'll retarget it to `main` and rebase once #1918 merges. How it works: - One pure module, `lib/build-prompt.mjs`, builds the payload and the prompt. The extension server and the canvas UI both use it, so the tab and the action can't drift. - The tab has **Copy prompt**, **Copy JSON**, **Open in Copilot**, and **Create issue** (the last two appear when the page links to its source repository). The fifth tab also gets a responsive toolbar fix. - Page content is untrusted. The JSON is fenced and followed by a reminder, the `Page:` line carries only the requested URL, and redirect targets, the canonical URL, and the detected repository appear only inside the JSON. - The payload is bounded. Strings, lists, and JSON-LD have fixed limits, and anything cut is named in `truncated`; JSON-LD that is left out is flagged `dataOmitted`. The limits are listed in the README. Known limit: a page that pushes every limit at once can still produce a prompt larger than the 512 KB request limit used by **Open in Copilot** and **Create issue**, and GitHub caps an issue body at 65,536 characters. Copying always works. ## Third-party links and affiliations None. The tests use reserved `.example` hostnames as hostile fixtures; no real third-party links are added. ## Validation - `node --check` on `extension.mjs`, `lib/parse-og.mjs`, and `lib/build-prompt.mjs`: all pass. - `node --test ".github/extensions/og-preview/tests/*.test.mjs"`: 72 pass, 0 fail. - Control: against the `lib` files from before the review fixes, 11 of the new and updated tests fail, so the tests catch the problems they target. - Live check in a real browser with the extension reloaded, on https://aspire.dev/reference/api/csharp/ (the page #1918 fixes): - Browse shows the whole page (6.45M characters, 4,996 package links, footer). - The Prompt tab shows all eight modules, the size readout (15,868 characters, about 3,967 tokens), the truncation notice, and a valid JSON snapshot with the agent-readiness checks joined in. - `get_prompt` returns the same shape; the `Page:` line shows only the requested URL and the detected repository appears only in the JSON. - UI, CSS, and the agent-readiness path aren't covered by CI; the live check above is the only coverage for them. The design linter still reports 3 findings in `styles.css` (`side-tab`, `bounce-easing`, `layout-transition`) that were there before this change. - CI: `OG preview tests` runs on this PR. `ci.yml` only runs for PRs into `main` or `release/*`, so it will run after the retarget. Co-authored-by: David Pine <7679720+IEvangelist@users.noreply.github.com> Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: ed12c02d-40f7-44b1-b708-2f2517aa89e7
David Pine (IEvangelist)
enabled auto-merge (squash)
October 28, 2026 14:21
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fixes the OG Viewer (OpenGraph Preview) canvas failing to load https://aspire.dev/reference/api/csharp/.
Root cause
fetchUrl()inlib/http-fetch.mjshas a hard 6 MiB safety cap (MAX_BYTES) and rejected withResponse exceeded the 6291456-byte limit.as soon as a response went over it. That page is HTTP 200text/htmlat 6,444,920 bytes (~150 KB over the cap, and growing with the API surface), even though its<title>and Open Graph tags are in the first ~44 KB. The rejection was swallowed into the generic "Couldn't load metadata..." banner, and the Browse tab showed "Couldn't load this page".The agent-readiness probes had the same problem at their 256 KiB cap:
probe()swallows every error into{ ok: false }, so a largellms-full.txt, a large Markdown response, or theAccept: text/htmlpage probe that feeds the Link-header check was silently reported as unavailable (a false negative).Fix (commit 1)
fetchUrlgets an opt-inonLimit: "error" | "truncate"option. The default is"error", so existing behavior is unchanged. In"truncate"mode it keeps exactly the firstmaxBytesbytes, closes the connection, and resolves withtruncated: true(falseotherwise).loadMetadata: it needs only the document head. The result gains an additivetruncatedfield./api/proxymain HTML document: browsers render partial HTML. A truncated non-HTML main document (for example a large image or PDF opened in Browse) is still rejected, so corrupt bytes are never served. Commit 3 raises this document's cap.probe(): truncated sizes are shown as256 KB+rather than claiming the full size./api/proxy/...subresources (a truncated script or stylesheet would be corrupt),/api/img, and/api/raw.MAX_BYTESis not raised. The README notes the cap and its behavior.Doubled error prefix (commit 2)
The failing canvas showed "Couldn't load metadata: Couldn't load metadata. Check the URL and selected-origin access." because
ui/app.jsprepended its own context to a server message that already starts with "Couldn't". A smallfailureMessage()helper skips the prefix in that case, and is applied to the agent-readiness banner as well. Server strings are unchanged:tests/server.test.mjsasserts on them and they are the fixed, no-leak messages. Other errors (for example network failures) still get the prefix.Browse tab cap (commit 3)
With commit 1 alone, Browse rendered the page, but a cut-off one: the main document was still capped at 6 MiB, and on this page the interactive content sits just past that mark. The search box starts at byte 6,304,083 (~12.6 KB past 6 MiB), the footer at 6,410,468, and
</html>at 6,444,813, so Browse silently dropped the intro, search UI, footer, and closing tag./api/proxymain-document fetch now uses its ownBROWSE_DOC_MAX_BYTEScap of 16 MiB. Past that it still truncates rather than rejects, so a very large page degrades instead of failing.MAX_BYTES(6 MiB), metadata reads,/api/proxy/...subresources,/api/img,/api/raw(1 MiB), and the 256 KiB readiness probes.Third-party links and affiliations
None.
Validation
node --test ".github/extensions/og-preview/tests/*.test.mjs"-> 34 tests, 34 pass, 0 fail.node --test .github\extensions\og-preview\tests\*.test.mjsgives the same result (Node v24.13.1; CI runs Node 24 via.github/workflows/og-preview-tests.yml, and theOG preview testsrun is green on the current tip). Before this change the suite was 25/25; this PR adds 9 tests.extension.mjs,lib/http-fetch.mjs,lib/agent-readiness.mjs) reverted tomain, 7 of the 34 tests fail (truncate mode, the oversized-page server test, and the agent-readiness probe test) and 27 pass. The 2 new tests that pass there (default and explicit"error"mode still reject over the cap) confirm default behavior is unchanged.extension.mjsreverted to the previous commit (no 16 MiB Browse cap), the updated oversized-page test fails and the other server tests still pass.truncated: true(single and multi-chunk); a body at or under the cap is returned whole withtruncated: false; an endless stream is closed after truncation; an oversized HTML page still returns metadata and a whole Browse document (a 7 MiB page, including content after the 6 MiB mark) while a 17 MiB page is cut but not rejected and oversized subresources and non-HTML main documents still fail; and the agent-readiness probes no longer report oversized responses as unavailable./api/fetchreturned 200 in ~1.7 s with<title>"C# API Reference | Aspire", og:title "C# API Reference · Aspire", og:imagehttps://aspire.dev/og-image.png, andtruncated: true(metadata keeps the 6 MiB cap). The page has 31 tags and the repository is detected./api/proxyreturned 200 with the full 6,451,498-byte document (the page plus ~6.6 KB of injected shim),</html>included. Before the Browse cap it was cut at 6,297,792 bytes with no closing tag.Response exceeded the 6291456-byte limit.http://127.0.0.1:1/) shows a single-prefix banner: "Couldn't load metadata. Check the URL and selected-origin access."node --check .github\extensions\og-preview\ui\app.js-> exit 0 (the UI has no automated tests).pnpm buildand the site's frontend tests, since the change is confined to.github/extensions/og-preview/.