diff --git a/docs/architecture/transport-and-protocol.md b/docs/architecture/transport-and-protocol.md index af30e96..a3591c3 100644 --- a/docs/architecture/transport-and-protocol.md +++ b/docs/architecture/transport-and-protocol.md @@ -101,6 +101,8 @@ Three kinds of target get a session of their own: **Execution context IDs are numbered per session.** Two renderers will both hand out context 1, so the map from context to frame is keyed by session and context together. A frame's context ID is only meaningful on the session that owns that frame, and every call site pairs the two. +**A frame's context ID is replaced on every navigation.** `Runtime.executionContextCreated` records the main world of the frame named in its `auxData` (an isolated world, with `isDefault: false`, never replaces it), `Runtime.executionContextDestroyed` drops the entry with that ID from the session that reported it, and `Runtime.executionContextsCleared` drops every entry that session reported. The three are read from one channel per session, so a navigation's teardown is always handled before the context that replaces it. They do not share a channel with `Page.loadEventFired`, so a navigation can complete while the old context is still recorded or before the new one is. Commands that name a context cover both cases. A frame with nothing recorded waits for its next context, bounded by the caller's timeout or 30 seconds, and fails at once with `MotusTargetClosedException` if the frame is detached or the page closes; the main frame instead leaves the ID out, which the browser resolves to the main frame's current main world. A command refused with "Cannot find context with specified id" is sent once more against whatever has been recorded since. The browser looks the context up before running anything, so the refused attempt never ran and the retry cannot run a script twice. BiDi has no counterpart: script there is addressed to a browsing context and runs in whatever realm it currently holds, so there is no ID to track or wait for. + ### Routing a frame to its session `Page.SessionFor(IFrame?)` is the single lookup. A frame that owns a session is routed to it; every other frame, including a same-process child of an out-of-process frame, falls through to the page's own session. That fallback is what keeps every existing round trip byte-identical. diff --git a/docs/guides/frames-and-iframes.md b/docs/guides/frames-and-iframes.md index 1d4881b..9a3c831 100644 --- a/docs/guides/frames-and-iframes.md +++ b/docs/guides/frames-and-iframes.md @@ -90,7 +90,9 @@ if (frame.IsDetached) } ``` -`IsDetached` is how a removed frame is told apart from a selector that simply did not match. A held `IFrame` answers rather than failing obscurely several calls later. +`IsDetached` is how a removed frame is told apart from a selector that simply did not match. A held `IFrame` answers rather than failing obscurely several calls later. Evaluating, reading content or resolving a locator through a detached frame throws `MotusTargetClosedException` straight away, and so does a call that was already waiting on the frame when it went. + +An evaluation, a locator or a content read issued the moment a frame finishes navigating runs in the new document. A frame's main world is replaced on every navigation and the browser reports the new one separately from the end of the navigation, so for a frame other than the main frame such a call waits for the new world to be reported, for up to 30 seconds or the call's own timeout, rather than running against the old document or in the main frame by mistake. --- @@ -159,7 +161,7 @@ Each of these costs real time to rediscover. **The endpoint's target list is not a list of a page's frames.** `http://127.0.0.1:9222/json/list` reports one entry per CDP target, and only a frame the browser put in its own process has a target of its own. So the list shows those, typed `iframe` rather than `page`, and shows nothing at all for every frame the page's own renderer hosts. A document with two frames, one cross-origin and one `srcdoc`, produces exactly one `iframe` entry. Counting entries there will undercount the frames on the page, and filtering for `page` entries will miss all of them. Motus builds `page.Frames` from auto-attach and the frame tree instead, which covers both kinds, so read frames from there and treat the target list as a view of processes rather than of documents. -**A frame changing process is not a frame going away, but a frame whose parent navigated is.** The browser announces a frame moved into another renderer process as a detach carrying the reason `swap`, and says the same for a document it puts into the back-forward cache. Motus tells both apart from a real removal: the frame stays in `page.Frames`, `IsDetached` stays false, no `FrameDetached` arrives, and the `IFrame` being held keeps working across the move. Navigation is the other case. Committing a new document in a frame drops the frames that frame hosted, because the new document reports its own, so a child frame is rediscovered after its parent navigates and comes back as a different `IFrame`, even when it loads the same URL. What that leaves to watch for is the handle from before. It reports `IsDetached`, and checking that is worth the line, because a frame the page no longer lists has no execution context of its own to aim at and an evaluation through it falls back to the page's, quietly answering for the main document rather than failing. Read the frame from `page.Frames` again after a navigation and wait for it as under [Traversing](#traversing), since for a moment after the parent commits its new document the subtree is empty. +**A frame changing process is not a frame going away, but a frame whose parent navigated is.** The browser announces a frame moved into another renderer process as a detach carrying the reason `swap`, and says the same for a document it puts into the back-forward cache. Motus tells both apart from a real removal: the frame stays in `page.Frames`, `IsDetached` stays false, no `FrameDetached` arrives, and the `IFrame` being held keeps working across the move. Navigation is the other case. Committing a new document in a frame drops the frames that frame hosted, because the new document reports its own, so a child frame is rediscovered after its parent navigates and comes back as a different `IFrame`, even when it loads the same URL. What that leaves to watch for is the handle from before. It reports `IsDetached`, and anything run through it throws `MotusTargetClosedException` rather than answering for some other document. Read the frame from `page.Frames` again after a navigation and wait for it as under [Traversing](#traversing), since for a moment after the parent commits its new document the subtree is empty. **Application globals are absent from isolated worlds.** An isolated world shares the frame's document and nothing else, so the DOM is fully present while everything the application put on `window` is not. An expression that reads a framework handle returns `undefined` there and the failure looks like the application not having loaded. Use `ExecutionWorld.Main`, which is the default, whenever you are reading application state. diff --git a/src/Motus.Mcp/Snapshot/FrameCollector.cs b/src/Motus.Mcp/Snapshot/FrameCollector.cs index fbc932d..f1134f9 100644 --- a/src/Motus.Mcp/Snapshot/FrameCollector.cs +++ b/src/Motus.Mcp/Snapshot/FrameCollector.cs @@ -32,6 +32,10 @@ internal static class FrameCollector private const string SelfIndexExpression = "(() => { const p = window.parent; for (let i = 0; i < p.length; i++) { if (p[i] === window) return i; } return -1; })()"; + // How long the frames of one document get to say which of their parent's contexts they are. + // A frame with a document answers in milliseconds. + private static readonly TimeSpan SelfIndexAllowance = TimeSpan.FromSeconds(1); + // Which child browsing context this element holds, asked of the document the element lives in. private const string HostIndexExpression = "el => { const w = el.ownerDocument.defaultView; for (let i = 0; i < w.length; i++) { if (w[i] === el.contentWindow) return i; } return -1; }"; @@ -182,24 +186,32 @@ void Walk(IFrame frame) /// Frames that cannot answer, and any index two frames both claim, are left out, so a reading /// that is not trustworthy falls through to the order the engine gives instead of guessing. /// + /// + /// A frame that has not loaded a document yet, such as a lazy iframe still below the fold, has + /// no context to evaluate in, and an evaluation there waits for one until its own timeout. So + /// every frame is asked at once and the answers are read after + /// at most: a frame that cannot answer costs the snapshot that allowance once, rather than a + /// full evaluation timeout for each such frame. + /// private static async Task> ByContextIndexAsync( IReadOnlyList" + + "
" + + $"" + + $""); + + var stopwatch = Stopwatch.StartNew(); + var text = TextOf(await SnapshotAsync()); + stopwatch.Stop(); + + StringAssert.Contains(text, "- button \"Pay now\""); + Assert.IsTrue( + stopwatch.Elapsed < TimeSpan.FromSeconds(10), + $"the snapshot took {stopwatch.ElapsedMilliseconds} ms"); + } + [TestMethod] public async Task WithNoRoomForFrames_ThePageSaysWhatItLeftOut() { diff --git a/tests/Motus.Mcp.Tests/Snapshot/InlineFrameSnapshotTests.cs b/tests/Motus.Mcp.Tests/Snapshot/InlineFrameSnapshotTests.cs index 2d6e63b..d36b08c 100644 --- a/tests/Motus.Mcp.Tests/Snapshot/InlineFrameSnapshotTests.cs +++ b/tests/Motus.Mcp.Tests/Snapshot/InlineFrameSnapshotTests.cs @@ -110,6 +110,30 @@ public async Task EachFrame_NumbersItsOwnRefsBehindItsOwnIndex() /// /// A page whose only content is an element hosting one frame, and that frame's own document. /// + [TestMethod] + public async Task AFrameThatCannotEvaluateYet_DoesNotHoldUpThePageSnapshot() + { + // A lazy iframe below the fold has no document, so an evaluation in it waits for one. + var page = new FakeToolPage(Document( + Node("Iframe", "Payment frame", 7), + Node("Iframe", "Lazy frame", 8))); + var main = new FakeToolFrame(page, "http://example.test/"); + page.FrameTree = main; + + var loaded = main.AddChild("http://example.test/frame.html"); + loaded.Snapshot = Document(Node("button", "Pay now", 42)); + + var lazy = main.AddChild(""); + lazy.EvaluateNeverCompletes = true; + lazy.Snapshot = Document(); + + var snapshot = new PageSnapshotService(page).TakeSnapshotAsync(); + var finished = await Task.WhenAny(snapshot, Task.Delay(TimeSpan.FromSeconds(10))); + + Assert.AreSame(snapshot, finished, "the snapshot waited on a frame that could not answer"); + StringAssert.Contains(await snapshot, "button \"Pay now\""); + } + private static FakeToolPage PageWithOneFrame(out FakeToolFrame frame) { var page = new FakeToolPage(Document(Node("Iframe", "Payment frame", 7))); diff --git a/tests/Motus.Mcp.Tests/Tools/CoreToolsFakes.cs b/tests/Motus.Mcp.Tests/Tools/CoreToolsFakes.cs index 816011f..7b821a7 100644 --- a/tests/Motus.Mcp.Tests/Tools/CoreToolsFakes.cs +++ b/tests/Motus.Mcp.Tests/Tools/CoreToolsFakes.cs @@ -1016,10 +1016,18 @@ public ILocator LocatorByBackendNodeId(long backendNodeId) return page.RecordingLocator; } + /// + /// When set, never completes, like a frame that has no document + /// to evaluate in yet. + /// + public bool EvaluateNeverCompletes { get; set; } + public Task EvaluateAsync(string expression, object? arg = null) { Evaluated.Add(expression); - return Task.FromResult(default(T)!); + return EvaluateNeverCompletes + ? new TaskCompletionSource().Task + : Task.FromResult(default(T)!); } public Task EvaluateAsync(string expression, object? arg, EvaluateOptions options) diff --git a/tests/Motus.Tests/Browser/FirefoxIntegrationTests.cs b/tests/Motus.Tests/Browser/FirefoxIntegrationTests.cs index 1ba9f42..73aabee 100644 --- a/tests/Motus.Tests/Browser/FirefoxIntegrationTests.cs +++ b/tests/Motus.Tests/Browser/FirefoxIntegrationTests.cs @@ -138,6 +138,38 @@ public async Task Firefox_NewPage_HasMainFrameAndTakesContentWithoutNavigation() Assert.AreEqual(2, await page.MainFrame.EvaluateAsync("1 + 1")); } + /// + /// Evaluating straight after a navigation works on Firefox too. BiDi addresses script to a + /// browsing context rather than to an execution context id, so there is no id to go stale, and + /// the waiting Chromium needs must not get in its way. + /// + [TestMethod] + public async Task Firefox_EvaluateRightAfterNavigation_Succeeds() + { + if (!FirefoxAvailable()) + { + Assert.Inconclusive("Firefox not found on this machine."); + return; + } + + await using var browser = await MotusLauncher.LaunchAsync( + new LaunchOptions { Headless = true, Channel = BrowserChannel.Firefox }); + + for (var i = 0; i < 30; i++) + { + var context = await browser.NewContextAsync(); + var page = await context.NewPageAsync(); + + await page.GotoAsync("about:blank"); + Assert.AreEqual(2, await page.EvaluateAsync("1+1"), $"iteration {i}"); + + await page.GotoAsync($"data:text/html,t{i}"); + Assert.AreEqual($"t{i}", await page.TitleAsync(), $"iteration {i}"); + + await context.CloseAsync(); + } + } + /// /// Pointer actions scroll their target into view with a script that runs in the page rather /// than with a Chromium-only command, so the script has to behave the same under Firefox. It is diff --git a/tests/Motus.Tests/Page/ExecutionContextIntegrationTests.cs b/tests/Motus.Tests/Page/ExecutionContextIntegrationTests.cs new file mode 100644 index 0000000..bafcb00 --- /dev/null +++ b/tests/Motus.Tests/Page/ExecutionContextIntegrationTests.cs @@ -0,0 +1,160 @@ +using Motus.Abstractions; + +namespace Motus.Tests.Page; + +/// +/// Evaluating the moment a navigation completes, against a real browser. +/// +/// +/// A navigation's load event and the new document's execution context arrive separately, so a +/// caller that evaluates straight after GotoAsync could reach the page before the new +/// context had been recorded and be refused with "Cannot find context with specified id". It only +/// happened some of the time, so these tests repeat the sequence rather than relying on one run. +/// +[TestClass] +[TestCategory("Integration")] +public class ExecutionContextIntegrationTests +{ + private const int Iterations = 40; + + private CrossOriginFixtureServer _server = null!; + private IBrowser? _browser; + + [TestInitialize] + public async Task Setup() + { + _server = new CrossOriginFixtureServer(); + + try + { + _browser = await MotusLauncher.LaunchAsync(new LaunchOptions + { + Headless = true, + Args = _server.IsolationArgs + }); + } + catch (FileNotFoundException) + { + Assert.Inconclusive("No browser found; skipping integration tests."); + } + } + + [TestCleanup] + public async Task Cleanup() + { + if (_browser is not null) + await _browser.DisposeAsync(); + + _server?.Dispose(); + } + + /// + /// The sequence the problem was reported with: a fresh context and page, one navigation, and + /// an evaluation straight after it. + /// + [TestMethod] + public async Task EvaluateRightAfterNavigatingAFreshPage_Succeeds() + { + for (var i = 0; i < Iterations; i++) + { + var context = await _browser!.NewContextAsync(); + var page = await context.NewPageAsync(); + + await page.GotoAsync("about:blank"); + Assert.AreEqual(2, await page.EvaluateAsync("1+1"), $"iteration {i}"); + + await page.GotoAsync($"data:text/html,t{i}

{i}

"); + Assert.AreEqual($"t{i}", await page.TitleAsync(), $"iteration {i}"); + StringAssert.Contains(await page.ContentAsync(), $"

{i}

"); + + await context.CloseAsync(); + } + } + + /// + /// A navigation to another site moves the page into a new renderer process, which replaces the + /// main world with one numbered by a different process. + /// + [TestMethod] + public async Task EvaluateRightAfterACrossOriginNavigation_RunsInTheNewDocument() + { + var page = await _browser!.NewPageAsync(); + + for (var i = 0; i < Iterations / 2; i++) + { + await page.GotoAsync(_server.PrimaryOrigin + "/deep.html"); + Assert.AreEqual("deep", await page.EvaluateAsync("window.marker"), $"iteration {i}"); + + await page.GotoAsync(_server.SecondaryOrigin + "/middle.html"); + Assert.AreEqual("middle", await page.EvaluateAsync("window.marker"), $"iteration {i}"); + Assert.AreEqual(_server.SecondaryOrigin, await page.EvaluateAsync("location.origin")); + } + } + + /// + /// A frame that navigates on its own replaces only its own main world. An evaluation in it must + /// reach the new document, not the old one and not the page's main frame. + /// + [TestMethod] + public async Task EvaluateInAFrameRightAfterItNavigates_RunsInThatFramesNewDocument() + { + var page = await _browser!.NewPageAsync(); + await page.GotoAsync(_server.OrderedUrl); + var frame = await FirstChildFrameAsync(page); + + for (var i = 0; i < Iterations / 4; i++) + { + await frame.GotoAsync(_server.PrimaryOrigin + "/deep.html"); + Assert.AreEqual("deep", await frame.EvaluateAsync("window.marker"), $"iteration {i}"); + Assert.AreEqual(0, await frame.EvaluateAsync("window.clicks")); + + await frame.GotoAsync(_server.PrimaryOrigin + "/ordered-child.html?again=" + i); + Assert.AreEqual( + "child", + await frame.EvaluateAsync("document.querySelector('p').textContent"), + $"iteration {i}"); + } + + Assert.IsNull( + await page.EvaluateAsync("window.marker ?? null"), + "the frame's evaluations never landed in the main frame"); + } + + /// + /// Same as above for a locator, which resolves through the frame's context rather than through + /// an evaluation of the caller's own. + /// + [TestMethod] + public async Task LocatorInAFrameRightAfterItNavigates_ResolvesInThatFramesNewDocument() + { + var page = await _browser!.NewPageAsync(); + await page.GotoAsync(_server.OrderedUrl); + var frame = await FirstChildFrameAsync(page); + + for (var i = 0; i < Iterations / 4; i++) + { + await frame.GotoAsync(_server.PrimaryOrigin + "/deep.html"); + Assert.AreEqual("deep", await frame.Locator("#target").TextContentAsync(), $"iteration {i}"); + + await frame.GotoAsync(_server.PrimaryOrigin + "/ordered-child.html?again=" + i); + Assert.AreEqual("child", await frame.Locator("p").TextContentAsync(), $"iteration {i}"); + } + } + + private static async Task