Fix browser video lighting with window area selection

README.md

@@ -59,7 +59,7 @@ Each mode remembers its tuning. Browsing views and previewing effects does not s
 | Mode | What you can do |
 | --- | --- |
 | **Color** | Choose a solid color with the wheel, RGB/hex input, palettes, or approximate white temperature. Tune bias lighting with per-channel gains and saved calibration notes. |
-| **Screen** | Sample a full display, a selected window, or a rectangular region. Adjust black-bar detection, RGB balance, gamma, and saturation. |
+| **Screen** | Sample a full display, a selected window, or a rectangular region. Preview a window and select just its video area to exclude browser toolbars and white margins. Adjust black-bar detection, RGB balance, gamma, and saturation. |
 | **Music** | React to computer audio or a microphone with spectrum, bass, level, and wave styles, plus beat response and gain/noise-gate/release tuning. |
 | **Blend** | Combine screen colors with audio-reactive brightness; keep the available source running if the other fails. |
 | **Effects** | Run Color flow, Aurora, Breathe, Comet, and Ember, or create layered effects with gradients, waves, breathing, and color sequences. |

control/Cargo.toml

@@ -1,6 +1,6 @@
 [package]
 name = "dynamiclight-control"
-version = "0.16.1"
+version = "0.16.2"
 edition = "2021"
 license = "MIT"
 rust-version = "1.99"

docs/capture-sources.md

@@ -1,12 +1,18 @@
 # Window and region capture
 
-In Screen or Blend, choose **Capture source**. Full display keeps the existing display selection. Rectangle captures a selected portion of that display; drag its outline or enter left/top/width/height percentages. The diagram previews geometry without taking a screenshot. Once started, the regular LED preview shows sampled edge colors.
+In Screen or Blend, choose **Capture source**. Full display keeps the existing display selection. Rectangle captures a selected portion of that display; drag its outline or enter left/top/width/height percentages. The display diagram previews geometry without taking a screenshot. Once started, the regular LED preview shows sampled edge colors.
+
+For a video playing inside a browser page, choose **Application window**, refresh the window list, and select the browser. Click **Take window preview**, then drag a rectangle around the video itself. **Apply source**, then **Start / Retry**. The lights sample the selected area's edges; including white page margins or toolbars can produce white LEDs even when the video is orange. Alternatively, enable **Select an area within this window** and enter percentages without taking a preview. Uncheck it to use the whole window, for example with fullscreen video.
+
+The preview button explicitly captures one still image of the chosen window. It is downscaled in memory, displayed only in this dialog, and discarded on close or source change; it is never saved, exported, or sent to the network. Opening the dialog and refreshing window names do not take screenshots. Taking a preview does not start LED output or authorize ongoing capture. Windows uses the same capture backend for the preview and lighting.
+
+Window capture follows the visible contents of the browser window, not a pinned tab or a detected video element. Switching tabs changes what is captured. A selected area keeps its relative position and size when the window resizes; take another preview and reselect the video if the page layout changes. Black-bar detection can trim dark letterboxing inside the selected area; it does not remove white browser UI.
 
 Window selection lists available applications and titles locally. Choose **Refresh windows**, select the intended window, and Apply. Applying a new source stops capture; Start / Retry is required to capture it. Movement and resizing are followed. Minimization, closure, or a revoked source interrupts screen capture. Blend can continue with its available audio source. Restore/reselect the window and Retry; opt-in capture recovery retains its usual behavior.
 
 Window IDs are bound to their process and the current DynamicLight session. Re-select a window after restarting DynamicLight. This prevents a saved window ID being silently reused for another application. Local profiles retain source bindings; exported scenes exclude window identities, titles, and region/display bindings.
 
-Rectangle coordinates are normalized to the selected display and rounded out to physical pixel boundaries. The rectangle therefore retains its relative coverage at different scale factors. Display resolution/rearrangement still uses the existing source-change recovery policy. The selected physical display determines LED edge allocation; a window or rectangle changes sampling coordinates without reassigning LEDs to different physical edges.
+Rectangle coordinates are normalized to the selected display, or to the selected window for a window area, and rounded out to physical pixel boundaries. The rectangle therefore retains its relative coverage at different scale factors. Display resolution/rearrangement still uses the existing source-change recovery policy. The selected physical display determines LED edge allocation; a window or rectangle changes sampling coordinates without reassigning LEDs to different physical edges.
 
 ## Platform behavior
 
@@ -16,4 +22,4 @@ Rectangle coordinates are normalized to the selected display and rounded out to
 
 Protected video, DRM, secure desktops, permission restrictions, and unsupported GPU paths can yield black or unavailable frames. The application does not bypass these protections. Captured pixels are sampled locally; only RGB lighting colors reach USB. Window titles and source IDs are excluded from diagnostic exports and the automation status API.
 
-Native validation includes rectangle rounding at multiple resolutions, source-change consent revocation, and an interactive Windows test that queries the audio device before repeatedly starting/stopping capture of its own generated fixture window, resizes it, then verifies capture terminates on closure. No USB or audio recording is used by that test. Each Windows lighting worker initializes the multithreaded Windows Runtime before audio discovery and retains it across capture sessions, preventing the audio library's default single-threaded COM initialization from blocking later window capture with `0x80010106`. A separate regression test checks that worker lifetime without opening capture or audio devices. Physical multi-DPI, protected video, macOS and Linux window behavior remain in the platform validation trackers.
+Native validation includes rectangle rounding at multiple resolutions, padded RGBA/BGRA sampling, source-change consent revocation, and export privacy. An interactive Windows test queries the audio device before repeatedly starting/stopping capture of its own generated fixture window. It reproduces white margins around an orange video, verifies selected-area output and one-shot preview colors, checks live red-to-blue changes, grows and shrinks the window, then verifies capture terminates on closure. No USB or audio recording is used by that test. Each Windows lighting worker initializes the multithreaded Windows Runtime before audio discovery and retains it across capture sessions, preventing the audio library's default single-threaded COM initialization from blocking later window capture with `0x80010106`. A separate regression test checks that worker lifetime without opening capture or audio devices. Physical multi-DPI, protected video, macOS and Linux window behavior remain in the platform validation trackers.

docs/release-notes.md

@@ -1,10 +1,12 @@
-DynamicLight 0.16.1 fixes Windows selected-window capture failing with “Cannot change thread mode after it is set” (0x80010106) after audio initialization.
+DynamicLight 0.16.2 adds video-area selection inside captured windows. A video playing inside a browser page can now drive the LEDs without white page margins and browser toolbars dominating the colors.
 
-Each lighting worker now initializes the Windows Runtime before audio discovery and keeps it initialized across capture sessions. Switching between Music, Screen, and Blend no longer creates that thread-mode conflict. Unrelated capture errors also no longer incorrectly suggest that your Windows version is too old.
+In Screen or Blend, open **Capture source → Application window**, select the browser, and click **Take window preview**. Drag around the video, then **Apply source → Start / Retry**. You can also select an area using percentages without taking a preview. Uncheck the area option for whole-window or fullscreen capture. The window follows its visible tab; after a page layout change, reselect the video area.
 
-Validation includes 100 passing native tests, a regression test verified to fail before the fix, and a real Windows capture fixture covering audio-device discovery, repeated capture startup/teardown, resizing, and window closure. The fixture records only its own generated window; it does not use USB or record audio.
+Previews are explicit, one-shot captures kept in memory while the source dialog is open. They do not start the lights or ongoing capture. Source bindings and crop coordinates stay local and are excluded from portable scene exports. Windows previews use the same capture backend as lighting, and undefined texture padding is excluded from both.
 
-All features from 0.16.0 remain available, including Studio effects and zones, RGB recording/replay, routines, independent controllers, game telemetry, declarative plugins, the CLI, and paired HTTPS phone controls.
+Validation includes native sampling and privacy tests, browser UI regression tests, and a real Windows capture fixture that reproduces white margins around an orange video, verifies the cropped colors, follows live color changes, resizes in both directions, and stops on closure. The fixture captures only its own generated window; it does not use USB or record audio.
+
+The Windows capture initialization fix from 0.16.1 remains included, along with Studio effects and zones, RGB recording/replay, routines, independent controllers, game telemetry, declarative plugins, the CLI, and paired HTTPS phone controls.
 
 Use **Settings → App updates → Check for updates → Install and restart** on Windows, writable macOS app installations, and Linux AppImage. Debian packages use the terminal installer or package manager. After restarting, select the intended window again and explicitly start capture. Windows selected-window capture requires Windows 10 1903 or newer.
 

package.json

@@ -1,6 +1,6 @@
 {
   "name": "dynamiclight",
-  "version": "0.16.1",
+  "version": "0.16.2",
   "private": true,
   "license": "MIT",
   "author": "mskutin",

scripts/ui-test.cjs

@@ -225,6 +225,7 @@ const { chromium } = require(process.env.PLAYWRIGHT_PATH || "playwright");
           progress: null,
         },
         updateInstalls: 0,
+        windowPreviewCalls: [],
         state,
         calls,
         emitEvent: (name, payload) => listeners[name]?.({ payload }),
@@ -369,7 +370,32 @@ const { chromium } = require(process.env.PLAYWRIGHT_PATH || "playwright");
                     session: "test",
                   },
                 },
+                {
+                  name: "Other window",
+                  source: {
+                    kind: "window",
+                    id: 789,
+                    pid: 456,
+                    app: "Player",
+                    session: "test",
+                  },
+                },
               ];
+            if (cmd === "capture_window_preview") {
+              fixture.windowPreviewCalls.push(structuredClone(args.source));
+              if (fixture.deferWindowPreview)
+                await new Promise((resolve) => {
+                  fixture.resolveWindowPreview = resolve;
+                });
+              if (fixture.failWindowPreview)
+                throw new Error("Window preview unavailable. Enter an area manually.");
+              const width = 40, height = 30, rgba = [];
+              for (let y = 0; y < height; ++y)
+                for (let x = 0; x < width; ++x)
+                  rgba.push(...(x >= 10 && x < 30 && y >= 6 && y < 24
+                    ? [240, 90, 20, 255] : [255, 255, 255, 255]));
+              return { width, height, rgba };
+            }
             if (cmd === "studio_library") {
               if (args.next) fixture.studioLibrary = args.next;
               return structuredClone(
@@ -1863,14 +1889,103 @@ const { chromium } = require(process.env.PLAYWRIGHT_PATH || "playwright");
     await page.locator("#source-kind").selectOption("window");
     await page.locator("#source-refresh").click();
     await page.locator("#source-windows").selectOption("0");
+    await page.locator("#source-crop").check();
+    for (const [key, value] of Object.entries({ left: "10", top: "10", width: "80", height: "80" }))
+      await page.locator("#source-" + key).fill(value);
+    assert.equal(
+      await page.evaluate(() => fixture.windowPreviewCalls.length),
+      0,
+      "Opening, refreshing, selecting a window and editing its area never capture an image",
+    );
+    await page.locator("#source-snapshot").click();
+    await page.waitForFunction(() => document.querySelector("#source-preview").width === 40);
+    assert.deepEqual(
+      await page.locator("#source-preview").evaluate((canvas) => {
+        const ctx = canvas.getContext("2d");
+        return [Array.from(ctx.getImageData(1, 1, 1, 1).data),
+          Array.from(ctx.getImageData(20, 15, 1, 1).data)];
+      }),
+      [[255, 255, 255, 255], [240, 90, 20, 255]],
+      "Explicit preview displays the video colors inside the white browser page",
+    );
+    await page.locator("#source-preview").scrollIntoViewIfNeeded();
+    const previewBox = await page.locator("#source-preview").boundingBox();
+    await page.mouse.move(previewBox.x + previewBox.width * .25, previewBox.y + previewBox.height * .2);
+    await page.mouse.down();
+    await page.mouse.move(previewBox.x + previewBox.width * .75, previewBox.y + previewBox.height * .8, { steps: 5 });
+    await page.mouse.up();
+    const videoArea = { left: 250, top: 200, width: 500, height: 600 };
+    const readAreaFields = () => page.evaluate(() => Object.fromEntries(
+      ["left", "top", "width", "height"].map((key) => [key, Number(document.querySelector("#source-" + key).value) * 10]),
+    ));
+    assert.deepEqual(await readAreaFields(), videoArea, "Dragging stores normalized window coordinates");
+    await page.screenshot({ path: ".local/ui-window-video-area.png" });
+    await page.locator("#source-refresh").click();
+    await page.waitForFunction(() => !document.querySelector("#source-refresh").disabled);
+    assert.equal(await page.locator("#source-windows").inputValue(), "0");
+    assert.equal(await page.locator("#source-crop").isChecked(), true);
+    assert.deepEqual(await readAreaFields(), videoArea, "Refreshing preserves the selected video area");
     await page.locator("#source-apply").click();
     await page.waitForFunction(
       () => !document.querySelector("#source-dialog").open,
     );
-    assert.equal(
-      await page.evaluate(() => fixture.state.settings.captureSource.id),
-      123,
+    assert.deepEqual(
+      await page.evaluate(() => fixture.state.settings.captureSource),
+      { kind: "window", id: 123, pid: 456, app: "Player", session: "test", crop: videoArea },
     );
+    await page.locator("#source-open").click();
+    await page.waitForFunction(() => document.querySelector("#source-dialog").open);
+    assert.equal(await page.locator("#source-windows").inputValue(), "0");
+    assert.equal(await page.locator("#source-crop").isChecked(), true);
+    assert.deepEqual(await readAreaFields(), videoArea, "Reopening restores the saved area");
+    assert.equal(await page.evaluate(() => fixture.windowPreviewCalls.length), 1);
+    assert.equal(await page.locator("#source-preview").evaluate((canvas) => canvas.width), 600,
+      "Reopening does not retain the private window image");
+    await page.setViewportSize({ width: 540, height: 780 });
+    assert(await page.locator("#source-dialog").evaluate((dialog) => dialog.scrollWidth <= dialog.clientWidth),
+      "Window video area controls fit a narrow window");
+    await page.screenshot({ path: ".local/ui-window-video-area-narrow.png" });
+    await page.setViewportSize({ width: 1280, height: 1000 });
+
+    // An in-flight native result must never reveal an old source after selection changes or closing.
+    await page.evaluate(() => { fixture.deferWindowPreview = true; });
+    await page.locator("#source-snapshot").click();
+    await page.waitForFunction(() => fixture.windowPreviewCalls.length === 2);
+    await page.locator("#source-windows").selectOption("1");
+    await page.evaluate(async () => {
+      fixture.resolveWindowPreview();
+      await new Promise((resolve) => requestAnimationFrame(resolve));
+    });
+    assert.equal(await page.locator("#source-preview").evaluate((canvas) => canvas.width), 600,
+      "Changing windows discards a late preview");
+    assert.equal(await page.locator("#source-crop").isChecked(), false);
+    await page.locator("#source-snapshot").click();
+    await page.waitForFunction(() => fixture.windowPreviewCalls.length === 3);
+    await page.locator("#source-cancel").click();
+    await page.evaluate(async () => {
+      fixture.resolveWindowPreview();
+      await new Promise((resolve) => requestAnimationFrame(resolve));
+      fixture.deferWindowPreview = false;
+    });
+    await page.locator("#source-open").click();
+    await page.waitForFunction(() => document.querySelector("#source-dialog").open);
+    assert.equal(await page.locator("#source-preview").evaluate((canvas) => canvas.width), 600,
+      "Closing discards a late preview, even after reopening");
+    assert.deepEqual(await readAreaFields(), videoArea, "Cancel preserves the saved area");
+    await page.evaluate(() => { fixture.failWindowPreview = true; });
+    await page.locator("#source-snapshot").click();
+    await page.waitForFunction(() => /Enter an area manually/.test(document.querySelector("#source-status").textContent));
+    await page.locator("#source-left").fill("20");
+    await page.locator("#source-apply").click();
+    await page.waitForFunction(() => !document.querySelector("#source-dialog").open);
+    assert.equal(await page.evaluate(() => fixture.state.settings.captureSource.crop.left), 200,
+      "Manual area selection remains usable when preview fails");
+    await page.locator("#source-open").click();
+    await page.locator("#source-crop").uncheck();
+    await page.locator("#source-apply").click();
+    await page.waitForFunction(() => !document.querySelector("#source-dialog").open);
+    assert.equal(await page.evaluate(() => fixture.state.settings.captureSource.crop), undefined,
+      "Disabling the area returns to the whole selected window");
     await page.locator('[data-view="settings"]').click();
     // Appearance preferences cover every supported palette and both color schemes.
     await page.setViewportSize({ width: 1120, height: 780 });

src-tauri/Cargo.toml

@@ -1,6 +1,6 @@
 [package]
 name = "dynamiclight"
-version = "0.16.1"
+version = "0.16.2"
 description = "Local USB monitor backlight controller"
 authors = ["mskutin"]
 license = "MIT"