Skip to content

Ship meeting AX chat capture and disclosure - #7034

Open
ComputelessComputer wants to merge 27 commits into
mainfrom
fix/aside-meet-chat-ax-truncation
Open

Ship meeting AX chat capture and disclosure#7034
ComputelessComputer wants to merge 27 commits into
mainfrom
fix/aside-meet-chat-ax-truncation

Conversation

@ComputelessComputer

@ComputelessComputer ComputelessComputer commented Aug 22, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • Hardens evidence-scoped AX and AT-SPI meeting chat capture for current Zoom, Meet, Teams, Webex, and Slack accessibility trees, including truncated browser snapshots.
  • Indexes captured meeting chat into session search and keeps chat sending strict and fail-closed.
  • Recognizes tier-one browsers plus secondary and AI browsers including Arc, BrowserOS, Comet, Dia, Helium, Hermes, Opera, SigmaOS, Vivaldi, and Zen. Their explicit process or bundle identifiers are required even when they share Chromium or Mozilla engines.
  • Retires AX, AT-SPI, platform-label, and summary/LLM speaker identification. Automatic speaker identity is now owned exclusively by the consented voiceprint path; anonymous provider diarization and manual speaker assignment remain.
  • Keeps legacy inferred identity hints readable without creating or propagating new ones, and rejects the retired AX confirmation source for new voiceprint exemplars.

Safety boundaries

  • Capture requires one evidence-backed meeting scope and excludes ambiguous or unsupported windows.
  • Incomplete snapshots may be read only after provider classification; mutation still refuses incomplete snapshots.
  • Chat send remains limited to a validated native Slack huddle on macOS.
  • The exported meeting accessibility inspection no longer exposes participant streams, names, active-speaker state, bounds, or speaker confidence.

Validation

  • pnpm fmt:check
  • pnpm -F desktop typecheck
  • pnpm -F desktop test — 390 files / 3376 tests
  • pnpm exec oxlint --quiet --format=github apps/desktop/src/ — 0 errors
  • cargo test --locked -p detect — 89 tests
  • cargo test --locked -p tauri-plugin-detect — 73 tests
  • cargo test --locked -p db-app --lib — 195 tests
  • cargo test -p desktop — 46 tests
  • affected Rust checks, DB clippy, both desktop distribution feature checks, scoped rustfmt, and generated binding stability passed

Note

High Risk
Touches macOS Accessibility mutation and chat capture, plus speaker-identity and voiceprint provenance. Fail-closed scoping reduces some risk, but AX tree walking and identity policy remain security-sensitive.

Overview
Ships evidence-scoped meeting chat capture for Zoom, Meet, Teams, Webex, and Slack (native and web), including truncated browser snapshots. Capture still requires a unique meeting scope; send stays fail-closed (validated native Slack huddle on macOS). Inspections no longer export participant streams, names, or active-speaker state.

Session search now indexes live meeting_chat documents in sort order (sender, text, links), omitting deleted messages.

Retires automatic speaker identification from enhance, AX/AT-SPI, and batch transfer. New identities come only from consented voiceprints plus anonymous diarization and manual assignment. Voiceprint inserts reject accessibility_active_speaker; only manual_speaker_assignment is allowed. Legacy automatic hints remain readable but are not created or copied onto refined transcripts.

Recognizes additional browsers (Comet, BrowserOS, Hermes, SigmaOS, and other Chromium/Mozilla variants) by explicit bundle/process IDs. AX helpers now use raw AX APIs to avoid aborting on stale UI errors, walk visible children on huge trees, and parse more chat layouts.

Reviewed by Cursor Bugbot for commit d1a1c5e. Bugbot is set up for automated code reviews on this repo. Configure here.

@netlify

netlify Bot commented Aug 22, 2026

Copy link
Copy Markdown

Deploy Preview for anarlog canceled.

Name Link
🔨 Latest commit d1a1c5e
🔍 Latest deploy log https://app.netlify.com/projects/anarlog/deploys/6a8b11518a8df1000827cdfd

Comment thread crates/detect/src/meeting_ax.rs Outdated
Comment thread crates/detect/src/meeting_ax/linux.rs
@ComputelessComputer

Copy link
Copy Markdown
Collaborator Author

Live macOS validation for be1bf15061:

  • Slack Huddle native (#test): Scope ✅, Chat ✅, Disclose ✅. The production AX transport opened the Huddle thread and sent through sendButton; capture preserved sender, text, and link.
  • Chrome + Google Meet: Scope ✅, Chat ✅. Incoming split AX leaves were assembled with sender, text, and link.
  • Aside + Google Meet: Scope ✅, Chat ✅. Walking the meeting window handles Aside's focused about:blank web area.
  • Speaker remains fail-closed: in a private two-endpoint Meet call with confirmed remote fake-microphone audio, neither Chrome nor Aside exposed an explicit AX speaking/highlight state. Slack also exposed no explicit local speaking state.

Local validation:

  • pnpm fmt:check
  • cargo test --locked -p detect — 81 passed
  • cargo check --locked -p desktop
  • cargo test --locked -p desktop — 44 passed

Linear ANLG-297's matrix is updated. ANLG-135 now meets its scoped Slack disclosure close gate; ANLG-76 and ANLG-81 remain open for the rest of the required host matrix.

Comment thread crates/detect/src/meeting_ax/analysis.rs
@ComputelessComputer

ComputelessComputer commented Aug 22, 2026

Copy link
Copy Markdown
Collaborator Author

Live native validation update (2026-08-23, commit b1a15fd8b0):

  • Zoom macOS: exactly one native meeting scoped; visible muted participant returned with stable AX identity and positive bounds; one self-authored chat captured with sender You, timestamp 5:05 AM, exact text, outgoing direction, and https://example.com/zoom. No explicit speaking/highlight state was exposed, so speaker detection correctly stayed empty.
  • Microsoft Teams macOS: exactly one authenticated native meeting scoped; visible anon cannon, Organizer, Muted roster row returned as an inactive participant with positive bounds; one chat captured with sender anon cannon, timestamp 5:08 AM, exact text, unknown direction, and https://example.com/teams. The native Sent … Today at … description does not identify author ownership, so direction is intentionally not inferred. No explicit speaker state was exposed, so speaker detection correctly stayed empty.
  • Host-version gaps fixed: direct AXWindows discovery; Zoom AXTabGroup audio-state labels; Teams Leave call control, roster rows, and Sent … Today at … message descriptions.
  • Consent safety aligned to ANLG-135: AX chat mutation is now enabled only for native Slack Huddles on macOS; Zoom, Meet, Teams, Webex, browsers, Linux, and unrelated apps fail closed.

Validated locally: pnpm exec dprint fmt; pnpm fmt:check; macOS cargo test --locked -p detect (84 pass); Linux/arm64 container cargo test --locked -p detect (87 pass); cargo check --locked -p desktop; cargo test --locked -p desktop (44 pass).

ANLG-297 remains open: live Speaker passes, Webex, the remaining tier-one browser/provider cells, and Linux live AT-SPI cells are still required by the Linear close rule.

Comment thread crates/detect/src/meeting_ax/analysis.rs
Comment thread crates/detect/src/meeting_ax/analysis.rs Outdated
Comment thread crates/detect/src/meeting_ax/node.rs Outdated
@ComputelessComputer

ComputelessComputer commented Aug 22, 2026

Copy link
Copy Markdown
Collaborator Author

Update for ea4fb05:

  • Fixed all four latest Bugbot findings and resolved their threads: Linux unrelated incomplete windows no longer over-block send, Teams Sent metadata no longer implies outgoing direction, explicit Teams speaker-state labels are normalized, and bare Leave requires meeting-specific companion evidence.
  • Hardened macOS AX access against stale-window error codes by using raw ApplicationServices status returns instead of assuming every status maps to the closed cidre AXError enum. A live production probe no longer aborts after closing a Huddle.
  • Revalidated native Teams live: active meeting scoped with no warnings; one chat message captured with timestamp and link, and direction now correctly remains unknown.
  • Local checks: macOS detect 85 passed; Linux/arm64 detect 89 passed; final desktop tests 44 passed; desktop cargo check passed; dprint format/check passed.
  • Final head validation is green: active desktop i18n, JS/typecheck/tests, Linux compile/tests, formatting, lint, zizmor, and Bugbot all passed. GitHub reports CLEAN and MERGEABLE. The separately listed red sentinel belongs to an older concurrency-canceled duplicate workflow.

@ComputelessComputer

ComputelessComputer commented Aug 22, 2026

Copy link
Copy Markdown
Collaborator Author

Linux headed AT-SPI QA found and fixed two process-discovery gaps in 2b44211bcc:

  • recognize Mozilla official firefox-bin
  • ignore Sysinfo user threads, zombie processes, and child processes whose parent belongs to the same meeting-app family

Ubuntu 24.04 x86_64 live proof: Firefox 154 exposed a real AT-SPI tree; before the fix the probe emitted dozens of duplicate Firefox inspections, and after the fix it emits exactly one top-level Firefox inspection. The public/first-run page remains safely unscoped; no live meeting matrix pass is claimed.

Local validation: pnpm exec dprint fmt, pnpm fmt:check, cargo check --locked -p detect, all 89 detector tests, and the headed meeting_ax_probe run. Optional cargo clippy -p detect -- -D warnings still reports five pre-existing style lints outside these hunks.

Fresh PR CI is fully green on this revision (JS, Linux desktop, fmt, lint, i18n, repository checks, and Cursor Bugbot); GitHub reports the PR CLEAN and mergeable.

Copy link
Copy Markdown
Collaborator Author

Live macOS tier-one Meet follow-up on b859383fce:

  • Safari, Firefox, and Edge now each pass live meeting scope and chat capture.
  • Capture preserves timestamps, links, and exposed remote sender metadata, and excludes Meet's pin-hover helper.
  • Read-only capture accepts a uniquely scoped visible but transiently disabled composer; mutation remains strict.
  • Controlled virtual-mic speaker transitions stayed fail-closed because none of the three hosts exposed explicit AX speaker/highlight state.
  • Local gates: dprint; macOS detect 86/86; desktop 44/44; Linux x86_64 detect 90/90; all affected checks pass.

Comment thread crates/detect/src/meeting_ax/analysis.rs
Comment thread crates/detect/src/meeting_ax/context.rs
Comment thread crates/detect/src/meeting_ax/analysis.rs
Comment thread crates/detect/src/meeting_ax/analysis.rs
Comment thread crates/detect/src/meeting_ax/analysis.rs
Comment thread crates/detect/src/meeting_ax/context.rs
Comment thread crates/detect/src/meeting_ax/analysis.rs
Comment thread crates/detect/src/meeting_ax/analysis.rs Outdated
@ComputelessComputer ComputelessComputer changed the title fix/aside-meet-chat-ax-truncation AX capturing features Aug 23, 2026
ComputelessComputer and others added 12 commits August 23, 2026 23:06
Aside + Google Meet trees overflow the old 18/1800 AX budget, so
disclosure failed closed even with one identified Meet tab. Accept
those roots, walk chat-labeled nodes first, and still refuse windows
that stay unclassified.
Aside titles the window Meet - xxx-xxxx-xxx while the focused AXWebArea is about:blank, so chat capture never got a meeting root. Prefer visible children when they are a subset, walk the window instead of that blank document, and keep capture identity on the Meet code.
Close gates live on ANLG-297 so chat, speaker, and disclosure share one host matrix.
@ComputelessComputer
ComputelessComputer force-pushed the fix/aside-meet-chat-ax-truncation branch from fe1f6d7 to fbc8d12 Compare August 23, 2026 14:15
Comment thread crates/detect/src/meeting_ax/context.rs
Comment thread crates/detect/src/meeting_ax.rs
@ComputelessComputer
ComputelessComputer force-pushed the fix/aside-meet-chat-ax-truncation branch from ee3a501 to a754406 Compare August 23, 2026 15:18
@ComputelessComputer ComputelessComputer changed the title AX capturing features Ship meeting AX chat capture and disclosure Aug 23, 2026

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.

There are 3 total unresolved issues (including 2 from previous reviews).

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit a754406. Configure here.

Comment thread crates/detect/src/meeting_ax.rs
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant