Skip to content

Support voice interaction as another assistant input and output #1582

Description

@asim

Priority: P1 — bring the requested foreground “Hey Micro” entry point forward alongside direct commands.

Voice should make the same assistant accessible hands-free, without creating another conversation or execution system.

Acceptance criteria

  • Add explicit microphone start/stop, recording feedback and transcript review/correction.
  • Send voice-derived requests through the same identity, context, command and task paths as text.
  • Offer optional spoken responses, mute and interruption controls.
  • Handle mobile permission denial, backgrounding, connection loss and accidental recording safely.
  • Define recording/transcript retention and deletion clearly; avoid ambient recording by default.
  • Test a complete voice request and follow-up on mobile/PWA; measure latency rather than assuming full realtime video infrastructure is required.

Dependencies and boundaries

User-facing acceptance criteria for existing #1487; use its transport work rather than duplicate it. Voice is the first slice; video is not a prerequisite.

Implementation approach

Audit current main first: some foundations and recent fixes already exist. Close remaining user-visible gaps and preserve working behaviour. This issue records the personal-assistant product backlog, not a claim that every underlying capability is absent. Add focused regression coverage for the behaviour changed.

Latest agreed scope: Hey Micro activates existing dictation

Keep the current Speak behaviour. Add opt-in “Hey Micro” detection to activate the existing microphone speech-to-text experience and fill the editable input. The user checks the transcript and presses Send; do not auto-send or change spoken-reply preferences.

This foreground browser implementation is in #1595. Listening must be visible and stop on backgrounding/navigation. Clearly disclose that browser recognition can send audio to its speech provider; do not claim local-only wake detection or OS-level background listening. The wake phrase is activation, not identity or authorization.

A separate speech backend, local wake engine and native/background support are possible later improvements, not prerequisites for this agreed slice. Actual microphone performance on phones remains a device validation item.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions