Sample Library Browser, Step Sequencer, MIDI, Sample Recording, Effects Rack

README.md

@@ -2,14 +2,47 @@
 
 A web and desktop app for managing Akai MPC sample projects and building drum kits. Load your existing MPC kits, preview samples on a familiar pad layout, rearrange and customize your pads, then export the result as a ready-to-use `.xpj` project file — drop it straight onto your MPC hardware or SD card and your kit is ready to play.
 
-Runs as a browser app or as a packaged macOS desktop app (Electron). The desktop build adds native file dialogs for opening and saving `.xpj` files and direct SD card workflow support.
+Runs as a browser app or as a packaged macOS desktop app (Electron). The desktop build adds the sample library browser, the sample recorder, native file dialogs, and the direct SD card workflow.
 
 ![Demo of MPC Sample.app](./demo.jpg)
 
 For a live demo visit [mpcsample.app](mpcsample.app)
 
 ---
 
+## Features
+
+**Kit building**
+- Load, browse, and edit existing MPC `.xpj` kits on a familiar 128-pad (8-bank) layout
+- Drag-and-drop WAV import onto any pad; move/swap pads; per-pad name, coarse/fine tune, and volume
+- Export to a ready-to-use `.xpj` project — drop it straight onto your MPC hardware or SD card
+
+**Sample library browser** (desktop)
+- Point it at a local folder of samples (tested with 25k+ files) — fast background scan + waveform thumbnails, with an on-disk cache so repeat scans are instant
+- Auto-tagging from folder/filename conventions (Kick, Snare, 808, Loop, Vocal, ...) plus manual tags, pack filtering, and drag-to-pad
+- Corrupt/unreadable audio files are automatically hidden rather than shown as silent dead ends
+
+**Step sequencer**
+- Live/MIDI-driven pattern recording (1–8 bars, 8th/16th/32nd resolution), per-step velocity, pattern persisted across reloads
+- Exports into the project's real, editable native MPC sequence data — not just in-app playback
+
+**Sample recording**
+- Record a sample directly from system/app audio (any browser tab, VLC, Plex, ...) with no virtual audio cable — live level meter, waveform preview, click-to-assign to a pad
+
+**Effects rack**
+- Stack multiple effects (filter, distortion, bit crusher, chorus, delay, reverb) and render the whole chain through the sample in a single offline pass
+- Preview the chain live before committing; fixes the hardware's add-effect → resample → add-next-effect workflow
+
+**MIDI**
+- Web MIDI in/out with your MPC Sample hardware — pad hits in both directions, correct bank-shifted note mapping, no special hardware config beyond the device's own MIDI settings
+
+**Desktop app / SD card workflow**
+- Writes projects straight to the MPC SD card unzipped (`.xpj` + `_[ProjectData]/*.wav`), no zip/unzip round-trip required
+- Auto-detects the mounted SD card by contents (any volume name), with safe eject
+- Native open/save `.xpj` dialogs, auto-open export folder
+
+---
+
 ## Requirements
 
 - Node.js >= 18

TODO.md

@@ -0,0 +1,45 @@
+# mpcsample fork — TODO
+
+Working branch: `feature/sample-library-browser` (also carries the MIDI work below — not yet split out).
+
+## Done
+
+- **Effects Rack (offline multi-effect chain)** (2026-10-02) — the hardware forces add-effect → resample → add-next-effect for every stacked effect, and firmware research turned up no MIDI/USB control surface over the effects engine to automate that loop remotely (see `~/dev/mpc-firmware-research/FINDINGS.md`: the device's richer `/api/v1` object-model API is real but gated behind on-device PIN pairing we can't complete from a connected computer). Worked around it in-app instead: `src/audio/effects.types.ts` (plain, serializable `EffectInstance` chain — filter/distortion/bitcrusher/chorus/delay/reverb) + `src/audio/effectsChain.ts` (`renderEffectsChain` builds the whole chain as Tone.js nodes inside one `Tone.Offline` pass and returns a finished `AudioBuffer`; `previewEffectsChain` shares the same node-building code on the realtime context so you can audition before committing). New `EffectsRackPanel.tsx`, opened via an "Effects…" button in `KitEditor`'s per-pad editor (`mpc:open-effects` event, mirrors the other `mpc:open-*` panels). Render re-encodes to WAV (`encodeWavPCM16`) and writes back onto the pad as a new `userSamples` entry (fresh `sampleId`, so no stale-buffer caching risk) — stack further chains by rendering again.
+
+
+- Sample Library Browser: scan/browse/preview/drag-to-pad for large local sample folders, offloaded to the Electron main process so it never blocks on huge libraries.
+- Sample tagging: pack facet (derived from folder structure) + auto-tag taxonomy from folder/filename keywords + manual tag editor, persisted separately from the scan cache.
+- Fixed playback of WAV files with a trailing RIFF `LIST`/`INFO` metadata chunk (FL Studio export quirk) — `mpc-sample://` protocol now serves only the header-declared audio-data range, not the raw file size.
+- MIDI input fixed: MPC Sample hardware shifts the outgoing note number by 16 per physical pad bank (Bank A = 36–51, Bank B = 52–67, ...) — the app previously only accepted notes 36–51, silently dropping every bank past A. Now derives the global pad index directly from the note number, and the on-screen bank follows the hardware's active bank automatically.
+  - Real hardware settings required (SHIFT + PAD 8 → MIDI Configuration menu): **MIDI Port = USB**, **Pad MIDI Out = Always**.
+
+- MIDI *output*, verified working end-to-end on real hardware: UI-triggered pad hits (on-screen click, keyboard shortcuts) send Note On/Off back out over USB MIDI to every connected output — implemented in `MidiInput` (now also `MidiOutputLike`), wired through `store.midiOutRef`/`setMidiOut`. `triggerPad`/`releasePad` take a `fromHardware` flag so hardware-originated triggers (from `useMidiInput`) don't get echoed back to the device. Requires **Pad MIDI In = On** on the MPC Sample (SHIFT + PAD 8 → MIDI Configuration).
+
+- Explored `mpc-sample.local` (the firmware-update page served over USB) — confirmed it's just the firmware updater with no additional control surface beyond MIDI worth building on for this product. Detailed research notes (including a firmware-level security review) are kept in a local, uncommitted file, not in this repo.
+- **Step sequencer (v1, live/MIDI-driven)** — shipped 2026-09-07. `src/sequencer/` (`sequencer.types.ts`, `sequencerStore.ts`, `PatternPlayer.ts`) + `src/components/StepSequencer.tsx`, opened via the repurposed "SEQ RECORD" button. Configurable bars (1–8) and resolution (8th/16th/32nd). Plays via the existing hardware-style PLAY/STOP transport buttons, triggering pads through `engineRef`/`midiOutRef` directly (bypassing `triggerPad`'s side effects) per the `PatternPlayer` hook already left in `store.pressPad`/`unpressPad`'s doc comments. Does **not** export to native `.xpj` sequences (see "Planned" below).
+
+- Step sequencer follow-ups (2026-09-07): pattern now **persists to localStorage** (`mpc.sequencerPattern`, survives reload — see `sequencerStore.ts`'s `readPersistedPattern`/`writePersistedPattern` + a `subscribe()` that writes on every pattern change, transient `isPlaying`/`currentStep` excluded). Added **per-step velocity editing**: scroll up/down on an active step cell to raise/lower its velocity (0.1 increments, floor 0.1), with opacity scaling to show velocity visually.
+
+- **Native `.xpj` sequence export** (2026-09-07) — the step sequencer pattern now exports into the project's real, editable native sequence data (not just live/MIDI playback). Reverse-engineered the event schema from a real recorded `.xpj` (note events live at `sequences[].value.trackClipMaps[0][entryIdx].value.eventList.events`, keyed per-track, 960 ticks/beat, `note.note = padIdx + 36`, `note.velocity` 0..1 — see `src/xpj/buildSequence.ts`). Wired into the existing kit-export pipeline (`buildXpj`/`buildProjectArtifacts`/`exportKitToZip`/`exportKitToDisk`) via an optional `pattern` param, and into `ExportButton.tsx` as an "Include step sequencer pattern" checkbox (shown only when the current pattern has active steps). Writes into a reserved sequence slot (`key === 0`); one pattern at a time, no song-mode export.
+
+- **Draggable panels + hideable HUD** (2026-09-08) — Sample Browser and Step Sequencer can now be dragged by their header to anywhere on screen (`src/hooks/useDraggablePanel.ts`; pointer-drag, matching the existing `Knob`/`PitchFader` idiom), double-click the header to reset to the docked position. Position isn't persisted across reload and dragging is disabled below the 480px mobile breakpoint. KitEditor intentionally left undragged. The bottom-left HUD (keyboard shortcuts/bank/MIDI status/toolbar) can now be hidden via a "HIDE" button, leaving a small always-visible restore tab in its place; `hudVisible` persists to `localStorage` alongside the other settings in `src/state/store.ts`.
+
+- **Record a sample from system/app audio** (2026-09-09) — no virtual audio cable needed; captures real macOS system-audio loopback (whatever's currently playing through the Mac's output — a browser tab, VLC, Plex, ...). New `src/components/SampleRecorder.tsx` panel (opened via the repurposed "SAMPLE RECORD" button, reuses `useDraggablePanel`): click Record, it captures immediately (no picker dialog — see below for why), shows a live level meter + elapsed time, then on Stop shows a waveform preview + native playback control and a 4×4 pad-picker grid to assign the take (v1 is click-to-assign, not drag-and-drop). Recording taps a dedicated native `AudioContext` (NOT `Tone.getContext().rawContext` — that's actually Tone's `standardized-audio-context` wrapper, which doesn't implement `createScriptProcessor`) via a `ScriptProcessorNode` routed through a muted gain node to avoid echoing captured audio back out, encoded to 16-bit PCM WAV by a new hand-rolled encoder (`src/audio/encodeWav.ts`, mirrors the existing hand-rolled WAV parsing in `electron/sampleLibrary.ts`).
+  - **Three real bugs found and fixed while getting this working, in order:**
+    1. `NotAllowedError: Permission denied` on every attempt even with Screen Recording granted in System Settings — Chromium checks a *separate* `"media"` permission (not just `"display-capture"`) to actually grant use of the resulting audio track; only `"display-capture"` had been allow-listed. Root-caused by attaching Chrome DevTools Protocol directly to the packaged app (`--remote-debugging-port`) and logging every permission string Electron's handlers actually saw. Fixed in `electron/main.ts`'s `isAllowedPermission`.
+    2. Once permission passed, macOS's native picker (`useSystemPicker: true`) appeared but every capture had zero audio tracks — this is a confirmed, still-open Electron bug (electron/electron#44685): the native picker never actually returns audio on macOS in this Electron version. Fixed by dropping `useSystemPicker` entirely and granting `audio: "loopback"` directly in `handleDisplayMediaRequest` (no more per-window picker UI — capture is now always system-wide).
+    3. With direct loopback granted, the audio track came back *looking* successful but was actually dead (`readyState: "ended"`, real audio playing, zero signal, no error at all) — root-caused against electron/electron#49607: recent Chromium defaults to Apple's new CoreAudio Tap API for macOS loopback capture, which needs a completely different, newly-introduced Info.plist key (`NSAudioCaptureUsageDescription`) that almost nothing has yet, and silently fails with **no fallback** to the old, already-working method when it's missing. Fixed by explicitly disabling that new path (`app.commandLine.appendSwitch("disable-features", "MacCatapLoopbackAudioForScreenShare")`, alongside `MacSckSystemAudioLoopbackOverride` to force the old ScreenCaptureKit path) — confirmed with real captured audio (`maxRms` well above silence, `readyState: "live"`) via CDP, and end-to-end through the actual UI (record → stop → waveform preview → pad assignment → sample plays from that pad).
+    4. Audio was capturing but sounded "horrible" — root-caused via `track.getSettings()`: Chromium's `audio: true` shorthand defaults `echoCancellation`/`noiseSuppression`/`autoGainControl` all to `true` and `channelCount` to mono, even for system-audio loopback. Those are voice-call DSP tuned for mic input, not music/video — applying them to loopback audio genuinely mangles it (confirmed numerically: a clean 440Hz test tone measured 431.5Hz with amplitude/RMS drift and 2 waveform discontinuities *with* the defaults on; disabling them and requesting `channelCount: 2` measured an exact 440.0Hz with a razor-flat RMS envelope and zero discontinuities). Fixed in `SampleRecorder.ts`'s `start()` — explicit `audio: {echoCancellation: false, noiseSuppression: false, autoGainControl: false, channelCount: 2}`.
+  - Needs `NSScreenCaptureUsageDescription` (the permission actually used) and `NSAudioCaptureUsageDescription` (defensive, matches Electron's own now-recommended default) in the packaged app's Info.plist (`electron-builder.yml`'s `mac.extendInfo`).
+  - **Known tradeoff:** since there's no more picker, capture is always system-wide — make sure only the thing you want recorded is actually playing.
+
+## Planned
+
+- Sample-accurate MIDI-out timing for the sequencer (v1 sends MIDI immediately per step rather than clock-aligned — a few ms of jitter, documented as a known v1 limitation).
+- Multiple saved patterns / song mode (currently one in-memory-and-persisted pattern at a time).
+- Swing/groove control.
+- Drag-and-drop pad assignment for recorded samples (v1 is click-to-assign only).
+
+## Reference
+
+- MPC Sample User Guide (Akai, v1.0): MIDI Configuration menu is on page 60. Firmware/web-updater section on page 4 (`mpc-sample.local`, fallback `192.168.155.1`).

electron-builder.yml

@@ -35,6 +35,14 @@ mac:
     - zip
   category: public.app-category.music
   identity: null
+  extendInfo:
+    NSScreenCaptureUsageDescription: "MPC Sample needs screen-recording access to capture audio from other apps (e.g. a browser tab, VLC, or Plex) when recording a sample."
+    # Recent Chromium's macOS audio-loopback capture also checks this key (a
+    # separate, newer permission from screen recording, tied to Apple's
+    # CoreAudio Tap API) — added defensively even though `disable-features:
+    # MacCatapLoopbackAudioForScreenShare` in electron/main.ts currently keeps
+    # capture on the older, NSScreenCaptureUsageDescription-gated path.
+    NSAudioCaptureUsageDescription: "MPC Sample needs audio-capture access to record audio from other apps (e.g. a browser tab, VLC, or Plex) when recording a sample."
 
 dmg:
   artifactName: ${productName}-${version}.${ext}