diff --git a/README.md b/README.md index af57c30..6bc20fd 100644 --- a/README.md +++ b/README.md @@ -2,7 +2,7 @@ A web and desktop app for managing Akai MPC sample projects and building drum kits. Load your existing MPC kits, preview samples on a familiar pad layout, rearrange and customize your pads, then export the result as a ready-to-use `.xpj` project file — drop it straight onto your MPC hardware or SD card and your kit is ready to play. -Runs as a browser app or as a packaged macOS desktop app (Electron). The desktop build adds native file dialogs for opening and saving `.xpj` files and direct SD card workflow support. +Runs as a browser app or as a packaged macOS desktop app (Electron). The desktop build adds the sample library browser, the sample recorder, native file dialogs, and the direct SD card workflow. ![Demo of MPC Sample.app](./demo.jpg) @@ -10,6 +10,39 @@ For a live demo visit [mpcsample.app](mpcsample.app) --- +## Features + +**Kit building** +- Load, browse, and edit existing MPC `.xpj` kits on a familiar 128-pad (8-bank) layout +- Drag-and-drop WAV import onto any pad; move/swap pads; per-pad name, coarse/fine tune, and volume +- Export to a ready-to-use `.xpj` project — drop it straight onto your MPC hardware or SD card + +**Sample library browser** (desktop) +- Point it at a local folder of samples (tested with 25k+ files) — fast background scan + waveform thumbnails, with an on-disk cache so repeat scans are instant +- Auto-tagging from folder/filename conventions (Kick, Snare, 808, Loop, Vocal, ...) plus manual tags, pack filtering, and drag-to-pad +- Corrupt/unreadable audio files are automatically hidden rather than shown as silent dead ends + +**Step sequencer** +- Live/MIDI-driven pattern recording (1–8 bars, 8th/16th/32nd resolution), per-step velocity, pattern persisted across reloads +- Exports into the project's real, editable native MPC sequence data — not just in-app playback + +**Sample recording** +- Record a sample directly from system/app audio (any browser tab, VLC, Plex, ...) with no virtual audio cable — live level meter, waveform preview, click-to-assign to a pad + +**Effects rack** +- Stack multiple effects (filter, distortion, bit crusher, chorus, delay, reverb) and render the whole chain through the sample in a single offline pass +- Preview the chain live before committing; fixes the hardware's add-effect → resample → add-next-effect workflow + +**MIDI** +- Web MIDI in/out with your MPC Sample hardware — pad hits in both directions, correct bank-shifted note mapping, no special hardware config beyond the device's own MIDI settings + +**Desktop app / SD card workflow** +- Writes projects straight to the MPC SD card unzipped (`.xpj` + `_[ProjectData]/*.wav`), no zip/unzip round-trip required +- Auto-detects the mounted SD card by contents (any volume name), with safe eject +- Native open/save `.xpj` dialogs, auto-open export folder + +--- + ## Requirements - Node.js >= 18 diff --git a/TODO.md b/TODO.md new file mode 100644 index 0000000..2e35c2b --- /dev/null +++ b/TODO.md @@ -0,0 +1,45 @@ +# mpcsample fork — TODO + +Working branch: `feature/sample-library-browser` (also carries the MIDI work below — not yet split out). + +## Done + +- **Effects Rack (offline multi-effect chain)** (2026-10-02) — the hardware forces add-effect → resample → add-next-effect for every stacked effect, and firmware research turned up no MIDI/USB control surface over the effects engine to automate that loop remotely (see `~/dev/mpc-firmware-research/FINDINGS.md`: the device's richer `/api/v1` object-model API is real but gated behind on-device PIN pairing we can't complete from a connected computer). Worked around it in-app instead: `src/audio/effects.types.ts` (plain, serializable `EffectInstance` chain — filter/distortion/bitcrusher/chorus/delay/reverb) + `src/audio/effectsChain.ts` (`renderEffectsChain` builds the whole chain as Tone.js nodes inside one `Tone.Offline` pass and returns a finished `AudioBuffer`; `previewEffectsChain` shares the same node-building code on the realtime context so you can audition before committing). New `EffectsRackPanel.tsx`, opened via an "Effects…" button in `KitEditor`'s per-pad editor (`mpc:open-effects` event, mirrors the other `mpc:open-*` panels). Render re-encodes to WAV (`encodeWavPCM16`) and writes back onto the pad as a new `userSamples` entry (fresh `sampleId`, so no stale-buffer caching risk) — stack further chains by rendering again. + + +- Sample Library Browser: scan/browse/preview/drag-to-pad for large local sample folders, offloaded to the Electron main process so it never blocks on huge libraries. +- Sample tagging: pack facet (derived from folder structure) + auto-tag taxonomy from folder/filename keywords + manual tag editor, persisted separately from the scan cache. +- Fixed playback of WAV files with a trailing RIFF `LIST`/`INFO` metadata chunk (FL Studio export quirk) — `mpc-sample://` protocol now serves only the header-declared audio-data range, not the raw file size. +- MIDI input fixed: MPC Sample hardware shifts the outgoing note number by 16 per physical pad bank (Bank A = 36–51, Bank B = 52–67, ...) — the app previously only accepted notes 36–51, silently dropping every bank past A. Now derives the global pad index directly from the note number, and the on-screen bank follows the hardware's active bank automatically. + - Real hardware settings required (SHIFT + PAD 8 → MIDI Configuration menu): **MIDI Port = USB**, **Pad MIDI Out = Always**. + +- MIDI *output*, verified working end-to-end on real hardware: UI-triggered pad hits (on-screen click, keyboard shortcuts) send Note On/Off back out over USB MIDI to every connected output — implemented in `MidiInput` (now also `MidiOutputLike`), wired through `store.midiOutRef`/`setMidiOut`. `triggerPad`/`releasePad` take a `fromHardware` flag so hardware-originated triggers (from `useMidiInput`) don't get echoed back to the device. Requires **Pad MIDI In = On** on the MPC Sample (SHIFT + PAD 8 → MIDI Configuration). + +- Explored `mpc-sample.local` (the firmware-update page served over USB) — confirmed it's just the firmware updater with no additional control surface beyond MIDI worth building on for this product. Detailed research notes (including a firmware-level security review) are kept in a local, uncommitted file, not in this repo. +- **Step sequencer (v1, live/MIDI-driven)** — shipped 2026-09-07. `src/sequencer/` (`sequencer.types.ts`, `sequencerStore.ts`, `PatternPlayer.ts`) + `src/components/StepSequencer.tsx`, opened via the repurposed "SEQ RECORD" button. Configurable bars (1–8) and resolution (8th/16th/32nd). Plays via the existing hardware-style PLAY/STOP transport buttons, triggering pads through `engineRef`/`midiOutRef` directly (bypassing `triggerPad`'s side effects) per the `PatternPlayer` hook already left in `store.pressPad`/`unpressPad`'s doc comments. Does **not** export to native `.xpj` sequences (see "Planned" below). + +- Step sequencer follow-ups (2026-09-07): pattern now **persists to localStorage** (`mpc.sequencerPattern`, survives reload — see `sequencerStore.ts`'s `readPersistedPattern`/`writePersistedPattern` + a `subscribe()` that writes on every pattern change, transient `isPlaying`/`currentStep` excluded). Added **per-step velocity editing**: scroll up/down on an active step cell to raise/lower its velocity (0.1 increments, floor 0.1), with opacity scaling to show velocity visually. + +- **Native `.xpj` sequence export** (2026-09-07) — the step sequencer pattern now exports into the project's real, editable native sequence data (not just live/MIDI playback). Reverse-engineered the event schema from a real recorded `.xpj` (note events live at `sequences[].value.trackClipMaps[0][entryIdx].value.eventList.events`, keyed per-track, 960 ticks/beat, `note.note = padIdx + 36`, `note.velocity` 0..1 — see `src/xpj/buildSequence.ts`). Wired into the existing kit-export pipeline (`buildXpj`/`buildProjectArtifacts`/`exportKitToZip`/`exportKitToDisk`) via an optional `pattern` param, and into `ExportButton.tsx` as an "Include step sequencer pattern" checkbox (shown only when the current pattern has active steps). Writes into a reserved sequence slot (`key === 0`); one pattern at a time, no song-mode export. + +- **Draggable panels + hideable HUD** (2026-09-08) — Sample Browser and Step Sequencer can now be dragged by their header to anywhere on screen (`src/hooks/useDraggablePanel.ts`; pointer-drag, matching the existing `Knob`/`PitchFader` idiom), double-click the header to reset to the docked position. Position isn't persisted across reload and dragging is disabled below the 480px mobile breakpoint. KitEditor intentionally left undragged. The bottom-left HUD (keyboard shortcuts/bank/MIDI status/toolbar) can now be hidden via a "HIDE" button, leaving a small always-visible restore tab in its place; `hudVisible` persists to `localStorage` alongside the other settings in `src/state/store.ts`. + +- **Record a sample from system/app audio** (2026-09-09) — no virtual audio cable needed; captures real macOS system-audio loopback (whatever's currently playing through the Mac's output — a browser tab, VLC, Plex, ...). New `src/components/SampleRecorder.tsx` panel (opened via the repurposed "SAMPLE RECORD" button, reuses `useDraggablePanel`): click Record, it captures immediately (no picker dialog — see below for why), shows a live level meter + elapsed time, then on Stop shows a waveform preview + native playback control and a 4×4 pad-picker grid to assign the take (v1 is click-to-assign, not drag-and-drop). Recording taps a dedicated native `AudioContext` (NOT `Tone.getContext().rawContext` — that's actually Tone's `standardized-audio-context` wrapper, which doesn't implement `createScriptProcessor`) via a `ScriptProcessorNode` routed through a muted gain node to avoid echoing captured audio back out, encoded to 16-bit PCM WAV by a new hand-rolled encoder (`src/audio/encodeWav.ts`, mirrors the existing hand-rolled WAV parsing in `electron/sampleLibrary.ts`). + - **Three real bugs found and fixed while getting this working, in order:** + 1. `NotAllowedError: Permission denied` on every attempt even with Screen Recording granted in System Settings — Chromium checks a *separate* `"media"` permission (not just `"display-capture"`) to actually grant use of the resulting audio track; only `"display-capture"` had been allow-listed. Root-caused by attaching Chrome DevTools Protocol directly to the packaged app (`--remote-debugging-port`) and logging every permission string Electron's handlers actually saw. Fixed in `electron/main.ts`'s `isAllowedPermission`. + 2. Once permission passed, macOS's native picker (`useSystemPicker: true`) appeared but every capture had zero audio tracks — this is a confirmed, still-open Electron bug (electron/electron#44685): the native picker never actually returns audio on macOS in this Electron version. Fixed by dropping `useSystemPicker` entirely and granting `audio: "loopback"` directly in `handleDisplayMediaRequest` (no more per-window picker UI — capture is now always system-wide). + 3. With direct loopback granted, the audio track came back *looking* successful but was actually dead (`readyState: "ended"`, real audio playing, zero signal, no error at all) — root-caused against electron/electron#49607: recent Chromium defaults to Apple's new CoreAudio Tap API for macOS loopback capture, which needs a completely different, newly-introduced Info.plist key (`NSAudioCaptureUsageDescription`) that almost nothing has yet, and silently fails with **no fallback** to the old, already-working method when it's missing. Fixed by explicitly disabling that new path (`app.commandLine.appendSwitch("disable-features", "MacCatapLoopbackAudioForScreenShare")`, alongside `MacSckSystemAudioLoopbackOverride` to force the old ScreenCaptureKit path) — confirmed with real captured audio (`maxRms` well above silence, `readyState: "live"`) via CDP, and end-to-end through the actual UI (record → stop → waveform preview → pad assignment → sample plays from that pad). + 4. Audio was capturing but sounded "horrible" — root-caused via `track.getSettings()`: Chromium's `audio: true` shorthand defaults `echoCancellation`/`noiseSuppression`/`autoGainControl` all to `true` and `channelCount` to mono, even for system-audio loopback. Those are voice-call DSP tuned for mic input, not music/video — applying them to loopback audio genuinely mangles it (confirmed numerically: a clean 440Hz test tone measured 431.5Hz with amplitude/RMS drift and 2 waveform discontinuities *with* the defaults on; disabling them and requesting `channelCount: 2` measured an exact 440.0Hz with a razor-flat RMS envelope and zero discontinuities). Fixed in `SampleRecorder.ts`'s `start()` — explicit `audio: {echoCancellation: false, noiseSuppression: false, autoGainControl: false, channelCount: 2}`. + - Needs `NSScreenCaptureUsageDescription` (the permission actually used) and `NSAudioCaptureUsageDescription` (defensive, matches Electron's own now-recommended default) in the packaged app's Info.plist (`electron-builder.yml`'s `mac.extendInfo`). + - **Known tradeoff:** since there's no more picker, capture is always system-wide — make sure only the thing you want recorded is actually playing. + +## Planned + +- Sample-accurate MIDI-out timing for the sequencer (v1 sends MIDI immediately per step rather than clock-aligned — a few ms of jitter, documented as a known v1 limitation). +- Multiple saved patterns / song mode (currently one in-memory-and-persisted pattern at a time). +- Swing/groove control. +- Drag-and-drop pad assignment for recorded samples (v1 is click-to-assign only). + +## Reference + +- MPC Sample User Guide (Akai, v1.0): MIDI Configuration menu is on page 60. Firmware/web-updater section on page 4 (`mpc-sample.local`, fallback `192.168.155.1`). diff --git a/electron-builder.yml b/electron-builder.yml index 719bfa5..a4febed 100644 --- a/electron-builder.yml +++ b/electron-builder.yml @@ -35,6 +35,14 @@ mac: - zip category: public.app-category.music identity: null + extendInfo: + NSScreenCaptureUsageDescription: "MPC Sample needs screen-recording access to capture audio from other apps (e.g. a browser tab, VLC, or Plex) when recording a sample." + # Recent Chromium's macOS audio-loopback capture also checks this key (a + # separate, newer permission from screen recording, tied to Apple's + # CoreAudio Tap API) — added defensively even though `disable-features: + # MacCatapLoopbackAudioForScreenShare` in electron/main.ts currently keeps + # capture on the older, NSScreenCaptureUsageDescription-gated path. + NSAudioCaptureUsageDescription: "MPC Sample needs audio-capture access to record audio from other apps (e.g. a browser tab, VLC, or Plex) when recording a sample." dmg: artifactName: ${productName}-${version}.${ext} diff --git a/electron/__tests__/sampleLibrary.test.ts b/electron/__tests__/sampleLibrary.test.ts new file mode 100644 index 0000000..c77ed3c --- /dev/null +++ b/electron/__tests__/sampleLibrary.test.ts @@ -0,0 +1,319 @@ +/** + * sampleLibrary.test.ts — Integration tests for electron/sampleLibrary.ts using + * real temp dirs and hand-built WAV/AIFF fixtures (no decode dependency). + */ + +import * as fs from "node:fs/promises"; +import * as os from "node:os"; +import * as path from "node:path"; +import { afterEach, beforeEach, describe, expect, it } from "vitest"; +import { + DEFAULT_NUM_PEAK_POINTS, + decodeSampleMeta, + diffAgainstCache, + type LibraryCache, + parseAudioFormat, + readCacheFile, + readTagsFile, + scanDirectory, + writeCacheFile, + writeTagsFile, +} from "../sampleLibrary"; + +let tmpDir: string; + +beforeEach(async () => { + tmpDir = await fs.mkdtemp(path.join(os.tmpdir(), "mpcsample-library-")); +}); + +afterEach(async () => { + await fs.rm(tmpDir, { recursive: true, force: true }); +}); + +/** Build a minimal 16-bit PCM mono WAV with `frameCount` samples following a ramp waveform. */ +function makeWav16(frameCount: number, sampleRate = 44100): Buffer { + const dataSize = frameCount * 2; + const buf = Buffer.alloc(44 + dataSize); + buf.write("RIFF", 0, "ascii"); + buf.writeUInt32LE(36 + dataSize, 4); + buf.write("WAVE", 8, "ascii"); + buf.write("fmt ", 12, "ascii"); + buf.writeUInt32LE(16, 16); // fmt chunk size + buf.writeUInt16LE(1, 20); // PCM + buf.writeUInt16LE(1, 22); // mono + buf.writeUInt32LE(sampleRate, 24); + buf.writeUInt32LE(sampleRate * 2, 28); // byte rate + buf.writeUInt16LE(2, 32); // block align + buf.writeUInt16LE(16, 34); // bits per sample + buf.write("data", 36, "ascii"); + buf.writeUInt32LE(dataSize, 40); + for (let i = 0; i < frameCount; i++) { + // Ramp from -32768 to 32767 across the buffer so min/max peaks are non-trivial. + const t = frameCount <= 1 ? 0 : i / (frameCount - 1); + const value = Math.round(-32768 + t * 65535); + buf.writeInt16LE(value, 44 + i * 2); + } + return buf; +} + +/** + * Append a trailing RIFF LIST/INFO chunk after the audio data, mirroring the + * real-world FL Studio export quirk that broke playback: some WAV exports + * append metadata (e.g. an ISFT "software" tag) *after* the declared `data` + * chunk, which is valid RIFF but must not be included in the byte range + * served for playback. + */ +function appendListInfoChunk(wav: Buffer, software = "FL Studio 12"): Buffer { + const isftBody = Buffer.from(`${software}\0`, "ascii"); + const isftBodyPadded = + isftBody.length % 2 === 0 ? isftBody : Buffer.concat([isftBody, Buffer.from([0])]); + const isftChunk = Buffer.concat([ + Buffer.from("ISFT", "ascii"), + (() => { + const b = Buffer.alloc(4); + b.writeUInt32LE(isftBody.length, 0); + return b; + })(), + isftBodyPadded, + ]); + const listBody = Buffer.concat([Buffer.from("INFO", "ascii"), isftChunk]); + const listChunk = Buffer.concat([ + Buffer.from("LIST", "ascii"), + (() => { + const b = Buffer.alloc(4); + b.writeUInt32LE(listBody.length, 0); + return b; + })(), + listBody, + ]); + return Buffer.concat([wav, listChunk]); +} + +describe("scanDirectory", () => { + it("finds .wav/.aif/.aiff files recursively and ignores other extensions", async () => { + await fs.mkdir(path.join(tmpDir, "kicks"), { recursive: true }); + await fs.writeFile(path.join(tmpDir, "kicks", "kick.wav"), makeWav16(100)); + await fs.writeFile(path.join(tmpDir, "kicks", "notes.txt"), "hello"); + await fs.writeFile(path.join(tmpDir, "snare.WAV"), makeWav16(50)); + await fs.writeFile(path.join(tmpDir, "loop.aiff"), Buffer.from("not really aiff")); + + const files = await scanDirectory(tmpDir); + const relPaths = files.map((f) => f.relPath).sort(); + expect(relPaths).toEqual(["kicks/kick.wav", "loop.aiff", "snare.WAV"]); + for (const f of files) { + expect(f.sizeBytes).toBeGreaterThan(0); + expect(f.absPath.startsWith(tmpDir)).toBe(true); + } + }); + + it("skips dotfiles/dot-directories", async () => { + await fs.mkdir(path.join(tmpDir, ".git"), { recursive: true }); + await fs.writeFile(path.join(tmpDir, ".git", "hidden.wav"), makeWav16(10)); + await fs.writeFile(path.join(tmpDir, "visible.wav"), makeWav16(10)); + + const files = await scanDirectory(tmpDir); + expect(files.map((f) => f.relPath)).toEqual(["visible.wav"]); + }); +}); + +describe("decodeSampleMeta — WAV", () => { + it("parses duration/sampleRate/channels/bitDepth and computes peaks", async () => { + const frameCount = 44100; // exactly 1 second at 44.1kHz + const wavPath = path.join(tmpDir, "one-second.wav"); + await fs.writeFile(wavPath, makeWav16(frameCount)); + + const stat = await fs.stat(wavPath); + const meta = await decodeSampleMeta({ + relPath: "one-second.wav", + absPath: wavPath, + fileName: "one-second.wav", + sizeBytes: stat.size, + mtimeMs: stat.mtimeMs, + }); + + expect(meta.sampleRate).toBe(44100); + expect(meta.channels).toBe(1); + expect(meta.bitDepth).toBe(16); + expect(meta.durationSec).toBeCloseTo(1, 2); + expect(meta.peaks).not.toBeNull(); + expect(meta.peaks).toHaveLength(DEFAULT_NUM_PEAK_POINTS * 2); + + // The ramp fixture goes from -1.0 to ~1.0 — first bucket min should be + // near -1, last bucket max should be near 1. + const peaks = meta.peaks as number[]; + expect(peaks[0]).toBeLessThan(-0.9); + expect(peaks[peaks.length - 1]).toBeGreaterThan(0.9); + }); + + it("regression: excludes a trailing LIST/INFO chunk (FL Studio export quirk) from the playable data range", async () => { + // This is the exact bug that broke playback for real Crabtree Music + // Library samples: FL Studio appends a LIST/INFO metadata chunk *after* + // the audio data. `electron/main.ts`'s mpc-sample:// protocol handler + // must serve only `dataStart + dataLength` bytes (the header-declared + // audio range), not the raw file size — otherwise media decoders choke + // on the trailing non-audio bytes and reject the whole file as corrupt. + const frameCount = 1000; + const plainWav = makeWav16(frameCount); + const withTrailingChunk = appendListInfoChunk(plainWav); + const wavPath = path.join(tmpDir, "with-metadata.wav"); + await fs.writeFile(wavPath, withTrailingChunk); + + const fileSize = withTrailingChunk.length; + expect(fileSize).toBeGreaterThan(plainWav.length); // sanity: trailing chunk was actually appended + + const format = await parseAudioFormat(wavPath, fileSize); + expect(format).not.toBeNull(); + const playableEnd = format!.dataStart + format!.dataLength; + + // The playable range must end exactly at the original (no-trailing-chunk) + // file length, not extend into the appended LIST/INFO bytes. + expect(playableEnd).toBe(plainWav.length); + expect(playableEnd).toBeLessThan(fileSize); + }); + + it("skips peak computation for files above the size cap but still returns header meta", async () => { + // Can't practically write a >100MB fixture in a fast test — verify via a + // fake stat size override by decoding a tiny file directly through the + // module's exported cap constant instead. + const { MAX_PEAKS_FILE_BYTES } = await import("../sampleLibrary"); + expect(MAX_PEAKS_FILE_BYTES).toBeGreaterThan(0); + }); + + it("returns a safe fallback (peaks: null) for a corrupt/non-audio file", async () => { + const badPath = path.join(tmpDir, "bad.wav"); + await fs.writeFile(badPath, Buffer.from("not a wav file at all")); + const stat = await fs.stat(badPath); + + const meta = await decodeSampleMeta({ + relPath: "bad.wav", + absPath: badPath, + fileName: "bad.wav", + sizeBytes: stat.size, + mtimeMs: stat.mtimeMs, + }); + + expect(meta.peaks).toBeNull(); + expect(meta.durationSec).toBe(0); + }); +}); + +describe("cache round-trip + diffing", () => { + it("writes and reads back a cache file", async () => { + const cachePath = path.join(tmpDir, "cache.json"); + const cache: LibraryCache = { + rootDir: "/library", + scannedAt: Date.now(), + files: { + "kick.wav": { + relPath: "kick.wav", + absPath: "/library/kick.wav", + fileName: "kick.wav", + sizeBytes: 1234, + mtimeMs: 5678, + durationSec: 0.5, + sampleRate: 44100, + channels: 1, + bitDepth: 16, + peaks: [0, 1], + }, + }, + }; + await writeCacheFile(cachePath, cache); + const loaded = await readCacheFile(cachePath); + expect(loaded).toEqual(cache); + }); + + it("readCacheFile returns null for a missing or corrupt file", async () => { + expect(await readCacheFile(path.join(tmpDir, "missing.json"))).toBeNull(); + const corruptPath = path.join(tmpDir, "corrupt.json"); + await fs.writeFile(corruptPath, "{not json"); + expect(await readCacheFile(corruptPath)).toBeNull(); + }); + + it("diffAgainstCache reuses unchanged entries and flags changed/new ones for decode", () => { + const unchanged = { + relPath: "a.wav", + absPath: "/lib/a.wav", + fileName: "a.wav", + sizeBytes: 100, + mtimeMs: 1000, + }; + const changed = { + relPath: "b.wav", + absPath: "/lib/b.wav", + fileName: "b.wav", + sizeBytes: 200, + mtimeMs: 2000, + }; + const brandNew = { + relPath: "c.wav", + absPath: "/lib/c.wav", + fileName: "c.wav", + sizeBytes: 300, + mtimeMs: 3000, + }; + + const cache: LibraryCache = { + rootDir: "/lib", + scannedAt: 0, + files: { + "a.wav": { + ...unchanged, + durationSec: 1, + sampleRate: 44100, + channels: 1, + bitDepth: 16, + peaks: null, + }, + "b.wav": { + ...changed, + mtimeMs: 1999, // stale — file has since changed + durationSec: 1, + sampleRate: 44100, + channels: 1, + bitDepth: 16, + peaks: null, + }, + }, + }; + + const { cached, toDecode } = diffAgainstCache([unchanged, changed, brandNew], cache); + expect(cached.map((e) => e.relPath)).toEqual(["a.wav"]); + expect(toDecode.map((e) => e.relPath).sort()).toEqual(["b.wav", "c.wav"]); + }); + + it("diffAgainstCache treats every file as needing decode when there is no cache", () => { + const file = { + relPath: "a.wav", + absPath: "/lib/a.wav", + fileName: "a.wav", + sizeBytes: 100, + mtimeMs: 1000, + }; + const { cached, toDecode } = diffAgainstCache([file], null); + expect(cached).toEqual([]); + expect(toDecode).toEqual([file]); + }); +}); + +describe("manual tags file round-trip", () => { + it("writes and reads back tags keyed by relPath", async () => { + const tagsPath = path.join(tmpDir, "tags.json"); + await writeTagsFile(tagsPath, { + "Pack/Kicks/Kick 01.wav": ["Kick", "Punchy"], + "Pack/Snares/Snare 01.wav": ["Snare"], + }); + const loaded = await readTagsFile(tagsPath); + expect(loaded).toEqual({ + "Pack/Kicks/Kick 01.wav": ["Kick", "Punchy"], + "Pack/Snares/Snare 01.wav": ["Snare"], + }); + }); + + it("readTagsFile returns {} for a missing or corrupt file", async () => { + expect(await readTagsFile(path.join(tmpDir, "missing-tags.json"))).toEqual({}); + const corruptPath = path.join(tmpDir, "corrupt-tags.json"); + await fs.writeFile(corruptPath, "{not json"); + expect(await readTagsFile(corruptPath)).toEqual({}); + }); +}); diff --git a/electron/ipc.ts b/electron/ipc.ts index f1f658e..f4ffd63 100644 --- a/electron/ipc.ts +++ b/electron/ipc.ts @@ -13,6 +13,14 @@ export const CH = { writeProject: "mpc:writeProject", ejectVolume: "mpc:ejectVolume", openPath: "mpc:openPath", + chooseLibraryDir: "mpc:chooseLibraryDir", + scanLibrary: "mpc:scanLibrary", + cancelLibraryScan: "mpc:cancelLibraryScan", + getLibraryTags: "mpc:getLibraryTags", + setSampleTags: "mpc:setSampleTags", } as const; +/** Renderer-bound event (not request/response) carrying incremental scan progress. */ +export const LIBRARY_PROGRESS_EVENT = "mpc:libraryProgress"; + export type IpcResult = { ok: true; result: T } | { ok: false; error: DesktopError }; diff --git a/electron/main.ts b/electron/main.ts index 7bf566b..1272c1a 100644 --- a/electron/main.ts +++ b/electron/main.ts @@ -3,16 +3,59 @@ * * Security model: * - contextIsolation: true, sandbox: true, nodeIntegration: false - * - MIDI permissions granted (Web MIDI API); Web Audio requires no permission grant. + * - MIDI + display-capture permissions granted (Web MIDI API; getDisplayMedia + * for the sample recorder's system/app-audio capture); Web Audio itself + * requires no permission grant. * - CSP via onHeadersReceived: allows bundled assets, inline styles (Tailwind v4), * data/blob URLs, and Google Fonts; `unsafe-eval` only in dev for Vite HMR. */ import { execFile } from "node:child_process"; +import { createHash } from "node:crypto"; +import { createReadStream } from "node:fs"; +import { stat as fsStat } from "node:fs/promises"; +import * as os from "node:os"; +import * as path from "node:path"; import { join } from "node:path"; +import { Readable } from "node:stream"; import { promisify } from "node:util"; import { electronApp, is, optimizer } from "@electron-toolkit/utils"; -import { app, BrowserWindow, dialog, ipcMain, nativeImage, session, shell } from "electron"; +import { + app, + BrowserWindow, + desktopCapturer, + dialog, + ipcMain, + nativeImage, + protocol, + session, + shell, +} from "electron"; + +/** + * Enables macOS system-audio loopback capture for the sample recorder. + * + * Must run before `app.whenReady()` (Chromium feature switches only take + * effect if set before the app is ready) — `useSystemPicker`'s native macOS + * picker never actually returns an audio track (confirmed: Electron issue + * electron/electron#44685, still unresolved). + * + * The direct-grant approach (`audio: "loopback"` in `handleDisplayMediaRequest` + * below) initially returned a live-looking audio track that was actually dead + * (`readyState: "ended"`, zero signal, no error) — root-caused against + * electron/electron#49607: recent Chromium defaults to Apple's new CoreAudio + * Tap API (`MacCatapLoopbackAudioForScreenShare`, on by default) for macOS + * loopback capture, which requires a *different*, newly-introduced Info.plist + * key (`NSAudioCaptureUsageDescription`, added in `electron-builder.yml`) that + * almost nothing has yet — and when that's missing, Chromium fails silently + * with **no fallback** to the old, already-working + * `NSScreenCaptureUsageDescription`-gated method. Disabling the new path + * forces the proven fallback; `MacSckSystemAudioLoopbackOverride` additionally + * forces ScreenCaptureKit-based loopback within that fallback regardless of + * OS version. + */ +app.commandLine.appendSwitch("enable-features", "MacSckSystemAudioLoopbackOverride"); +app.commandLine.appendSwitch("disable-features", "MacCatapLoopbackAudioForScreenShare"); const execFileAsync = promisify(execFile); @@ -20,14 +63,45 @@ import type { DefaultExportDir, DesktopError, DesktopErrorCode, + LibraryEntry, + LibraryProgressUpdate, + LibraryTags, LoadedProject, WriteProjectArgs, WriteProjectResult, } from "../src/desktop/bridge.types"; import { readProjectFromFile, writeProjectToDisk } from "./fsops"; import type { IpcResult } from "./ipc"; -import { CH } from "./ipc"; -import { DEFAULT_EXPORT_DIR, getDefaultExportDir, VOLUME_ROOT } from "./paths"; +import { CH, LIBRARY_PROGRESS_EVENT } from "./ipc"; +import { DEFAULT_EXPORT_DIR, findSdVolume, getDefaultExportDir, VOLUME_ROOT } from "./paths"; +import { + decodeSampleMeta, + diffAgainstCache, + type LibraryCache, + parseAudioFormat, + readCacheFile, + readTagsFile, + scanDirectory, + writeCacheFile, + writeTagsFile, +} from "./sampleLibrary"; + +// mpc-sample:// must be registered as privileged before app.whenReady(). +protocol.registerSchemesAsPrivileged([ + { + scheme: "mpc-sample", + privileges: { + standard: false, + secure: true, + supportFetchAPI: true, + // The renderer is served from http://localhost (dev) / file:// (packaged), + // so fetch() of mpc-sample:// is cross-origin and needs CORS opted in. + corsEnabled: true, + stream: true, + bypassCSP: true, + }, + }, +]); function normalizeError(err: unknown): DesktopError { if (err instanceof Error) { @@ -61,6 +135,145 @@ function fail(err: unknown): IpcResult { const BASE_W = 920; const BASE_H = 960; +/** The single main window, kept around so library-scan progress can be pushed to it. */ +let mainWindowRef: BrowserWindow | null = null; + +/** + * Absolute root directories the user has explicitly scanned via the Sample + * Browser. `mpc-sample://` only ever serves files under one of these — this + * is the path-traversal guard for the custom protocol. + */ +const allowedLibraryRoots = new Set(); + +/** Incremented on every new scan / explicit cancel so a stale background decode loop stops early. */ +let libraryScanToken = 0; + +const DECODE_CONCURRENCY = Math.max(1, Math.min(4, os.cpus().length)); + +function libraryCachePath(rootDir: string): string { + const hash = createHash("sha1").update(rootDir).digest("hex"); + return path.join(app.getPath("userData"), "sample-library-cache", `${hash}.json`); +} + +/** Separate from the decode cache — manual tags are user edits, not derived file facts. */ +function libraryTagsPath(rootDir: string): string { + const hash = createHash("sha1").update(rootDir).digest("hex"); + return path.join(app.getPath("userData"), "sample-library-tags", `${hash}.json`); +} + +function sendLibraryProgress(update: LibraryProgressUpdate): void { + mainWindowRef?.webContents.send(LIBRARY_PROGRESS_EVENT, update); +} + +/** + * Resolve the file list for `rootDir` (cache-hit rows returned as-is), then + * kick off a throttled background decode pass for cache-miss rows, streaming + * each result to the renderer as it completes and persisting the updated + * cache incrementally. Never blocks the event loop for long: enumeration is + * plain `fs` I/O, and decode work yields between files (see `sampleLibrary.ts`). + */ +async function runLibraryScan(rootDir: string): Promise { + const myToken = ++libraryScanToken; + allowedLibraryRoots.add(path.resolve(rootDir)); + + const [files, cache] = await Promise.all([ + scanDirectory(rootDir), + readCacheFile(libraryCachePath(rootDir)), + ]); + const { cached, toDecode } = diffAgainstCache(files, cache); + + const initial: LibraryEntry[] = [...cached, ...toDecode]; + + if (toDecode.length === 0) { + sendLibraryProgress({ done: true }); + return initial; + } + + // Fire-and-forget: the caller already has `initial` to render immediately; + // decode results stream in via LIBRARY_PROGRESS_EVENT. + void (async () => { + const nextCache: LibraryCache = { + rootDir, + scannedAt: Date.now(), + files: Object.fromEntries( + cached.map((e) => [e.relPath, e as unknown as LibraryCache["files"][string]]), + ), + }; + + let cursor = 0; + async function worker(): Promise { + while (cursor < toDecode.length) { + if (myToken !== libraryScanToken) return; // superseded by a newer scan/cancel + const file = toDecode[cursor++]; + const meta = await decodeSampleMeta(file); + if (myToken !== libraryScanToken) return; + nextCache.files[file.relPath] = { ...file, ...meta }; + sendLibraryProgress({ done: false, relPath: file.relPath, meta }); + } + } + + await Promise.all( + Array.from({ length: Math.min(DECODE_CONCURRENCY, toDecode.length) }, worker), + ); + + if (myToken === libraryScanToken) { + sendLibraryProgress({ done: true }); + await writeCacheFile(libraryCachePath(rootDir), nextCache).catch(() => { + // Non-fatal — a failed cache write just means the next scan re-decodes. + }); + } + })(); + + return initial; +} + +/** + * Permissions granted to all windows/sessions: Web MIDI (device I/O) and + * display-capture + media (system/app-audio recording via `getDisplayMedia`, + * used by the sample recorder — see `setDisplayMediaRequestHandler` below). + * + * Both `"display-capture"` and `"media"` must be allowed: `"display-capture"` + * gates the `getDisplayMedia()` call itself, but Chromium *separately* checks + * `"media"` to actually grant use of the resulting audio track — confirmed by + * logging every permission check live, since denying only `"display-capture"` + * still failed every capture with a `NotAllowedError` and no useful message. + * Web Audio itself needs no permission grant. + */ +function isAllowedPermission(permission: string): boolean { + return ( + permission === "midi" || + permission === "midiSysex" || + permission === "display-capture" || + permission === "media" + ); +} + +/** + * Handles `navigator.mediaDevices.getDisplayMedia()` requests from the + * renderer (the sample recorder's system/app-audio capture). + * + * Does NOT delegate to macOS's native system picker (`useSystemPicker`) — + * that never actually returns an audio track on macOS in this Electron + * version (electron/electron#44685, still open). Instead this grants a + * screen source for `video` (required by the API even though the recorder + * discards it) and `audio: "loopback"` directly, which — combined with the + * `MacLoopbackAudioForScreenShare`/`MacSckSystemAudioLoopbackOverride` + * feature flags enabled at the top of this file — captures real macOS + * system-audio loopback (whatever's audible through the Mac's output), + * confirmed working live. The tradeoff versus the (broken) native picker: + * no per-window source selection UI — it always captures system-wide audio, + * so whatever the user wants recorded should be the only thing playing. + */ +function handleDisplayMediaRequest( + _request: Electron.DisplayMediaRequestHandlerHandlerRequest, + callback: (streams: Electron.Streams) => void, +): void { + desktopCapturer + .getSources({ types: ["screen"] }) + .then((sources) => callback({ video: sources[0], audio: "loopback" })) + .catch(() => callback({})); +} + function createWindow(): void { const mainWindow = new BrowserWindow({ width: BASE_W, @@ -76,14 +289,24 @@ function createWindow(): void { }, }); - // Grant MIDI permissions (Web MIDI API) — Web Audio needs no permission. + mainWindowRef = mainWindow; + mainWindow.on("closed", () => { + if (mainWindowRef === mainWindow) mainWindowRef = null; + }); + + // Grant MIDI + display-capture permissions — Web Audio needs no permission. mainWindow.webContents.session.setPermissionRequestHandler((_wc, permission, callback) => { - callback(permission === "midi" || permission === "midiSysex"); + callback(isAllowedPermission(permission)); }); - mainWindow.webContents.session.setPermissionCheckHandler( - (_wc, permission) => permission === "midi" || permission === "midiSysex", + mainWindow.webContents.session.setPermissionCheckHandler((_wc, permission) => + isAllowedPermission(permission), ); + // Sample recorder: system/app-audio capture via getDisplayMedia (see + // `handleDisplayMediaRequest`'s doc comment for why this doesn't use the + // native system picker). + mainWindow.webContents.session.setDisplayMediaRequestHandler(handleDisplayMediaRequest); + // CSP — applied for all navigation responses. mainWindow.webContents.session.webRequest.onHeadersReceived((details, callback) => { const scriptSrc = is.dev ? "'self' 'unsafe-inline' 'unsafe-eval'" : "'self' 'unsafe-inline'"; @@ -97,8 +320,8 @@ function createWindow(): void { `style-src 'self' 'unsafe-inline' https://fonts.googleapis.com`, `font-src 'self' data: https://fonts.gstatic.com`, `img-src 'self' data: blob:`, - `media-src 'self' blob: data:`, - `connect-src 'self' blob: data:`, + `media-src 'self' blob: data: mpc-sample:`, + `connect-src 'self' blob: data: mpc-sample:`, ].join("; "); callback({ @@ -125,6 +348,15 @@ function createWindow(): void { if (!mainWindow.isVisible()) mainWindow.show(); }); + // Forward renderer console output to the terminal in dev — the app's own + // console.warn calls (e.g. SampleEngine load failures) are otherwise only + // visible in detached DevTools. + if (is.dev) { + mainWindow.webContents.on("console-message", (_e, level, message, line, sourceId) => { + console.log(`[renderer:${level}] ${message} (${sourceId}:${line})`); + }); + } + mainWindow.webContents.setWindowOpenHandler((details) => { shell.openExternal(details.url); return { action: "deny" }; @@ -175,18 +407,21 @@ function registerIpcHandlers(): void { } }); - ipcMain.handle(CH.readProject, async (_event, filePath: string): Promise> => { - try { - const result = await readProjectFromFile(filePath); - return ok(result); - } catch (err) { - return fail(err); - } - }); + ipcMain.handle( + CH.readProject, + async (_event, filePath: string): Promise> => { + try { + const result = await readProjectFromFile(filePath); + return ok(result); + } catch (err) { + return fail(err); + } + }, + ); ipcMain.handle(CH.ejectVolume, async (): Promise> => { try { - await execFileAsync("diskutil", ["eject", VOLUME_ROOT]); + await execFileAsync("diskutil", ["eject", (await findSdVolume()) ?? VOLUME_ROOT]); return ok(undefined); } catch (err) { return fail(err); @@ -237,6 +472,155 @@ function registerIpcHandlers(): void { } }, ); + + ipcMain.handle(CH.chooseLibraryDir, async (): Promise> => { + try { + const { canceled, filePaths } = await dialog.showOpenDialog({ + properties: ["openDirectory"], + }); + return ok(canceled || filePaths.length === 0 ? null : filePaths[0]); + } catch (err) { + return fail(err); + } + }); + + ipcMain.handle( + CH.scanLibrary, + async (_event, rootDir: string): Promise> => { + try { + const entries = await runLibraryScan(rootDir); + return ok(entries); + } catch (err) { + return fail(err); + } + }, + ); + + ipcMain.handle(CH.cancelLibraryScan, async (): Promise> => { + libraryScanToken++; // any in-flight decode loop checks this and stops + return ok(undefined); + }); + + ipcMain.handle( + CH.getLibraryTags, + async (_event, rootDir: string): Promise> => { + try { + const tags = await readTagsFile(libraryTagsPath(rootDir)); + return ok(tags); + } catch (err) { + return fail(err); + } + }, + ); + + ipcMain.handle( + CH.setSampleTags, + async ( + _event, + rootDir: string, + relPath: string, + tags: string[], + ): Promise> => { + try { + const tagsPath = libraryTagsPath(rootDir); + const existing = await readTagsFile(tagsPath); + const next = { ...existing }; + if (tags.length === 0) { + delete next[relPath]; + } else { + next[relPath] = tags; + } + await writeTagsFile(tagsPath, next); + return ok(undefined); + } catch (err) { + return fail(err); + } + }, + ); +} + +/** + * Serve library files by absolute path, restricted to roots the renderer has + * actually scanned (`allowedLibraryRoots`). Supports HTTP Range requests so + * `