1 Commits

Author SHA1 Message Date
vadimwit 3258060661 essentia additions: probably not good enough to add 10x app disk size 2026-03-09 22:44:15 +00:00
27 changed files with 443 additions and 5343 deletions
-5
View File
@@ -1,5 +0,0 @@
{
"projects": {
"default": "itsamejms"
}
}
+24 -60
View File
@@ -9,69 +9,33 @@ permissions:
contents: write contents: write
jobs: jobs:
build: build-windows:
runs-on: ${{ matrix.os }} runs-on: windows-latest
strategy:
fail-fast: false # Ensures one OS failing doesn't kill the others
matrix:
include:
- os: windows-latest
command: npm run electron:build:win
artifact_pattern: "release/*.exe"
- os: macos-latest
# Added --universal here if you want to support both Intel and Apple Silicon
command: npm run electron:build:mac -- --universal
artifact_pattern: "release/*.dmg"
- os: ubuntu-latest
command: npm run electron:build:linux
artifact_pattern: "release/*.AppImage"
env:
# This fixes the "GitHub Personal Access Token is not set" error
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
# Prevents errors related to missing code-signing certificates
CSC_IDENTITY_AUTO_DISCOVERY: false
steps: steps:
- uses: actions/checkout@v4 - uses: actions/checkout@v4
- uses: actions/setup-node@v4 - uses: actions/setup-node@v4
with: with:
node-version: 20 node-version: 20
cache: 'npm' - run: npm install
- run: npm run electron:build:win
- name: Install dependencies
run: npm install
- name: Build Application
run: ${{ matrix.command }}
- name: Upload Artifacts
uses: actions/upload-artifact@v4
with:
name: artifacts-${{ matrix.os }}
path: ${{ matrix.artifact_pattern }}
if-no-files-found: error
publish:
needs: build
runs-on: ubuntu-latest
steps:
- name: Download all artifacts
uses: actions/download-artifact@v4
with:
# This downloads all "artifacts-*" into a folder named 'all-outputs'
path: all-outputs
merge-multiple: true
- name: List files for debugging
run: ls -R all-outputs
- name: Create GitHub Release
uses: softprops/action-gh-release@v2
with:
# Point directly to the folder where all OS builds are merged
files: all-outputs/*
generate_release_notes: true
env: env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} CSC_IDENTITY_AUTO_DISCOVERY: false
WIN_CSC_LINK: ''
- uses: softprops/action-gh-release@v2
with:
files: releases/*.exe
build-mac:
runs-on: macos-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with:
node-version: 20
- run: npm install
- run: npm run electron:build:mac
env:
CSC_IDENTITY_AUTO_DISCOVERY: false
- uses: softprops/action-gh-release@v2
with:
files: releases/*.dmg
-5
View File
@@ -13,8 +13,3 @@ debug.log
releases/ releases/
release/ release/
.DS_Store .DS_Store
# Firebase
.firebase/
firebase-debug.log
ui-debug.log
-42
View File
@@ -70,42 +70,6 @@ npm run electron:build:linux
Output is placed in `frontend/release/`. Output is placed in `frontend/release/`.
## Releasing / Tagging
To create a GitHub release and trigger the CI build pipeline, create an annotated tag and push it to origin. The release workflow runs on tags matching `v*` (for example `v0.6.1`).
Local tagging example:
```bash
# update package.json version first if desired
git tag -a v0.6.1 -m "Release v0.6.1"
git push origin v0.6.1
```
What the GitHub Action does (`.github/workflows/release.yml`):
- Listens for pushed tags `v*` and runs a matrix build across Windows, macOS and Linux.
- macOS is built as a universal binary (`--universal`) so a single DMG supports both Intel and Apple Silicon.
- Each matrix job builds the installer using `electron-builder`, uploads its artifacts, and a final `publish` job aggregates all artifacts into one GitHub release.
If you prefer to run builds locally before tagging, use the npm scripts in the repository root:
```bash
# Windows NSIS
npm run electron:build:win
# macOS DMG (universal)
npm run electron:build:mac -- --universal
# Linux AppImage
npm run electron:build:linux
```
CI notes / troubleshooting
- The workflow uploads artifacts from `release/` into the release. Ensure `package.json` build `directories.output` matches the workflow's expected `release/` folder.
- If mac packaging for x64 on ARM-hosted runners fails, switch to `--universal` (already configured) or build x64 on an Intel runner.
- To test the workflow locally, consider using `nektos/act` or push a temporary tag like `vtest`.
## Design Tokens ## Design Tokens
All colors are defined in `frontend/tailwind.config.js` and can be referenced by name in any component. All colors are defined in `frontend/tailwind.config.js` and can be referenced by name in any component.
@@ -132,9 +96,3 @@ Audio is captured via the browser's Web Audio API and processed in two parallel
2. **Chord path** — 16384-sample FFT (2.7 Hz/bin) with harmonic summation chroma extraction across 804000 Hz. The averaged chroma vector is matched against chord templates (major, minor, dom7, min7, dim, half-dim, aug, sus4, add9) using a weighted coverage score. Consecutive identical detections are required before a chord is committed, preventing transient false positives. 2. **Chord path** — 16384-sample FFT (2.7 Hz/bin) with harmonic summation chroma extraction across 804000 Hz. The averaged chroma vector is matched against chord templates (major, minor, dom7, min7, dim, half-dim, aug, sus4, add9) using a weighted coverage score. Consecutive identical detections are required before a chord is committed, preventing transient false positives.
The top-3 key candidates are shown in real time as clickable chips. Locking a key in Beginner mode restricts chord matching to the 7 diatonic chords; Advanced mode allows chromatic/borrowed chords. The top-3 key candidates are shown in real time as clickable chips. Locking a key in Beginner mode restricts chord matching to the 7 diatonic chords; Advanced mode allows chromatic/borrowed chords.
## Deploy
- firebase login
- npm run build
- firebase deploy --only hosting:jambuddy
-307
View File
@@ -1,307 +0,0 @@
# Plan: PWA + Firebase Hosting for JamBuddy
Status: **proposed**. Each phase is independently shippable; stop after any one.
---
## 0. Why this is low-risk
The Electron shell (`electron/main.cjs`, `preload.cjs`) does **nothing but host a window** — no IPC, no native APIs, `sandbox:false` only to give the renderer full Web Audio. All logic lives in the React renderer, which already runs browser-only via `npm run dev`. So the web build is the same `vite build` output the desktop installer already ships; we just add a manifest + service worker and serve `dist/` over HTTPS. Firebase gives us HTTPS for free, which `getUserMedia` (the mic) requires — that's the only hard web constraint.
Nothing in the app makes network calls today, so there's no API surface to re-host and no secrets to manage.
---
## Phase 1 — PWA scaffolding (no Firebase yet)
Goal: `vite build` produces an installable, offline-capable web app served from any static host.
### 1.1 Add `vite-plugin-pwa`
Only new dependency. Wraps Workbox under a Vite plugin — generates the service worker and injects the manifest. Cheaper than hand-rolling a SW and gets update flow for free.
```js
// vite.config.js
import { VitePWA } from 'vite-plugin-pwa'
export default defineConfig({
plugins: [
react(),
tailwindcss(),
VitePWA({
registerType: 'autoUpdate',
includeAssets: ['favicon.ico', 'apple-touch-icon.png'],
manifest: {
name: 'JamBuddy — WhatTheFlat',
short_name: 'JamBuddy',
description: 'Real-time key and chord detection for live jams.',
theme_color: '#0f0f0f', // matches bg-surface token
background_color: '#0f0f0f',
display: 'standalone',
orientation: 'any',
start_url: '/',
scope: '/',
icons: [ /* see 1.3 */ ],
},
workbox: {
globPatterns: ['**/*.{js,css,html,svg,png,ico,woff2}'],
// Audio is captured live — nothing to cache from a remote. Precache the app shell only.
navigateFallback: '/index.html',
},
}),
],
})
```
### 1.2 Create `public/` directory
Vite serves `public/` at root and copies it verbatim into `dist/`. We need it for:
- `favicon.ico`, `apple-touch-icon.png` (180×180)
- Any static PWA icons referenced by the manifest
- (Optional) `robots.txt` — not needed for an app, skip.
The existing `src/assets/whattheflat-logo.png` is imported in `App.jsx` and bundled by Vite — leave that as-is; **don't** move it. The manifest icons must live in `public/` because they're referenced by URL, not imported.
### 1.3 Generate icon set from the existing logo
Derive all required sizes from `assets/whattheflat-logo.png` (one source image → PNG output). Required manifest entries:
```js
icons: [
{ src: '/icon-192.png', sizes: '192x192', type: 'image/png', purpose: 'any' },
{ src: '/icon-512.png', sizes: '512x512', type: 'image/png', purpose: 'any' },
{ src: '/icon-512-maskable.png', sizes: '512x512', type: 'image/png', purpose: 'maskable' },
]
```
Plus `apple-touch-icon.png` (180×180) in `public/` — iOS ignores the manifest and uses this `<link>`. The maskable variant needs safe-zone padding (~10%) so the logo isn't cropped by Android's circular mask.
One-liner with ImageMagick if available, else export manually from the source PNG:
```bash
magick assets/whattheflat-logo.png -resize 192x192 public/icon-192.png
magick assets/whattheflat-logo.png -resize 512x512 public/icon-512.png
magick assets/whattheflat-logo.png -resize 180x180 public/apple-touch-icon.png
```
### 1.4 Update `index.html`
Two problems with the current `<head>`:
1. **No PWA tags** — add theme-color, manifest link, apple-touch-icon, apple-mobile-web-app-capable, description.
2. **CSP is hardcoded for dev**`connect-src 'self' http://localhost:5173 ws://localhost:5173` and `script-src 'self' 'unsafe-eval'`. The `unsafe-eval` and the localhost entries are Vite dev artifacts; production Vite doesn't need `unsafe-eval`. Keep a CSP (it's good hygiene and the app is genuinely offline) but drop the dev bits:
```html
<meta name="theme-color" content="#0f0f0f" />
<meta name="description" content="Real-time key and chord detection for live jams." />
<link rel="manifest" href="/manifest.webmanifest" />
<link rel="apple-touch-icon" href="/apple-touch-icon.png" />
<meta name="apple-mobile-web-app-capable" content="yes" />
<meta name="apple-mobile-web-app-status-bar-style" content="black-translucent" />
```
CSP (production):
```
default-src 'self';
script-src 'self';
style-src 'self' 'unsafe-inline';
img-src 'self' data:;
connect-src 'self';
media-src 'self' blob:; <!-- the mic stream lives in a blob -->
```
`blob:` in `media-src` is required because `getUserMedia` streams are blob-backed and the analyser reads them. Verify nothing else regresses — `vite-plugin-pwa` registers the SW from a same-origin script, so `'self'` covers it.
Move the dev-only CSP into a Vite conditional so dev still allows HMR/websocket:
```js
// vite.config.js — inject dev CSP via plugin, or keep a separate dev index template.
```
Simplest: keep one CSP in `index.html` with the production values, and in dev let Vite's own injected tags coexist (HMR works over the same origin websocket; CSP `connect-src 'self'` already allows it). If dev breaks, add `ws://localhost:5173` only in a dev-specific block.
### 1.5 Verify
- `npm run build``dist/` contains `manifest.webmanifest`, `registerSW.js`, `sw.js`, and all `public/` icons.
- Serve `dist/` locally with `npm run preview` and use Chrome DevTools → Application → Manifest (should show icons, installability ✓) and Service Workers (registered).
- Install to desktop / "Add to Home Screen" on mobile.
- Confirm the app still works **fully offline** after first load (airplane mode). Since there are no network calls, the only risk is the SW failing to precache — Workbox handles this.
---
## Phase 2 — Firebase Hosting setup
Goal: serve `dist/` over HTTPS on a `*.web.app` domain (or custom domain).
### 2.1 Install the CLI
```bash
npm install -D firebase-tools
```
### 2.2 Init
```bash
npx firebase init hosting
```
Answer:
- **Public directory:** `dist`
- **Single-page app (rewrite all urls to /index.html):** Yes — though the app has no client-side routing today, this protects future routes and is harmless.
- **Set up automatic builds with GitHub:** **No.** (You're not using GitHub.)
- **File `dist/index.html` already exists — overwrite?** **No.** This is critical — saying yes would clobber the Vite-built index.
This creates:
- `firebase.json`
- `.firebaserc`
### 2.3 `firebase.json` (target shape)
```json
{
"hosting": {
"public": "dist",
"ignore": ["firebase.json", "**/.*", "**/node_modules/**"],
"rewrites": [{ "source": "**", "destination": "/index.html" }],
"headers": [
{
"source": "**/*.@(js|css|svg|png|ico|woff2|webmanifest)",
"headers": [
{ "key": "Cache-Control", "value": "public, max-age=31536000, immutable" }
]
},
{
"source": "/sw.js",
"headers": [
{ "key": "Cache-Control", "value": "no-cache" }
]
},
{
"source": "/index.html",
"headers": [
{ "key": "Cache-Control", "value": "no-cache" }
]
}
]
}
}
```
Why the headers: Vite hashes asset filenames, so JS/CSS/icons are content-addressed → cache them **immortal**. `index.html` and `sw.js` must stay fresh so updates propagate — `no-cache` revalidates them every time. (Note: with `vite-plugin-pwa`'s `autoUpdate`, the SW self-updates; the `no-cache` on `sw.js` is belt-and-braces.)
### 2.4 First deploy
```bash
npm run build
npx firebase deploy --only hosting
```
Output: `https://<project>.web.app`. The mic will now work because the origin is HTTPS.
### 2.5 `.gitignore`
Add `.firebase/` and `firebase-debug.log` (already in a sane gitignore, but confirm).
---
## Phase 3 — (optional) custom domain
In Firebase console → Hosting → Add custom domain. Add the DNS TXT/CNAME records it gives you. Lets Encrypt cert is auto-provisioned. No app change needed.
---
---
## Phase 4 — Web optimization checklist
Things that matter specifically for shipping this app on the web. Most are cheap; do them before the first public deploy.
### 4.1 Drop the dead dependency
`audiomotion-analyzer` is listed in `package.json` but **imported nowhere** in `src/`. Vite tree-shakes unused ESM, so it likely doesn't bloat the production bundle — but it still inflates `npm install` and is misleading. Either:
- **Remove it** from `dependencies` (one-line `npm uninstall audiomotion-analyzer`), or
- If something depends on it being present (it doesn't — grep confirms zero imports), wire it up.
Recommend: remove.
### 4.2 Audit bundle size
```bash
npm run build
npx vite-bundle-visualizer # one-off, no config needed
```
Expected heavy bits: `pitchy` (small, pure JS) and React 19. `electron` / `electron-builder` are devDeps and excluded from the web build. If the visualizer shows anything surprising (e.g. a transitive dep pulling in moment/lodash), trim it.
### 4.3 Code-split the heavy, rarely-used views
`DebugView`, `DrumView`, and `Tuner` are collapsible panels most users never open. They import theory helpers and own audio contexts. Lazy-load them so the initial bundle stays lean:
```jsx
const Tuner = lazy(() => import('./components/Tuner'))
const DebugView = lazy(() => import('./components/DebugView'))
const DrumView = lazy(() => import('./components/DrumView'))
```
Wrap in `<Suspense fallback={null}>`. These are already behind toggle buttons, so there's no UX cost. This is the single biggest web win — the core pitch/chord flow loads first.
### 4.4 Audio permissions & UX on the web
- **HTTPS** — covered by Firebase. `getUserMedia` is blocked on plain HTTP (except `localhost`).
- **User gesture** — the existing "Start Listening" button already provides the gesture the browser requires to call `getUserMedia`. Don't change this; never auto-start listening on load.
- **Mobile Safari quirks** — iOS Safari:
- Requires `audio: { echoCancellation: false }` style may differ; test the existing `AudioCapture` constraints on an iPhone.
- AudioContext must be resumed after a gesture — already handled since capture starts on click.
- `sharedArrayBuffer` is not needed by this app (no worklets), so no COOP/COEP headers required. Good — don't add them; they'd complicate the Firebase static host.
- **Permissions API** — optionally surface "microphone blocked" before the user clicks, using `navigator.permissions.query({ name: 'microphone' })`. The existing `micError` state already covers the denied case; this is a polish step, not required.
### 4.5 Mobile layout
The app uses Tailwind with responsive `lg:` breakpoints already (e.g. the progressions sidebar is `hidden lg:block`). For a real mobile deploy:
- Test the fretboard / piano SVGs at phone widths — they're SVG so they scale, but text labels may crowd.
- The `controls bar` wraps with `flex-wrap` — good. Verify the key-lock select dropdowns are tappable.
- Consider a `display: standalone` install prompt; the manifest already sets `standalone`.
- This is a "play and look at the screen" app — portrait guitar fretboard orientation may want `orientation: 'portrait'` in the manifest instead of `'any'`. Decide based on the primary instrument view.
### 4.6 Performance for the audio hot loop
Already mostly tuned (the dual-analyser design is deliberate). Web-specific notes:
- `requestAnimationFrame` driving the analyser is fine on web — no change.
- `Float32Array` chroma math is cheap. No change needed.
- Don't add a Workbox runtime cache for audio — there's no audio to cache; everything is live.
- If you ever add an AudioWorklet, its file must be served with the correct MIME and same-origin — Workbox precaches it as part of `globPatterns` only if it's emitted to `dist/`. Not needed today.
### 4.7 SEO / social (minimal — this is an app, not content)
- `<title>` and meta description (added in 1.4) are enough for the web.app URL to have a sane share card.
- Add `og:title` / `og:description` / `og:image` to `index.html` if you want a nice link preview (point `og:image` to `/icon-512.png`). Optional.
- No sitemap needed for a single-page offline app.
### 4.8 Keep Electron working
The desktop build must not regress:
- `electron-builder` config in `package.json` is untouched by any of the above.
- The CSP change in `index.html` affects both targets — verify the desktop window still loads (it serves the built `dist/index.html`, so the new production CSP applies there too; that's fine and arguably better).
- `vite-plugin-pwa` is a dev dependency; it only runs at build time and doesn't touch the renderer runtime in a way that breaks Electron.
- Run `npm run electron:dev` after Phase 1 to confirm the Electron path still works.
---
## What stays Electron-only
Nothing in this plan removes or forks code. The Electron and web builds share the exact same `dist/` output. The only per-target differences are:
- Electron adds the `electron/main.cjs` shell and `electron-builder` packaging.
- Web adds the PWA manifest + SW (ignored by Electron) and is served over HTTPS.
If a future feature needs a native API (filesystem dialogs, auto-launch, tray), it would go through Electron's preload/IPC — which currently exposes nothing. At that point you'd add an `isElectron` guard and a `preload` IPC bridge. Not needed for any current feature.
---
## Suggested order of operations
1. **Phase 1.2** — create `public/`, drop in icons (Phase 1.3).
2. **Phase 1.1 + 1.4** — add `vite-plugin-pwa`, update `index.html`, relax CSP.
3. Verify with `npm run preview` + DevTools (Phase 1.5).
4. **Phase 2**`firebase init hosting`, hand-tune `firebase.json`, first deploy.
5. **Phase 4.1** — remove dead `audiomotion-analyzer` dep.
6. **Phase 4.3** — lazy-load the three collapsible views.
7. Polish: mobile pass (4.5), bundle audit (4.2), optional OG tags (4.7).
Each step is independently revertible. Stop after step 4 and you have a working, installable, offline PWA on the web; the rest is optimization.
+1 -1
View File
@@ -23,7 +23,7 @@ function createWindow() {
height: 900, height: 900,
minWidth: 620, minWidth: 620,
minHeight: 600, minHeight: 600,
title: 'WhatTheFlat ♭? - JamBuddy', title: 'WhatTheFlat',
icon: path.join(__dirname, '../assets/whattheflat-logo.png'), icon: path.join(__dirname, '../assets/whattheflat-logo.png'),
webPreferences: { webPreferences: {
preload: path.join(__dirname, 'preload.cjs'), preload: path.join(__dirname, 'preload.cjs'),
-28
View File
@@ -1,28 +0,0 @@
{
"hosting": {
"site": "jambuddy",
"public": "dist",
"ignore": ["firebase.json", "**/.*", "**/node_modules/**"],
"rewrites": [{ "source": "**", "destination": "/index.html" }],
"headers": [
{
"source": "**/*.@(js|css|svg|png|ico|woff2|webmanifest)",
"headers": [
{ "key": "Cache-Control", "value": "public, max-age=31536000, immutable" }
]
},
{
"source": "/sw.js",
"headers": [
{ "key": "Cache-Control", "value": "no-cache" }
]
},
{
"source": "/index.html",
"headers": [
{ "key": "Cache-Control", "value": "no-cache" }
]
}
]
}
}
+3 -10
View File
@@ -2,16 +2,9 @@
<html lang="en"> <html lang="en">
<head> <head>
<meta charset="UTF-8" /> <meta charset="UTF-8" />
<meta name="viewport" content="width=device-width, initial-scale=1.0, viewport-fit=cover" /> <meta name="viewport" content="width=device-width, initial-scale=1.0" />
<meta name="description" content="Real-time key and chord detection for live jams. Play any instrument into your mic and JamBuddy identifies the key, chords, and tempo — fully offline." /> <meta http-equiv="Content-Security-Policy" content="default-src 'self'; script-src 'self' 'unsafe-eval'; connect-src 'self' http://localhost:5173 ws://localhost:5173; style-src 'self' 'unsafe-inline'; img-src 'self' data:;">
<meta name="theme-color" content="#0f0f0f" /> <title>WhatTheFlat</title>
<link rel="apple-touch-icon" href="apple-touch-icon.png" />
<meta name="apple-mobile-web-app-capable" content="yes" />
<meta name="apple-mobile-web-app-status-bar-style" content="black-translucent" />
<meta name="apple-mobile-web-app-title" content="JamBuddy" />
<!-- Production CSP. dev HMR runs same-origin so 'self' covers the ws upgrade. -->
<meta http-equiv="Content-Security-Policy" content="default-src 'self'; script-src 'self'; style-src 'self' 'unsafe-inline'; img-src 'self' data:; media-src 'self' blob:; connect-src 'self';" />
<title>WhatTheFlat ♭? - JamBuddy</title>
</head> </head>
<body> <body>
<div id="root"></div> <div id="root"></div>
+107 -3922
View File
File diff suppressed because it is too large Load Diff
+7 -6
View File
@@ -1,7 +1,7 @@
{ {
"name": "jambuddy", "name": "whattheflat",
"description": "A jam session companion app that provides a chromatic tuner, chord progressions, and more.", "description": "A jam session companion app that provides a chromatic tuner, chord progressions, and more.",
"version": "0.6.2", "version": "0.6.0",
"private": true, "private": true,
"type": "module", "type": "module",
"main": "electron/main.cjs", "main": "electron/main.cjs",
@@ -13,9 +13,11 @@
"electron:build": "vite build && electron-builder", "electron:build": "vite build && electron-builder",
"electron:build:win": "vite build && electron-builder --win --publish never", "electron:build:win": "vite build && electron-builder --win --publish never",
"electron:build:mac": "vite build && electron-builder --mac --publish never", "electron:build:mac": "vite build && electron-builder --mac --publish never",
"electron:build:linux": "vite build && electron-builder --linux --publish never" "electron:build:linux": "vite build && electron-builder --linux"
}, },
"dependencies": { "dependencies": {
"audiomotion-analyzer": "^4.5.4",
"essentia.js": "^0.1.3",
"pitchy": "^4.1.0", "pitchy": "^4.1.0",
"react": "^19.2.4", "react": "^19.2.4",
"react-dom": "^19.2.4" "react-dom": "^19.2.4"
@@ -30,12 +32,11 @@
"electron-builder": "^26.8.1", "electron-builder": "^26.8.1",
"tailwindcss": "^4.2.1", "tailwindcss": "^4.2.1",
"vite": "^7.3.1", "vite": "^7.3.1",
"vite-plugin-pwa": "^1.3.0",
"wait-on": "^9.0.4" "wait-on": "^9.0.4"
}, },
"build": { "build": {
"appId": "com.jambuddy.app", "appId": "com.whattheflat.app",
"productName": "JamBuddy", "productName": "WhatTheFlat",
"files": [ "files": [
"dist/**/*", "dist/**/*",
"electron/**/*", "electron/**/*",
Binary file not shown.

Before

Width:  |  Height:  |  Size: 18 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 20 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 49 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 67 KiB

+52 -106
View File
@@ -1,20 +1,16 @@
import { useState, useCallback, useRef, useEffect, lazy, Suspense } from 'react' import { useState, useCallback, useRef, useEffect } from 'react'
import AudioCapture from './components/AudioCapture' import AudioCapture from './components/AudioCapture'
import ProgressionBanner from './components/ProgressionBanner' import ProgressionBanner from './components/ProgressionBanner'
import ProgressionSuggestions from './components/ProgressionSuggestions' import ProgressionSuggestions from './components/ProgressionSuggestions'
import Fretboard from './components/Fretboard' import Fretboard from './components/Fretboard'
import BassFretboard from './components/BassFretboard' import BassFretboard from './components/BassFretboard'
import Tuner from './components/Tuner'
import Piano from './components/Piano' import Piano from './components/Piano'
import Settings from './components/Settings' import Settings from './components/Settings'
import DebugView from './components/DebugView'
import { NOTES, detectKey, detectTopKeys, matchChordFromChroma, detectRepeatingProgression, getChordTones, getChordCandidates, getNoteHistoryAnalysis } from './lib/theory' import { NOTES, detectKey, detectTopKeys, matchChordFromChroma, detectRepeatingProgression, getChordTones, getChordCandidates, getNoteHistoryAnalysis } from './lib/theory'
import settingIcon from './assets/setting-icon.png' import settingIcon from './assets/setting-icon.png'
import viewIcon from './assets/view.png'
// ponytail: collapsible panels most users never open — lazy-load so the core
// pitch/chord flow ships in the initial bundle. Suspense fallback null: they
// render behind a toggle button, so there's no visible flash.
const Tuner = lazy(() => import('./components/Tuner'))
const DebugView = lazy(() => import('./components/DebugView'))
const DrumView = lazy(() => import('./components/DrumView'))
const DEFAULTS = { const DEFAULTS = {
// Key detection // Key detection
@@ -29,20 +25,15 @@ const DEFAULTS = {
// Audio input // Audio input
minClarity: 0.80, minClarity: 0.80,
minVolume: 0.01, minVolume: 0.01,
// Selected device (null = system default) // Experimental
audioDeviceId: null, useEssentia: false,
}
function loadStored(key, fallback) {
try { const v = localStorage.getItem(key); return v !== null ? JSON.parse(v) : fallback }
catch { return fallback }
} }
export default function App() { export default function App() {
// ── Config ─────────────────────────────────────────────────────────────────── // ── Config ───────────────────────────────────────────────────────────────────
const [config, setConfig] = useState(() => ({ ...DEFAULTS, ...loadStored('wtf_config', {}) })) const [config, setConfig] = useState(DEFAULTS)
const configRef = useRef(config) const configRef = useRef(DEFAULTS)
useEffect(() => { configRef.current = config; localStorage.setItem('wtf_config', JSON.stringify(config)) }, [config]) useEffect(() => { configRef.current = config }, [config])
const [showSettings, setShowSettings] = useState(false) const [showSettings, setShowSettings] = useState(false)
@@ -54,11 +45,10 @@ export default function App() {
const [isListening, setIsListening] = useState(false) const [isListening, setIsListening] = useState(false)
// ── Instrument view + tuner ─────────────────────────────────────────────────── // ── Instrument view + tuner ───────────────────────────────────────────────────
const [instrument, setInstrument] = useState('piano') // 'piano' | 'guitar' | 'bass' const [instrument, setInstrument] = useState('guitar') // 'guitar' | 'bass' | 'piano'
const [showTuner, setShowTuner] = useState(false) const [showTuner, setShowTuner] = useState(false)
const [showDebug, setShowDebug] = useState(false) const [showDebug, setShowDebug] = useState(false)
const [showDrumView, setShowDrumView] = useState(false) const [monoColor, setMonoColor] = useState(false)
const [monoColor, setMonoColor] = useState(() => loadStored('wtf_monoColor', false))
// ── Mic permission error ────────────────────────────────────────────────────── // ── Mic permission error ──────────────────────────────────────────────────────
const [micError, setMicError] = useState(null) const [micError, setMicError] = useState(null)
@@ -67,17 +57,11 @@ export default function App() {
const [debugChroma, setDebugChroma] = useState(null) const [debugChroma, setDebugChroma] = useState(null)
const [debugCandidates, setDebugCandidates] = useState([]) const [debugCandidates, setDebugCandidates] = useState([])
const [debugNoteAnalysis, setDebugNoteAnalysis] = useState(null) const [debugNoteAnalysis, setDebugNoteAnalysis] = useState(null)
const [debugWaveform, setDebugWaveform] = useState(null)
// ── Stable refs for values used inside callbacks ────────────────────────────── // ── Stable refs for values used inside callbacks ──────────────────────────────
const showDebugRef = useRef(showDebug) const showDebugRef = useRef(showDebug)
const showDrumViewRef = useRef(showDrumView)
const lockedKeyRef = useRef(null) const lockedKeyRef = useRef(null)
const listenStartRef = useRef(null)
useEffect(() => { showDebugRef.current = showDebug }, [showDebug]) useEffect(() => { showDebugRef.current = showDebug }, [showDebug])
useEffect(() => { showDrumViewRef.current = showDrumView }, [showDrumView])
useEffect(() => { localStorage.setItem('wtf_monoColor', JSON.stringify(monoColor)) }, [monoColor])
useEffect(() => { if (isListening) listenStartRef.current = Date.now() }, [isListening])
// ── BPM estimation from onset timestamps ───────────────────────────────────── // ── BPM estimation from onset timestamps ─────────────────────────────────────
const [bpm, setBpm] = useState(null) const [bpm, setBpm] = useState(null)
@@ -110,7 +94,6 @@ export default function App() {
const chromaIdxRef = useRef(0) const chromaIdxRef = useRef(0)
const chordVotesRef = useRef([]) const chordVotesRef = useRef([])
const progressionVoteRef = useRef(null) const progressionVoteRef = useRef(null)
const progressionMissRef = useRef(0)
const pendingKeyRef = useRef(null) const pendingKeyRef = useRef(null)
// Keep refs in sync // Keep refs in sync
@@ -128,16 +111,7 @@ export default function App() {
// ── Detect progression — require 2 consecutive identical results to commit ──── // ── Detect progression — require 2 consecutive identical results to commit ────
useEffect(() => { useEffect(() => {
const detected = detectRepeatingProgression(chordHistory) const detected = detectRepeatingProgression(chordHistory)
if (!detected) { if (!detected) return
progressionMissRef.current++
// Clear stale loop after 4 chord changes with no pattern found
if (progressionMissRef.current >= 4) {
setDetectedProgression(null)
progressionVoteRef.current = null
}
return
}
progressionMissRef.current = 0
const key = detected.join(',') const key = detected.join(',')
if (progressionVoteRef.current === key) { if (progressionVoteRef.current === key) {
setDetectedProgression(detected) setDetectedProgression(detected)
@@ -153,13 +127,11 @@ export default function App() {
keyVotesRef.current = [] keyVotesRef.current = []
chordVotesRef.current = [] chordVotesRef.current = []
progressionVoteRef.current = null progressionVoteRef.current = null
progressionMissRef.current = 0
pendingKeyRef.current = null pendingKeyRef.current = null
chromaIdxRef.current = 0 chromaIdxRef.current = 0
chromaRingRef.current = Array.from({ length: cfg.chromaSmooth }, () => new Float32Array(12)) chromaRingRef.current = Array.from({ length: cfg.chromaSmooth }, () => new Float32Array(12))
onsetTimestampsRef.current = [] onsetTimestampsRef.current = []
bpmSmoothRef.current = null bpmSmoothRef.current = null
listenStartRef.current = Date.now()
setKeyInfo(null) setKeyInfo(null)
setLockedKey(null) setLockedKey(null)
effectiveKeyRef.current = null effectiveKeyRef.current = null
@@ -171,7 +143,6 @@ export default function App() {
setDebugChroma(null) setDebugChroma(null)
setDebugCandidates([]) setDebugCandidates([])
setDebugNoteAnalysis(null) setDebugNoteAnalysis(null)
setDebugWaveform(null)
} }
// ── Key lock handlers ───────────────────────────────────────────────────────── // ── Key lock handlers ─────────────────────────────────────────────────────────
@@ -194,13 +165,6 @@ export default function App() {
effectiveKeyRef.current = keyInfo effectiveKeyRef.current = keyInfo
} }
// ── Waveform handler: feeds oscilloscope / drum view ─────────────────────────
const handleWaveform = useCallback((data) => {
if (showDebugRef.current || showDrumViewRef.current) {
setDebugWaveform({ ...data, onsets: [...onsetTimestampsRef.current] })
}
}, [])
// ── Note handler: drives key detection (pitch-based) ────────────────────────── // ── Note handler: drives key detection (pitch-based) ──────────────────────────
const handleNote = useCallback(({ pitchClass }) => { const handleNote = useCallback(({ pitchClass }) => {
const cfg = configRef.current const cfg = configRef.current
@@ -212,11 +176,7 @@ export default function App() {
const result = detectKey(history) const result = detectKey(history)
setTopKeyCandidates(detectTopKeys(history)) setTopKeyCandidates(detectTopKeys(history))
if (showDebugRef.current) { if (showDebugRef.current) setDebugNoteAnalysis(getNoteHistoryAnalysis(history))
const analysis = getNoteHistoryAnalysis(history)
analysis.sessionSecs = listenStartRef.current ? Math.floor((Date.now() - listenStartRef.current) / 1000) : 0
setDebugNoteAnalysis(analysis)
}
if (result.confidence < 0.5) return if (result.confidence < 0.5) return
const votes = keyVotesRef.current const votes = keyVotesRef.current
@@ -268,7 +228,7 @@ export default function App() {
if (showDebugRef.current) { if (showDebugRef.current) {
setDebugChroma([...avg]) setDebugChroma([...avg])
setDebugCandidates(getChordCandidates(avg, key, bassPC, 5)) setDebugCandidates(getChordCandidates(avg, key, bassPC))
} }
// Stability gate — if chroma is still changing across frames, we're mid-transition. // Stability gate — if chroma is still changing across frames, we're mid-transition.
@@ -279,7 +239,10 @@ export default function App() {
for (const frame of ring) { const d = frame[i] - avg[i]; v += d * d } for (const frame of ring) { const d = frame[i] - avg[i]; v += d * d }
if (v / cfg.chromaSmooth > maxVar) maxVar = v / cfg.chromaSmooth if (v / cfg.chromaSmooth > maxVar) maxVar = v / cfg.chromaSmooth
} }
if (maxVar > 0.05) return if (maxVar > 0.05) {
chordVotesRef.current = []
return
}
const chord = matchChordFromChroma(avg, key, bassPC, false, cfg.chordMinScore) const chord = matchChordFromChroma(avg, key, bassPC, false, cfg.chordMinScore)
if (!chord) { if (!chord) {
@@ -295,7 +258,7 @@ export default function App() {
const winner = votes[0] const winner = votes[0]
setChordHistory(prev => { setChordHistory(prev => {
if (prev[prev.length - 1] === winner) return prev if (prev[prev.length - 1] === winner) return prev
return [...prev.slice(-48), winner] return [...prev.slice(-30), winner]
}) })
// Inject chord tones into note history to anchor key detection // Inject chord tones into note history to anchor key detection
@@ -362,25 +325,40 @@ export default function App() {
config={config} config={config}
onChange={updateConfig} onChange={updateConfig}
onClose={() => setShowSettings(false)} onClose={() => setShowSettings(false)}
onReset={() => { setConfig(DEFAULTS); setMonoColor(false) }} onReset={() => setConfig(DEFAULTS)}
monoColor={monoColor}
onMonoColorChange={setMonoColor}
/> />
) )
} }
return ( return (
<div className="app-shell min-h-screen bg-surface text-white"> <div className="min-h-screen bg-surface text-white p-3">
{/* ── Header ── */} {/* ── Header ── */}
<header className="mb-2 flex items-center justify-between gap-2 flex-wrap"> <header className="mb-2 flex items-center justify-between">
<div> <div>
<h1 className="text-xl font-bold text-accent"> <h1 className="text-xl font-bold text-accent">
WhatTheFlat <span className="text-gray-600">&#9837;?</span> <span className="text-amber-400">- JamBuddy</span> WhatTheFlat <span className="text-gray-600">&#9837;?</span>
</h1> </h1>
<p className="text-xs text-gray-600 hidden sm:block">Real-time key detection for live jams</p> <p className="text-xs text-gray-600">Real-time key detection for live jams</p>
</div> </div>
<div className="flex gap-2 items-center"> <div className="flex gap-2 items-center">
<button
onClick={() => setMonoColor(v => !v)}
className={`p-2 rounded-full border transition-all ${monoColor ? 'border-accent bg-accent/10' : 'border-border hover:border-gray-400'}`}
title="Mono color mode"
>
<svg width="20" height="20" viewBox="0 0 20 20" fill="none" style={{ opacity: 0.75 }}>
<circle cx="6" cy="10" r="4" fill={monoColor ? '#a855f7' : '#a855f7'} />
<circle cx="13" cy="10" r="4" fill={monoColor ? '#c084fc' : '#f59e0b'} />
</svg>
</button>
<button
onClick={() => setShowDebug(v => !v)}
className={`p-2 rounded-full border transition-all ${showDebug ? 'border-accent bg-accent/10' : 'border-border hover:border-gray-400'}`}
title="Behind the scenes"
>
<img src={viewIcon} alt="Debug view" className="w-5 h-5" style={{ filter: 'invert(1) opacity(0.75)' }} />
</button>
<button <button
onClick={() => setShowSettings(true)} onClick={() => setShowSettings(true)}
className="p-2 rounded-full border border-border hover:border-gray-400 transition-all" className="p-2 rounded-full border border-border hover:border-gray-400 transition-all"
@@ -418,9 +396,9 @@ export default function App() {
onChange={e => setInstrument(e.target.value)} onChange={e => setInstrument(e.target.value)}
className="appearance-none bg-surface border border-border hover:border-gray-500 focus:border-accent focus:outline-none rounded-lg pl-3 pr-7 py-1 text-sm text-gray-200 cursor-pointer transition-colors" className="appearance-none bg-surface border border-border hover:border-gray-500 focus:border-accent focus:outline-none rounded-lg pl-3 pr-7 py-1 text-sm text-gray-200 cursor-pointer transition-colors"
> >
<option value="piano">Piano</option>
<option value="guitar">Guitar</option> <option value="guitar">Guitar</option>
<option value="bass">Bass</option> <option value="bass">Bass</option>
<option value="piano">Piano</option>
</select> </select>
<span className="pointer-events-none absolute right-2 top-1/2 -translate-y-1/2 text-gray-500 text-xs"></span> <span className="pointer-events-none absolute right-2 top-1/2 -translate-y-1/2 text-gray-500 text-xs"></span>
</div> </div>
@@ -516,11 +494,10 @@ export default function App() {
onNote={handleNote} onNote={handleNote}
onChroma={handleChroma} onChroma={handleChroma}
onOnset={handleOnset} onOnset={handleOnset}
onWaveform={handleWaveform}
isListening={isListening} isListening={isListening}
minClarity={config.minClarity} minClarity={config.minClarity}
minVolume={config.minVolume} minVolume={config.minVolume}
audioDeviceId={config.audioDeviceId} useEssentia={config.useEssentia}
onPermissionError={() => { onPermissionError={() => {
setMicError(true) setMicError(true)
setIsListening(false) setIsListening(false)
@@ -543,75 +520,44 @@ export default function App() {
/> />
{/* ── Instrument + progressions row ── */} {/* ── Instrument + progressions row ── */}
<div className="flex flex-col lg:flex-row gap-3 mb-3 items-stretch"> <div className="flex gap-3 mb-3 items-stretch">
<div className="w-full lg:w-[70%] min-w-0"> <div className="w-full lg:w-[70%] min-w-0">
{instrument === 'guitar' && <Fretboard keyInfo={effectiveKey} currentChord={currentChord} pentatonicOnly={false} monoColor={monoColor} />} {instrument === 'guitar' && <Fretboard keyInfo={effectiveKey} currentChord={currentChord} pentatonicOnly={false} monoColor={monoColor} />}
{instrument === 'bass' && <BassFretboard keyInfo={effectiveKey} currentChord={currentChord} monoColor={monoColor} />} {instrument === 'bass' && <BassFretboard keyInfo={effectiveKey} currentChord={currentChord} monoColor={monoColor} />}
{instrument === 'piano' && <Piano keyInfo={effectiveKey} currentChord={currentChord} monoColor={monoColor} />} {instrument === 'piano' && <Piano keyInfo={effectiveKey} currentChord={currentChord} monoColor={monoColor} />}
</div> </div>
<div className="w-full lg:w-[30%] min-w-0 lg:relative"> <div className="hidden lg:block w-[30%] min-w-0 relative">
<div className="lg:absolute lg:inset-0"> <div className="absolute inset-0">
<ProgressionSuggestions keyInfo={effectiveKey} currentChord={currentChord} /> <ProgressionSuggestions keyInfo={effectiveKey} currentChord={currentChord} />
</div> </div>
</div> </div>
</div> </div>
{/* ── Behind the scenes — collapsible ── */} {/* ── Behind the scenes debug view ── */}
<div className="mb-3 bg-panel border border-border rounded-xl overflow-hidden">
<button
onClick={() => setShowDebug(v => !v)}
className="w-full flex items-center justify-between px-4 py-2 text-sm text-gray-400 hover:text-gray-200 transition-all"
>
<span>BEHIND THE SCENES</span>
<span>{showDebug ? '▲' : '▼'}</span>
</button>
{showDebug && ( {showDebug && (
<div className="border-t border-border p-4"> <div className="mb-3">
<Suspense fallback={null}>
<DebugView <DebugView
chroma={debugChroma} chroma={debugChroma}
chordCandidates={debugCandidates} chordCandidates={debugCandidates}
noteAnalysis={debugNoteAnalysis} noteAnalysis={debugNoteAnalysis}
waveform={debugWaveform}
keyInfo={effectiveKey} keyInfo={effectiveKey}
currentChord={currentChord} currentChord={currentChord}
instrument={instrument} instrument={instrument}
monoColor={monoColor}
/> />
</Suspense>
</div> </div>
)} )}
</div>
{/* ── Rhythm / drum analyser — collapsible ── */}
<div className="mb-3 bg-panel border border-border rounded-xl overflow-hidden">
<button
onClick={() => setShowDrumView(v => !v)}
className="w-full flex items-center justify-between px-4 py-2 text-sm text-gray-400 hover:text-gray-200 transition-all"
>
<span>RHYTHM ANALYSER</span>
<span>{showDrumView ? '▲' : '▼'}</span>
</button>
{showDrumView && (
<div className="border-t border-border p-4">
<Suspense fallback={null}>
<DrumView waveform={debugWaveform} bpm={bpm} />
</Suspense>
</div>
)}
</div>
{/* ── Tuner — collapsible ── */} {/* ── Tuner — collapsible ── */}
<div className="bg-panel border border-border rounded-xl overflow-hidden"> <div>
<button <button
onClick={() => setShowTuner(v => !v)} onClick={() => setShowTuner(v => !v)}
className="w-full flex items-center justify-between px-4 py-2 text-sm text-gray-400 hover:text-gray-200 transition-all" className="w-full flex items-center justify-between px-4 py-2 bg-panel border border-border rounded-xl text-sm text-gray-400 hover:text-gray-200 hover:border-gray-500 transition-all"
> >
<span>TUNER</span> <span>Tuner</span>
<span>{showTuner ? '▲' : '▼'}</span> <span>{showTuner ? '▲' : '▼'}</span>
</button> </button>
{showTuner && <div className="border-t border-border"><Suspense fallback={null}><Tuner /></Suspense></div>} {showTuner && <div className="mt-2"><Tuner /></div>}
</div> </div>
</div> </div>
) )
+22 -81
View File
@@ -1,6 +1,7 @@
import { useEffect, useRef, useCallback } from 'react' import { useEffect, useRef, useCallback } from 'react'
import { PitchDetector } from 'pitchy' import { PitchDetector } from 'pitchy'
import { NOTES } from '../lib/theory' import { NOTES } from '../lib/theory'
import { initEssentia, getEssentia, computeHPCP } from '../lib/essentiaHPCP'
// Why two analysers? // Why two analysers?
// //
@@ -81,7 +82,7 @@ function detectBassPC(freqData, sampleRate, fftSize) {
return ((bestMidi % 12) + 12) % 12 return ((bestMidi % 12) + 12) % 12
} }
export default function AudioCapture({ onNote, onChroma, onOnset, onWaveform, isListening, minClarity = 0.80, minVolume = 0.01, onPermissionError, audioDeviceId = null }) { export default function AudioCapture({ onNote, onChroma, onOnset, isListening, minClarity = 0.80, minVolume = 0.01, onPermissionError, useEssentia = false }) {
const audioCtxRef = useRef(null) const audioCtxRef = useRef(null)
const timeBufRef = useRef(null) const timeBufRef = useRef(null)
const freqBufRef = useRef(null) const freqBufRef = useRef(null)
@@ -94,24 +95,26 @@ export default function AudioCapture({ onNote, onChroma, onOnset, onWaveform, is
const onNoteRef = useRef(onNote) const onNoteRef = useRef(onNote)
const onChromaRef = useRef(onChroma) const onChromaRef = useRef(onChroma)
const onOnsetRef = useRef(onOnset) const onOnsetRef = useRef(onOnset)
const onWaveformRef = useRef(onWaveform)
const onPermissionErrorRef = useRef(onPermissionError) const onPermissionErrorRef = useRef(onPermissionError)
const minClarityRef = useRef(minClarity) const minClarityRef = useRef(minClarity)
const minVolumeRef = useRef(minVolume) const minVolumeRef = useRef(minVolume)
const useEssentiaRef = useRef(useEssentia)
const smoothRmsRef = useRef(0) const smoothRmsRef = useRef(0)
const lastOnsetRef = useRef(0) const lastOnsetRef = useRef(0)
const specPeakRef = useRef(null) // peak-hold spectrum for display lingering
useEffect(() => { onNoteRef.current = onNote }, [onNote]) useEffect(() => { onNoteRef.current = onNote }, [onNote])
useEffect(() => { onChromaRef.current = onChroma }, [onChroma]) useEffect(() => { onChromaRef.current = onChroma }, [onChroma])
useEffect(() => { onOnsetRef.current = onOnset }, [onOnset]) useEffect(() => { onOnsetRef.current = onOnset }, [onOnset])
useEffect(() => { onWaveformRef.current = onWaveform }, [onWaveform])
useEffect(() => { onPermissionErrorRef.current = onPermissionError }, [onPermissionError]) useEffect(() => { onPermissionErrorRef.current = onPermissionError }, [onPermissionError])
useEffect(() => { minClarityRef.current = minClarity }, [minClarity]) useEffect(() => { minClarityRef.current = minClarity }, [minClarity])
useEffect(() => { minVolumeRef.current = minVolume }, [minVolume]) useEffect(() => { minVolumeRef.current = minVolume }, [minVolume])
useEffect(() => { useEssentiaRef.current = useEssentia }, [useEssentia])
// Pre-load Essentia WASM as soon as the component mounts it's a singleton,
// so repeated calls just return the cached instance.
useEffect(() => { initEssentia().catch(console.error) }, [])
const stop = useCallback(() => { const stop = useCallback(() => {
activeRef.current = false activeRef.current = false
specPeakRef.current = null
if (rafRef.current) { cancelAnimationFrame(rafRef.current); rafRef.current = null } if (rafRef.current) { cancelAnimationFrame(rafRef.current); rafRef.current = null }
if (streamRef.current) { streamRef.current.getTracks().forEach(t => t.stop()); streamRef.current = null } if (streamRef.current) { streamRef.current.getTracks().forEach(t => t.stop()); streamRef.current = null }
if (audioCtxRef.current) { audioCtxRef.current.close(); audioCtxRef.current = null } if (audioCtxRef.current) { audioCtxRef.current.close(); audioCtxRef.current = null }
@@ -121,34 +124,8 @@ export default function AudioCapture({ onNote, onChroma, onOnset, onWaveform, is
stop() stop()
let stream let stream
try { try {
// Helpful debug: list available media devices before requesting permission stream = await navigator.mediaDevices.getUserMedia({ audio: true })
try {
if (navigator.mediaDevices && navigator.mediaDevices.enumerateDevices) {
const devices = await navigator.mediaDevices.enumerateDevices()
const audioIns = devices.filter(d => d.kind === 'audioinput')
console.log('Audio inputs available:', audioIns)
}
} catch (e) {
console.warn('enumerateDevices failed', e)
}
const constraints = audioDeviceId
? { audio: { deviceId: { exact: audioDeviceId } } }
: { audio: true }
console.log('Requesting getUserMedia with constraints:', constraints)
stream = await navigator.mediaDevices.getUserMedia(constraints)
} catch (err) { } catch (err) {
// If permission denied or other error, surface extra diagnostics when possible
console.warn('getUserMedia failed', err)
try {
if (navigator.permissions && navigator.permissions.query) {
const p = await navigator.permissions.query({ name: 'microphone' })
console.log('microphone permission state:', p.state)
}
} catch (e) {
// ignore; not all environments support Permissions API for microphone
}
onPermissionErrorRef.current?.(err) onPermissionErrorRef.current?.(err)
return return
} }
@@ -161,14 +138,6 @@ export default function AudioCapture({ onNote, onChroma, onOnset, onWaveform, is
audioCtxRef.current = ctx audioCtxRef.current = ctx
const source = ctx.createMediaStreamSource(stream) const source = ctx.createMediaStreamSource(stream)
// Debug: log the acquired audio tracks and labels/deviceIds
try {
const tracks = stream.getAudioTracks()
console.log('Acquired audio tracks:', tracks.map(t => ({ label: t.label, id: t.id, enabled: t.enabled, muted: t.muted })))
} catch (e) {
console.warn('Could not inspect stream tracks', e)
}
// Small analyser pitch detection needs fast time-domain data // Small analyser pitch detection needs fast time-domain data
const pa = ctx.createAnalyser() const pa = ctx.createAnalyser()
pa.fftSize = PITCH_FFT pa.fftSize = PITCH_FFT
@@ -201,46 +170,6 @@ export default function AudioCapture({ onNote, onChroma, onOnset, onWaveform, is
onOnsetRef.current?.() onOnsetRef.current?.()
} }
// Always fire waveform callback downsample 4096 512 points + log-binned spectrum
if (onWaveformRef.current) {
const stride = 8 // 4096 / 8 = 512 points
const wave = new Float32Array(PITCH_FFT / stride)
for (let i = 0; i < wave.length; i++) wave[i] = timeBuf[i * stride]
// Log-binned frequency spectrum: 256 bins from 40 Hz 4000 Hz
const LOG_BINS = 256
const F_MIN = 40, F_MAX = 4000
const binHz = ctx.sampleRate / ca.fftSize
const freqBuf = freqBufRef.current
ca.getFloatFrequencyData(freqBuf)
const spectrum = new Float32Array(LOG_BINS)
for (let b = 0; b < LOG_BINS; b++) {
const f = F_MIN * Math.pow(F_MAX / F_MIN, b / (LOG_BINS - 1))
const bin = Math.round(f / binHz)
if (bin < freqBuf.length) {
const db = freqBuf[bin]
spectrum[b] = db < NOISE_FLOOR ? 0 : Math.max(0, (db - NOISE_FLOOR) / (-NOISE_FLOOR))
}
}
// Peak-hold with exponential decay spectrum rises instantly, falls slowly
if (!specPeakRef.current) specPeakRef.current = new Float32Array(LOG_BINS)
const peak = specPeakRef.current
for (let b = 0; b < LOG_BINS; b++) {
peak[b] = spectrum[b] > peak[b] ? spectrum[b] : peak[b] * 0.92
}
let detectedFreq = null, detectedNote = null
if (rms >= minVolumeRef.current) {
const [f, c] = detectorRef.current.findPitch(timeBuf, ctx.sampleRate)
if (c >= minClarityRef.current && f > 60 && f < 4200) {
detectedFreq = f
detectedNote = NOTES[((Math.round(12 * Math.log2(f / 440) + 69) % 12) + 12) % 12]
}
}
onWaveformRef.current({ wave, rms, detectedFreq, detectedNote, spectrum: peak })
}
if (rms >= minVolumeRef.current) { if (rms >= minVolumeRef.current) {
const [freq, clarity] = detectorRef.current.findPitch(timeBuf, ctx.sampleRate) const [freq, clarity] = detectorRef.current.findPitch(timeBuf, ctx.sampleRate)
if (clarity >= minClarityRef.current && freq > 60 && freq < 4200) { if (clarity >= minClarityRef.current && freq > 60 && freq < 4200) {
@@ -252,8 +181,20 @@ export default function AudioCapture({ onNote, onChroma, onOnset, onWaveform, is
if (onChromaRef.current) { if (onChromaRef.current) {
const freqBuf = freqBufRef.current const freqBuf = freqBufRef.current
ca.getFloatFrequencyData(freqBuf) ca.getFloatFrequencyData(freqBuf)
let chroma
const essentia = useEssentiaRef.current ? getEssentia() : null
if (essentia) {
try {
chroma = computeHPCP(essentia, freqBuf, ctx.sampleRate)
} catch (err) {
console.error('[Essentia] computeHPCP failed, falling back to custom chroma:', err)
chroma = computeChroma(freqBuf, ctx.sampleRate, ca.fftSize)
}
} else {
chroma = computeChroma(freqBuf, ctx.sampleRate, ca.fftSize)
}
onChromaRef.current( onChromaRef.current(
computeChroma(freqBuf, ctx.sampleRate, ca.fftSize), chroma,
detectBassPC(freqBuf, ctx.sampleRate, ca.fftSize) detectBassPC(freqBuf, ctx.sampleRate, ca.fftSize)
) )
} }
+24 -367
View File
@@ -1,4 +1,3 @@
import { useRef } from 'react'
import { getScale, getChordTones, NOTES } from '../lib/theory' import { getScale, getChordTones, NOTES } from '../lib/theory'
// SVG Piano 2 octaves (C3B4) // SVG Piano 2 octaves (C3B4)
@@ -15,7 +14,7 @@ const BLACK_OCT = [
] ]
const WHITE_LABELS = ['C3','D3','E3','F3','G3','A3','B3','C4','D4','E4','F4','G4','A4','B4'] const WHITE_LABELS = ['C3','D3','E3','F3','G3','A3','B3','C4','D4','E4','F4','G4','A4','B4']
function PianoSVG({ values, keyNotes, chordNotes, keyH = KEY_H, showPct = false, monoColor = false }) { function PianoSVG({ values, keyNotes, chordNotes, keyH = KEY_H, showPct = false }) {
const max = Math.max(...values, 0.01) const max = Math.max(...values, 0.01)
const wKeys = [] const wKeys = []
const bKeys = [] const bKeys = []
@@ -36,7 +35,7 @@ function PianoSVG({ values, keyNotes, chordNotes, keyH = KEY_H, showPct = false,
const fillColor = inChord const fillColor = inChord
? `rgba(167,139,250,${0.12 + energy * 0.88})` ? `rgba(167,139,250,${0.12 + energy * 0.88})`
: inKey : inKey
? monoColor ? `rgba(192,132,252,${0.1 + energy * 0.7})` : `rgba(251,191,36,${0.1 + energy * 0.7})` ? `rgba(251,191,36,${0.1 + energy * 0.7})`
: `rgba(180,180,190,${0.05 + energy * 0.2})` : `rgba(180,180,190,${0.05 + energy * 0.2})`
const pct = showPct ? Math.round(values[pc] * 100) : 0 const pct = showPct ? Math.round(values[pc] * 100) : 0
@@ -44,7 +43,7 @@ function PianoSVG({ values, keyNotes, chordNotes, keyH = KEY_H, showPct = false,
<g key={`w${wi}`}> <g key={`w${wi}`}>
<rect x={x+1} y={3} width={KEY_W-2} height={keyH} <rect x={x+1} y={3} width={KEY_W-2} height={keyH}
rx={3} fill="rgb(20,20,26)" stroke="rgba(255,255,255,0.08)" strokeWidth={1} /> rx={3} fill="rgb(20,20,26)" stroke="rgba(255,255,255,0.08)" strokeWidth={1} />
{energy > (inChord || inKey ? 0.12 : 0.35) && ( {energy > 0.04 && (
<rect <rect
x={x+1} y={3 + keyH * (1 - Math.min(energy, 1) * 0.85)} x={x+1} y={3 + keyH * (1 - Math.min(energy, 1) * 0.85)}
width={KEY_W-2} height={keyH * Math.min(energy, 1) * 0.85} width={KEY_W-2} height={keyH * Math.min(energy, 1) * 0.85}
@@ -56,7 +55,7 @@ function PianoSVG({ values, keyNotes, chordNotes, keyH = KEY_H, showPct = false,
</text> </text>
{showPct && pct > 0 && ( {showPct && pct > 0 && (
<text x={x + KEY_W/2} y={keyH + 14} textAnchor="middle" fontSize={8} <text x={x + KEY_W/2} y={keyH + 14} textAnchor="middle" fontSize={8}
fill={inKey ? (monoColor ? 'rgb(192,132,252)' : 'rgb(251,191,36)') : 'rgba(100,100,110,0.8)'}> fill={inKey ? 'rgb(251,191,36)' : 'rgba(100,100,110,0.8)'}>
{pct}% {pct}%
</text> </text>
)} )}
@@ -70,34 +69,17 @@ function PianoSVG({ values, keyNotes, chordNotes, keyH = KEY_H, showPct = false,
const inChord = chordNotes?.has(pc) const inChord = chordNotes?.has(pc)
const inKey = keyNotes?.has(pc) const inKey = keyNotes?.has(pc)
const x = wi * KEY_W + KEY_W - BLACK_W / 2 const x = wi * KEY_W + KEY_W - BLACK_W / 2
const fillColor = inChord const bg = inChord
? 'rgba(139,92,246,0.9)' ? `rgba(139,92,246,${0.4 + energy * 0.6})`
: inKey : inKey
? monoColor ? 'rgba(192,132,252,0.85)' : 'rgba(180,130,0,0.85)' ? `rgba(180,130,0,${0.35 + energy * 0.55})`
: 'rgba(70,70,80,0.75)' : `rgba(12,12,16,0.95)`
const pct = showPct ? Math.round(values[pc] * 100) : 0
return ( return (
<g key={`b${i}`}> <g key={`b${i}`}>
{/* Base */}
<rect x={x} y={3} width={BLACK_W} height={BLACK_H} <rect x={x} y={3} width={BLACK_W} height={BLACK_H}
rx={2} fill="rgb(14,14,18)" stroke="rgba(255,255,255,0.06)" strokeWidth={1} /> rx={2} fill={bg} stroke="rgba(255,255,255,0.06)" strokeWidth={1} />
{/* Partial fill from bottom — same mechanic as white keys */} <text x={x + BLACK_W/2} y={BLACK_H - 5} textAnchor="middle" fontSize={7}
{energy > (inChord || inKey ? 0.05 : 0.35) && (
<rect
x={x} y={3 + BLACK_H * (1 - Math.min(energy, 1) * 0.9)}
width={BLACK_W} height={BLACK_H * Math.min(energy, 1) * 0.9}
rx={1} fill={fillColor} />
)}
{/* % label near top of key (inside) */}
{showPct && pct > 0 && (
<text x={x + BLACK_W/2} y={3 + 10} textAnchor="middle" fontSize={7}
fill={inKey || inChord ? 'rgba(220,220,230,0.9)' : 'rgba(110,110,120,0.7)'}>
{pct}%
</text>
)}
{/* Note name near bottom of key */}
<text x={x + BLACK_W/2} y={3 + BLACK_H - 5} textAnchor="middle" fontSize={7}
fill={inKey || inChord ? 'rgba(210,210,220,0.85)' : 'rgba(110,110,120,0.6)'}> fill={inKey || inChord ? 'rgba(210,210,220,0.85)' : 'rgba(110,110,120,0.6)'}>
{NOTES[pc]} {NOTES[pc]}
</text> </text>
@@ -131,7 +113,7 @@ const MF_H = MF_PAD_T + 5 * MF_STR_H + MF_PAD_B
const mfFretX = f => MF_NUT_X + (f - 0.5) * MF_FRET_W const mfFretX = f => MF_NUT_X + (f - 0.5) * MF_FRET_W
const mfStringY = si => MF_PAD_T + si * MF_STR_H const mfStringY = si => MF_PAD_T + si * MF_STR_H
function MiniFretboard({ values, keyNotes, chordNotes, monoColor = false }) { function MiniFretboard({ values, keyNotes, chordNotes }) {
const max = Math.max(...values, 0.01) const max = Math.max(...values, 0.01)
return ( return (
@@ -188,8 +170,7 @@ function MiniFretboard({ values, keyNotes, chordNotes, monoColor = false }) {
const energy = values[pc] / max const energy = values[pc] / max
const inChord = chordNotes?.has(pc) const inChord = chordNotes?.has(pc)
const inKey = keyNotes?.has(pc) const inKey = keyNotes?.has(pc)
if (!inChord && !inKey && energy < 0.35) return null if (!inChord && !inKey && energy < 0.12) return null
if ((inChord || inKey) && energy < 0.08) return null
const cx = fi === 0 ? MF_OPEN_X : mfFretX(fi) const cx = fi === 0 ? MF_OPEN_X : mfFretX(fi)
const cy = mfStringY(si) const cy = mfStringY(si)
@@ -199,8 +180,8 @@ function MiniFretboard({ values, keyNotes, chordNotes, monoColor = false }) {
fill = `rgba(168,85,247,${0.3 + energy * 0.7})` fill = `rgba(168,85,247,${0.3 + energy * 0.7})`
textFill = '#fff' textFill = '#fff'
} else if (inKey) { } else if (inKey) {
fill = monoColor ? `rgba(192,132,252,${0.2 + energy * 0.75})` : `rgba(245,158,11,${0.2 + energy * 0.75})` fill = `rgba(245,158,11,${0.2 + energy * 0.75})`
textFill = monoColor ? '#fff' : 'rgba(0,0,0,0.85)' textFill = 'rgba(0,0,0,0.85)'
} else { } else {
fill = `rgba(100,100,120,${energy * 0.7})` fill = `rgba(100,100,120,${energy * 0.7})`
textFill = 'rgba(180,180,190,0.7)' textFill = 'rgba(180,180,190,0.7)'
@@ -210,7 +191,7 @@ function MiniFretboard({ values, keyNotes, chordNotes, monoColor = false }) {
<g key={`${si}-${fi}`}> <g key={`${si}-${fi}`}>
{energy > 0.3 && (inChord || inKey) && ( {energy > 0.3 && (inChord || inKey) && (
<circle cx={cx} cy={cy} r={MF_DOT_R + 4} <circle cx={cx} cy={cy} r={MF_DOT_R + 4}
fill={inChord ? 'rgba(168,85,247,0.25)' : monoColor ? 'rgba(192,132,252,0.2)' : 'rgba(245,158,11,0.2)'} fill={inChord ? 'rgba(168,85,247,0.25)' : 'rgba(245,158,11,0.2)'}
style={{ filter: 'blur(4px)' }} /> style={{ filter: 'blur(4px)' }} />
)} )}
<circle cx={cx} cy={cy} r={MF_DOT_R} fill={fill} /> <circle cx={cx} cy={cy} r={MF_DOT_R} fill={fill} />
@@ -225,338 +206,27 @@ function MiniFretboard({ values, keyNotes, chordNotes, monoColor = false }) {
) )
} }
// Oscilloscope strip
const OSC_W = 600
const OSC_H = 110
function Oscilloscope({ waveform }) {
const { wave, rms, detectedFreq, detectedNote } = waveform || {}
const silent = !rms || rms < 0.005
// Note scroll history last 5 distinct notes
const noteHistoryRef = useRef([]) // [{ note, freq, id }, ...] oldest first
const lastNoteRef = useRef(null)
const noteIdRef = useRef(0)
const lastDisplayRef = useRef(null) // last detected note shown in header never flickers
if (detectedNote && detectedNote !== lastNoteRef.current) {
lastNoteRef.current = detectedNote
lastDisplayRef.current = { note: detectedNote, freq: detectedFreq }
noteHistoryRef.current.push({ note: detectedNote, freq: detectedFreq, id: noteIdRef.current++ })
if (noteHistoryRef.current.length > 5) noteHistoryRef.current.shift()
} else if (detectedFreq && detectedNote) {
lastDisplayRef.current = { note: detectedNote, freq: detectedFreq }
}
// Ghost waveform holds the last clear-pitch shape, fades slowly
const ghostRef = useRef({ path: '', fill: '', opacity: 0 })
if (detectedFreq) {
ghostRef.current = { path: '', fill: '', opacity: 1 } // will be filled below
} else {
ghostRef.current = { ...ghostRef.current, opacity: ghostRef.current.opacity * 0.97 }
}
let path = '', sinePath = ''
if (wave?.length) {
const mid = OSC_H / 2
const waveAmp = Math.max(...wave.map(Math.abs), 0.001)
const gain = Math.min((OSC_H * 0.44) / waveAmp, OSC_H * 0.44)
const lo = Math.floor(wave.length / 4)
const hi = Math.floor(wave.length / 2)
let offset = lo
for (let i = lo; i < hi - 1; i++) {
if (wave[i] <= 0 && wave[i + 1] > 0) { offset = i; break }
}
const drawLen = Math.min(wave.length - offset, Math.floor(wave.length * 0.85))
const step = OSC_W / drawLen
path = Array.from({ length: drawLen }, (_, i) => {
const v = wave[offset + i]
return `${i === 0 ? 'M' : 'L'}${(i * step).toFixed(1)},${(mid - v * gain).toFixed(1)}`
}).join(' ')
// Capture ghost path when we have a clear pitch
if (detectedFreq) {
ghostRef.current.path = path
ghostRef.current.fill = path + ` L${OSC_W},${mid} L0,${mid} Z`
}
if (detectedFreq) {
const effectiveSR = 44100 / 8
const sineAmp = Math.min(waveAmp * gain * 0.55, OSC_H * 0.38)
sinePath = Array.from({ length: 300 }, (_, i) => {
const t = i / 299
const x = (t * OSC_W).toFixed(1)
const phase = ((offset + t * drawLen) / effectiveSR) * detectedFreq * Math.PI * 2
const y = (mid - Math.sin(phase) * sineAmp).toFixed(1)
return `${i === 0 ? 'M' : 'L'}${x},${y}`
}).join(' ')
}
}
const lineColor = detectedFreq
? 'rgba(168,85,247,0.9)'
: silent ? 'rgba(50,50,60,0.8)' : 'rgba(100,200,140,0.75)'
const ghost = ghostRef.current
const ghostOp = ghost.opacity
const noteHistory = noteHistoryRef.current
const lastDisplay = lastDisplayRef.current
return (
<div>
<div className="flex items-center justify-between mb-1">
<p className="text-xs text-gray-600 uppercase tracking-widest">Oscilloscope raw mic input</p>
<div className="flex items-center gap-3">
{lastDisplay && (
<>
<span className={`text-xs font-bold ${detectedFreq ? 'text-accent' : 'text-gray-500'}`}>{lastDisplay.note}</span>
<span className="text-xs text-gray-500 tabular-nums">{lastDisplay.freq.toFixed(1)} Hz</span>
<span className="text-xs text-gray-600 tabular-nums">{(1000 / lastDisplay.freq).toFixed(2)} ms / cycle</span>
</>
)}
{silent && <span className="text-xs text-gray-700">silence</span>}
<span className="text-xs text-gray-700 tabular-nums">rms {rms ? (rms * 100).toFixed(1) : '0.0'}%</span>
</div>
</div>
<svg viewBox={`0 0 ${OSC_W} ${OSC_H}`} width="100%" style={{ display: 'block' }}
className="rounded-lg bg-surface border border-border">
{/* Zero line */}
<line x1={0} y1={OSC_H / 2} x2={OSC_W} y2={OSC_H / 2}
stroke="rgba(255,255,255,0.05)" strokeWidth={0.5} />
{/* Ghost waveform — previous clear-pitch shape fading out */}
{ghost.path && ghostOp > 0.04 && !detectedFreq && (
<>
<path d={ghost.fill} fill={`rgba(168,85,247,${(ghostOp * 0.06).toFixed(3)})`} />
<path d={ghost.path} fill="none"
stroke={`rgba(150,120,200,${(ghostOp * 0.35).toFixed(3)})`}
strokeWidth={0.8} strokeLinejoin="round" strokeLinecap="round" />
</>
)}
{/* Fill body */}
{path && (
<path
d={`${path} L${OSC_W},${OSC_H / 2} L0,${OSC_H / 2} Z`}
fill={detectedFreq
? 'rgba(168,85,247,0.08)'
: silent ? 'none' : 'rgba(90,190,130,0.07)'}
/>
)}
{/* Waveform line */}
{path && <path d={path} fill="none" stroke={lineColor} strokeWidth={0.9}
strokeLinejoin="round" strokeLinecap="round" />}
{/* Sine overlay */}
{sinePath && <path d={sinePath} fill="none"
stroke="rgba(168,85,247,0.28)" strokeWidth={0.9}
strokeLinejoin="round" strokeDasharray="5 4" />}
{/* Scrolling note history — newest on right, slides left on each new note */}
{noteHistory.map((entry, i) => {
const age = noteHistory.length - 1 - i // 0 = newest
const x = OSC_W - 28 - age * 100
const op = (1 - age * 0.18).toFixed(2)
const isNew = age === 0
return (
<g key={entry.id}
style={{ transform: `translateX(${x}px)`, transition: 'transform 0.45s cubic-bezier(0.4,0,0.2,1)' }}>
<text x={0} y={OSC_H - 18} textAnchor="middle"
fontSize={isNew ? 13 : 11} fontWeight={isNew ? '700' : '400'}
fill={isNew ? `rgba(168,85,247,${op})` : `rgba(160,130,210,${op})`}>
{entry.note}
</text>
<text x={0} y={OSC_H - 7} textAnchor="middle" fontSize={7}
fill={`rgba(120,100,160,${(parseFloat(op) * 0.7).toFixed(2)})`}>
{entry.freq ? entry.freq.toFixed(0) : ''}Hz
</text>
</g>
)
})}
</svg>
</div>
)
}
// Frequency spectrum
const SPEC_H = 130
const SPEC_F_MIN = 40
const SPEC_F_MAX = 4000
const SPEC_LOG = Math.log(SPEC_F_MAX / SPEC_F_MIN)
// Map a frequency in Hz to an x pixel position (log scale)
function specX(f, w) {
if (f <= SPEC_F_MIN) return 0
if (f >= SPEC_F_MAX) return w
return w * Math.log(f / SPEC_F_MIN) / SPEC_LOG
}
const SPEC_GRID = [
{ label: 'E2', freq: 82.4 },
{ label: 'C3', freq: 130.8 },
{ label: 'E3', freq: 164.8 },
{ label: 'A3', freq: 220 },
{ label: 'C4', freq: 261.6 },
{ label: 'E4', freq: 329.6 },
{ label: 'A4', freq: 440 },
{ label: 'C5', freq: 523.3 },
{ label: 'C6', freq: 1046.5},
{ label: 'C7', freq: 2093 },
]
function SpectrumPanel({ spectrum, detectedFreq }) {
const W = OSC_W
const ghostRef = useRef(null)
const ghostFreqRef = useRef(null) // { freq, opacity }
// Ghost frequency lines lock on detection, decay slowly when gone
if (detectedFreq) {
ghostFreqRef.current = { freq: detectedFreq, opacity: 1 }
} else if (ghostFreqRef.current) {
ghostFreqRef.current = { freq: ghostFreqRef.current.freq, opacity: ghostFreqRef.current.opacity * 0.97 }
}
const ghostFreq = ghostFreqRef.current?.opacity > 0.04 ? ghostFreqRef.current.freq : null
const ghostOpacity = ghostFreqRef.current?.opacity ?? 0
// Ghost: rises instantly with signal, decays very slowly lingers as grey
if (spectrum?.length) {
if (!ghostRef.current) ghostRef.current = new Float32Array(spectrum.length)
const ghost = ghostRef.current
for (let i = 0; i < spectrum.length; i++) {
ghost[i] = spectrum[i] > ghost[i] ? spectrum[i] : ghost[i] * 0.988
}
}
let fillPath = '', strokePath = '', ghostFill = '', ghostStroke = ''
if (spectrum?.length) {
const n = spectrum.length
const pts = Array.from({ length: n }, (_, i) => {
const x = ((i / (n - 1)) * W).toFixed(1)
const y = (SPEC_H * (1 - spectrum[i])).toFixed(1)
return `${i === 0 ? 'M' : 'L'}${x},${y}`
}).join(' ')
strokePath = pts
fillPath = pts + ` L${W},${SPEC_H} L0,${SPEC_H} Z`
const ghost = ghostRef.current
if (ghost) {
const gpts = Array.from({ length: n }, (_, i) => {
const x = ((i / (n - 1)) * W).toFixed(1)
const y = (SPEC_H * (1 - ghost[i])).toFixed(1)
return `${i === 0 ? 'M' : 'L'}${x},${y}`
}).join(' ')
ghostStroke = gpts
ghostFill = gpts + ` L${W},${SPEC_H} L0,${SPEC_H} Z`
}
}
return (
<div>
<p className="text-xs text-gray-600 uppercase tracking-widest mb-1">
Frequency spectrum 40 Hz 4 kHz (log scale)
</p>
<svg viewBox={`0 0 ${W} ${SPEC_H}`} width="100%" style={{ display: 'block' }}
className="rounded-lg bg-surface border border-border">
{/* Note grid lines */}
{SPEC_GRID.map(({ label, freq }) => {
const x = specX(freq, W).toFixed(1)
return (
<g key={label}>
<line x1={x} y1={0} x2={x} y2={SPEC_H - 14}
stroke="rgba(255,255,255,0.06)" strokeWidth={1} />
<text x={x} y={SPEC_H - 3} textAnchor="middle" fontSize={7.5}
fill="rgba(80,80,95,0.9)">{label}</text>
</g>
)
})}
{/* Ghost — slow-decaying grey residue from previous peaks */}
{ghostFill && (
<>
<path d={ghostFill} fill="rgba(120,120,130,0.08)" />
<path d={ghostStroke} fill="none" stroke="rgba(130,130,145,0.30)" strokeWidth={0.7} />
</>
)}
{/* Live spectrum fill + stroke */}
{fillPath && (
<>
<path d={fillPath} fill="rgba(80,180,130,0.13)" />
<path d={strokePath} fill="none" stroke="rgba(90,200,145,0.55)" strokeWidth={0.8} />
</>
)}
{/* Fundamental + harmonics */}
{ghostFreq && [1, 2, 3, 4, 5].map(h => {
const hf = ghostFreq * h
if (hf > SPEC_F_MAX) return null
const xNum = specX(hf, W)
const x = xNum.toFixed(1)
const midi = Math.round(12 * Math.log2(hf / 440) + 69)
const note = NOTES[((midi % 12) + 12) % 12]
const oct = Math.floor(midi / 12) - 1
// Place label left of line near the right edge, right of line elsewhere
const labelX = xNum > W - 40 ? xNum - 3 : xNum + 3
const anchor = xNum > W - 40 ? 'end' : 'start'
if (h === 1) {
const op = (0.9 * ghostOpacity).toFixed(3)
const textOp = (ghostOpacity * 0.95).toFixed(3)
return (
<g key={h}>
<line x1={x} y1={0} x2={x} y2={SPEC_H - 14}
stroke={`rgba(168,85,247,${op})`} strokeWidth={1.2} />
<text x={labelX} y={10} textAnchor={anchor} fontSize={8} fontWeight="700"
fill={`rgba(168,85,247,${textOp})`}>{note}{oct}</text>
<text x={labelX} y={20} textAnchor={anchor} fontSize={7}
fill={`rgba(168,85,247,${(ghostOpacity * 0.55).toFixed(3)})`}>f</text>
</g>
)
}
const op = ((0.5 - (h - 2) * 0.1) * ghostOpacity).toFixed(3)
return (
<g key={h}>
<line x1={x} y1={0} x2={x} y2={SPEC_H - 14}
stroke={`rgba(168,85,247,${op})`} strokeWidth={0.7} strokeDasharray="3 4" />
<text x={labelX} y={10} textAnchor={anchor} fontSize={7.5}
fill={`rgba(168,85,247,${op})`}>{note}{oct}</text>
<text x={labelX} y={19} textAnchor={anchor} fontSize={7}
fill={`rgba(168,85,247,${(parseFloat(op) * 0.7).toFixed(3)})`}>{h}f</text>
</g>
)
})}
</svg>
</div>
)
}
// Main component // Main component
export default function DebugView({ chroma, chordCandidates, noteAnalysis, waveform, keyInfo, currentChord, instrument = 'guitar', monoColor = false }) { export default function DebugView({ chroma, chordCandidates, noteAnalysis, keyInfo, currentChord, instrument = 'guitar' }) {
const keyPCs = new Set(keyInfo ? getScale(keyInfo.root, keyInfo.mode).map(n => NOTES.indexOf(n)) : []) const keyPCs = new Set(keyInfo ? getScale(keyInfo.root, keyInfo.mode).map(n => NOTES.indexOf(n)) : [])
const chordPCs = new Set(currentChord ? getChordTones(currentChord).map(n => NOTES.indexOf(n)) : []) const chordPCs = new Set(currentChord ? getChordTones(currentChord).map(n => NOTES.indexOf(n)) : [])
const chromaArr = chroma ? [...chroma] : new Array(12).fill(0) const chromaArr = chroma ? [...chroma] : new Array(12).fill(0)
const histFreq = noteAnalysis ? noteAnalysis.freq : new Array(12).fill(0) const histFreq = noteAnalysis ? noteAnalysis.freq : new Array(12).fill(0)
const topKeys = noteAnalysis ? noteAnalysis.topKeys : [] const topKeys = noteAnalysis ? noteAnalysis.topKeys : []
const totalNotes = noteAnalysis?.total ?? 0
const sessionSecs = noteAnalysis?.sessionSecs ?? 0
const sessionLabel = sessionSecs >= 60
? `${Math.floor(sessionSecs / 60)}m ${sessionSecs % 60}s`
: `${sessionSecs}s`
const topScore = chordCandidates[0]?.score ?? 1 const topScore = chordCandidates[0]?.score ?? 1
const topKeyScore = topKeys[0]?.score ?? 1 const topKeyScore = topKeys[0]?.score ?? 1
return ( return (
<div className="flex flex-col gap-4"> <div className="bg-panel border border-border rounded-2xl p-4 flex flex-col gap-4">
{/* ── Live chroma visualization (instrument-synced) ── */} <p className="text-xs text-gray-500 uppercase tracking-widest shrink-0">Behind the Scenes</p>
{/* ── Live chroma visualization (instrument-synced) ── */}
<div> <div>
<p className="text-xs text-gray-600 uppercase tracking-widest mb-2">Live chroma what the engine hears right now</p> <p className="text-xs text-gray-600 uppercase tracking-widest mb-2">Live chroma what the engine hears right now</p>
{instrument === 'guitar' {instrument === 'guitar'
? <MiniFretboard values={chromaArr} keyNotes={keyPCs} chordNotes={chordPCs} monoColor={monoColor} /> ? <MiniFretboard values={chromaArr} keyNotes={keyPCs} chordNotes={chordPCs} />
: <PianoSVG values={chromaArr} keyNotes={keyPCs} chordNotes={chordPCs} keyH={90} monoColor={monoColor} /> : <PianoSVG values={chromaArr} keyNotes={keyPCs} chordNotes={chordPCs} keyH={90} />
} }
</div> </div>
@@ -599,15 +269,8 @@ export default function DebugView({ chroma, chordCandidates, noteAnalysis, wavef
{/* Col 2: Note history piano with % labels */} {/* Col 2: Note history piano with % labels */}
<div> <div>
<div className="flex items-baseline justify-between mb-2"> <p className="text-xs text-gray-600 uppercase tracking-widest mb-2">Note history key evidence</p>
<p className="text-xs text-gray-600 uppercase tracking-widest">Note history</p> <PianoSVG values={histFreq} keyNotes={keyPCs} chordNotes={chordPCs} keyH={70} showPct={true} />
{totalNotes > 0 && (
<span className="text-[10px] text-gray-600 tabular-nums">
{totalNotes.toLocaleString()} notes · {sessionLabel}
</span>
)}
</div>
<PianoSVG values={histFreq} keyNotes={keyPCs} chordNotes={chordPCs} keyH={70} showPct={true} monoColor={monoColor} />
</div> </div>
{/* Col 3: Key candidates */} {/* Col 3: Key candidates */}
@@ -635,12 +298,6 @@ export default function DebugView({ chroma, chordCandidates, noteAnalysis, wavef
</div> </div>
</div> </div>
{/* ── Oscilloscope + spectrum ── */}
<div className="flex flex-col gap-3">
<Oscilloscope waveform={waveform} />
<SpectrumPanel spectrum={waveform?.spectrum} detectedFreq={waveform?.detectedFreq} />
</div>
</div> </div>
) )
} }
-181
View File
@@ -1,181 +0,0 @@
import { useRef, useEffect } from 'react'
// Map a frequency (Hz) to a bin index in the 256-bin log spectrum (404000 Hz)
function freqToBin(freq) {
return Math.round(255 * Math.log(freq / 40) / Math.log(4000 / 40))
}
function bandMax(spectrum, lo, hi) {
if (!spectrum) return 0
const a = freqToBin(lo)
const b = Math.min(freqToBin(hi), spectrum.length - 1)
let max = 0
for (let i = a; i <= b; i++) if (spectrum[i] > max) max = spectrum[i]
return max
}
const BANDS = [
{ label: 'Kick', lo: 40, hi: 120, color: '#ef4444' },
{ label: 'Snare', lo: 120, hi: 300, color: '#f59e0b' },
{ label: 'Mid', lo: 300, hi: 1000, color: '#22c55e' },
{ label: 'Presence', lo: 1000, hi: 4000, color: '#60a5fa' },
]
const TIMELINE_MS = 4000 // onset timeline window
const RMS_HISTORY = 180 // ~3s at 60fps
export default function DrumView({ waveform, bpm }) {
const rmsHistRef = useRef([])
const beatCanvasRef = useRef(null)
const rmsCanvasRef = useRef(null)
const spectrum = waveform?.spectrum ?? null
const bandLevels = BANDS.map(b => bandMax(spectrum, b.lo, b.hi))
// Accumulate RMS history
useEffect(() => {
if (waveform == null) return
const h = rmsHistRef.current
h.push(Math.min(waveform.rms * 10, 1))
if (h.length > RMS_HISTORY) h.shift()
}, [waveform])
// Draw onset / beat timeline
useEffect(() => {
const canvas = beatCanvasRef.current
if (!canvas) return
const ctx = canvas.getContext('2d')
const W = canvas.width, H = canvas.height
ctx.fillStyle = '#0a0a0a'
ctx.fillRect(0, 0, W, H)
const onsets = waveform?.onsets ?? []
const now = performance.now()
// Beat grid aligned to the most recent onset
if (bpm) {
const beatMs = 60000 / bpm
const numBeats = Math.ceil(TIMELINE_MS / beatMs) + 1
const latest = onsets[onsets.length - 1]
const phase = latest != null ? (now - latest) % beatMs : 0
for (let b = 0; b <= numBeats; b++) {
const ageMs = b * beatMs - phase
if (ageMs < 0 || ageMs > TIMELINE_MS) continue
const x = W * (1 - ageMs / TIMELINE_MS)
ctx.strokeStyle = 'rgba(255,255,255,0.07)'
ctx.lineWidth = 1
ctx.beginPath(); ctx.moveTo(x, 0); ctx.lineTo(x, H); ctx.stroke()
}
}
// Onset dots + vertical tails
const recent = onsets.filter(t => now - t <= TIMELINE_MS)
for (const t of recent) {
const age = now - t
const x = W * (1 - age / TIMELINE_MS)
const alpha = Math.pow(1 - age / TIMELINE_MS, 0.4)
ctx.strokeStyle = `rgba(168,85,247,${(alpha * 0.35).toFixed(2)})`
ctx.lineWidth = 1
ctx.beginPath(); ctx.moveTo(x, 0); ctx.lineTo(x, H); ctx.stroke()
ctx.fillStyle = `rgba(168,85,247,${alpha.toFixed(2)})`
ctx.beginPath(); ctx.arc(x, H / 2, 5, 0, Math.PI * 2); ctx.fill()
}
// "Now" edge
ctx.strokeStyle = 'rgba(255,255,255,0.18)'
ctx.lineWidth = 2
ctx.beginPath(); ctx.moveTo(W - 1, 0); ctx.lineTo(W - 1, H); ctx.stroke()
}, [waveform, bpm])
// Draw RMS envelope
useEffect(() => {
const canvas = rmsCanvasRef.current
if (!canvas) return
const ctx = canvas.getContext('2d')
const W = canvas.width, H = canvas.height
const h = rmsHistRef.current
ctx.fillStyle = '#0a0a0a'
ctx.fillRect(0, 0, W, H)
if (h.length < 2) return
const barW = W / RMS_HISTORY
for (let i = 0; i < h.length; i++) {
const x = W * (i / RMS_HISTORY)
const barH = h[i] * H
const v = Math.round(h[i] * 160 + 60)
ctx.fillStyle = `rgb(${v},30,${v})`
ctx.fillRect(x, H - barH, Math.max(barW - 0.5, 1), barH)
}
}, [waveform])
const noData = !waveform
return (
<div className="space-y-5">
{/* Band meters */}
<div>
<p className="text-xs text-gray-600 font-mono uppercase tracking-widest mb-2">Frequency Bands</p>
<div className="flex gap-3" style={{ height: 96 }}>
{BANDS.map((b, i) => (
<div key={b.label} className="flex flex-col items-center gap-1 flex-1">
<div className="flex-1 w-full bg-gray-900 rounded-sm relative overflow-hidden">
{noData ? (
<div className="absolute inset-0 flex items-center justify-center">
<span className="text-[9px] text-gray-700 font-mono"></span>
</div>
) : (
<div
className="absolute bottom-0 left-0 right-0 rounded-sm"
style={{
height: `${bandLevels[i] * 100}%`,
backgroundColor: b.color,
transition: 'height 60ms linear',
}}
/>
)}
</div>
<span className="text-[10px] font-mono text-gray-500 uppercase">{b.label}</span>
</div>
))}
</div>
</div>
{/* Onset timeline */}
<div>
<p className="text-xs text-gray-600 font-mono uppercase tracking-widest mb-2">
Onset Timeline{bpm ? ` · ${bpm} BPM` : ''}
<span className="ml-2 text-gray-700 normal-case"> 4 seconds</span>
</p>
<canvas
ref={beatCanvasRef}
width={800}
height={56}
className="w-full rounded"
style={{ height: 56 }}
/>
{noData && (
<p className="text-xs text-gray-700 font-mono mt-1 text-center">Start listening to see hits</p>
)}
</div>
{/* Volume envelope */}
<div>
<p className="text-xs text-gray-600 font-mono uppercase tracking-widest mb-2">
Volume Envelope
<span className="ml-2 text-gray-700 normal-case"> ~3 seconds</span>
</p>
<canvas
ref={rmsCanvasRef}
width={800}
height={56}
className="w-full rounded"
style={{ height: 56 }}
/>
</div>
</div>
)
}
+2 -2
View File
@@ -38,7 +38,7 @@ export default function ProgressionBanner({ chordHistory, keyInfo, detectedProgr
}, [current]) }, [current])
return ( return (
<div className="bg-panel border border-border rounded-2xl p-4 mb-3 flex flex-col-reverse lg:flex-row gap-4"> <div className="bg-panel border border-border rounded-2xl p-4 mb-3 flex gap-4">
{/* ── Left: key + chord history + loop ── */} {/* ── Left: key + chord history + loop ── */}
<div className="w-full lg:w-[70%] min-w-0 flex flex-col gap-2"> <div className="w-full lg:w-[70%] min-w-0 flex flex-col gap-2">
@@ -130,7 +130,7 @@ export default function ProgressionBanner({ chordHistory, keyInfo, detectedProgr
<div className="hidden lg:block w-px bg-border shrink-0" /> <div className="hidden lg:block w-px bg-border shrink-0" />
{/* ── Right: big chord ── */} {/* ── Right: big chord ── */}
<div className="flex w-full lg:w-[30%] flex-col items-center justify-center gap-1 py-2 lg:py-0"> <div className="hidden lg:flex w-[30%] flex-col items-center justify-center gap-1">
{current ? ( {current ? (
<> <>
<p className="text-xs text-gray-600 uppercase tracking-widest">Now Playing</p> <p className="text-xs text-gray-600 uppercase tracking-widest">Now Playing</p>
+33 -76
View File
@@ -1,5 +1,3 @@
import { useRef, useEffect, useState } from 'react'
const SETTINGS = [ const SETTINGS = [
{ {
section: 'Chord Detection', section: 'Chord Detection',
@@ -72,52 +70,20 @@ const SETTINGS = [
}, },
] ]
export default function Settings({ config, onChange, onClose, onReset, monoColor, onMonoColorChange }) { const TOGGLES = [
// Snapshot on mount so Cancel can restore {
const savedConfig = useRef(config) section: 'Experimental',
const savedMono = useRef(monoColor) items: [
{
function handleCancel() { key: 'useEssentia',
Object.entries(savedConfig.current).forEach(([k, v]) => onChange(k, v)) label: 'Essentia HPCP',
onMonoColorChange(savedMono.current) desc: 'Replace custom chroma with Essentia\'s Harmonic Pitch Class Profile (spectral peaks + 8 harmonics). More accurate on polyphonic input. May be slower.',
onClose() },
} ],
},
function DeviceSelector({ config, onChange }) { ]
const [devices, setDevices] = useState([])
async function refresh() {
try {
if (!navigator.mediaDevices || !navigator.mediaDevices.enumerateDevices) return
const list = await navigator.mediaDevices.enumerateDevices()
setDevices(list.filter(d => d.kind === 'audioinput'))
} catch (e) {
console.warn('enumerateDevices failed', e)
}
}
useEffect(() => { refresh() }, [])
return (
<div className="space-y-3">
<div className="flex gap-3 items-center">
<select
value={config.audioDeviceId ?? ''}
onChange={e => onChange('audioDeviceId', e.target.value === '' ? null : e.target.value)}
className="appearance-none bg-surface border border-border hover:border-gray-500 focus:border-accent focus:outline-none rounded-lg pl-3 pr-7 py-1 text-sm text-gray-200 cursor-pointer transition-colors w-full"
>
<option value="">System default</option>
{devices.map((d, i) => (
<option key={d.deviceId || i} value={d.deviceId}>{d.label || `Microphone ${i + 1}`}</option>
))}
</select>
<button onClick={refresh} className="px-3 py-1 rounded-lg border border-border text-sm text-gray-400">Refresh</button>
</div>
<div className="text-xs text-gray-600">If device labels are empty, grant microphone permission first and hit Refresh.</div>
</div>
)
}
export default function Settings({ config, onChange, onClose, onReset }) {
return ( return (
<div className="fixed inset-0 bg-surface z-50 overflow-y-auto"> <div className="fixed inset-0 bg-surface z-50 overflow-y-auto">
<div className="max-w-2xl mx-auto px-6 py-8"> <div className="max-w-2xl mx-auto px-6 py-8">
@@ -134,42 +100,41 @@ export default function Settings({ config, onChange, onClose, onReset, monoColor
> >
Reset defaults Reset defaults
</button> </button>
<button
onClick={handleCancel}
className="px-4 py-2 rounded-lg text-sm border border-border text-gray-500 hover:text-gray-300 hover:border-gray-400 transition-all"
>
Cancel
</button>
<button <button
onClick={onClose} onClick={onClose}
className="px-5 py-2 rounded-lg text-sm bg-accent hover:bg-purple-600 text-white font-semibold transition-all" className="px-5 py-2 rounded-lg text-sm bg-accent hover:bg-purple-600 text-white font-semibold transition-all"
> >
Save Done
</button> </button>
</div> </div>
</div> </div>
<div className="space-y-8"> <div className="space-y-8">
{TOGGLES.map(section => (
{/* ── Display ── */} <div key={section.section}>
<div>
<h3 className="text-xs uppercase tracking-widest text-gray-500 mb-4 border-b border-border pb-2"> <h3 className="text-xs uppercase tracking-widest text-gray-500 mb-4 border-b border-border pb-2">
Display {section.section}
</h3> </h3>
<div className="flex items-center justify-between"> <div className="space-y-4">
{section.items.map(item => (
<label key={item.key} className="flex items-start gap-3 cursor-pointer group">
<input
type="checkbox"
checked={!!config[item.key]}
onChange={e => onChange(item.key, e.target.checked)}
className="mt-0.5 accent-purple-500 w-4 h-4 flex-shrink-0"
/>
<div> <div>
<p className="text-sm font-semibold text-gray-200">Mono Color Mode</p> <div className="text-sm font-semibold text-gray-200 group-hover:text-white transition-colors">
<p className="text-xs text-gray-600 mt-0.5">Use a single purple palette instead of purple + amber for note tiers.</p> {item.label}
</div> </div>
<button <div className="text-xs text-gray-600 mt-0.5">{item.desc}</div>
onClick={() => onMonoColorChange(v => !v)} </div>
className={`relative w-11 h-6 rounded-full transition-colors ${monoColor ? 'bg-accent' : 'bg-gray-700'}`} </label>
> ))}
<span className={`absolute top-0.5 left-0.5 w-5 h-5 rounded-full bg-white shadow transition-transform ${monoColor ? 'translate-x-5' : 'translate-x-0'}`} />
</button>
</div> </div>
</div> </div>
))}
{SETTINGS.map(section => ( {SETTINGS.map(section => (
<div key={section.section}> <div key={section.section}>
<h3 className="text-xs uppercase tracking-widest text-gray-500 mb-4 border-b border-border pb-2"> <h3 className="text-xs uppercase tracking-widest text-gray-500 mb-4 border-b border-border pb-2">
@@ -210,14 +175,6 @@ export default function Settings({ config, onChange, onClose, onReset, monoColor
</div> </div>
</div> </div>
))} ))}
{/* Audio device selector */}
<div>
<h3 className="text-xs uppercase tracking-widest text-gray-500 mb-4 border-b border-border pb-2">
Microphone
</h3>
<DeviceSelector config={config} onChange={onChange} />
</div>
</div> </div>
</div> </div>
+5 -2
View File
@@ -108,8 +108,10 @@ export default function Tuner() {
return ( return (
<div className="p-6 text-center"> <div className="p-6 bg-panel border border-border rounded-xl text-center">
<div className="flex items-start justify-end mb-4"> <div className="flex items-start justify-between mb-4">
<h3 className="text-lg font-semibold">Tuner</h3>
<div>
<button <button
onClick={() => (isListening ? stopListening() : startListening())} onClick={() => (isListening ? stopListening() : startListening())}
className={`px-4 py-2 rounded-full text-sm font-semibold ${isListening ? 'bg-red-600' : 'bg-accent'}`} className={`px-4 py-2 rounded-full text-sm font-semibold ${isListening ? 'bg-red-600' : 'bg-accent'}`}
@@ -117,6 +119,7 @@ export default function Tuner() {
{isListening ? 'Stop' : 'Start'} {isListening ? 'Stop' : 'Start'}
</button> </button>
</div> </div>
</div>
<div className="w-full flex flex-col items-center"> <div className="w-full flex flex-col items-center">
<div className="w-full max-w-3xl"> <div className="w-full max-w-3xl">
-11
View File
@@ -5,15 +5,4 @@ body {
background-color: #0f0f0f; background-color: #0f0f0f;
color: #f5f5f5; color: #f5f5f5;
font-family: system-ui, -apple-system, sans-serif; font-family: system-ui, -apple-system, sans-serif;
/* ponytail: mobile PWA full-bleed bg with content cleared from notch / home indicator. viewport-fit=cover lets bg extend under insets; app-shell pads content back in. */
overscroll-behavior: none;
-webkit-tap-highlight-color: transparent;
}
.app-shell {
padding:
max(0.75rem, env(safe-area-inset-top))
max(0.75rem, env(safe-area-inset-right))
max(0.75rem, env(safe-area-inset-bottom))
max(0.75rem, env(safe-area-inset-left));
} }
+98
View File
@@ -0,0 +1,98 @@
// Essentia HPCP pipeline — replaces our custom computeChroma() when enabled.
//
// Pipeline (using Web Audio FFT data directly to skip WASM re-FFT):
// freqBuf (dB from AnalyserNode) → linear magnitude → SpectralPeaks → HPCP
//
// Pitch class ordering:
// Essentia HPCP[0] = A (referenceFrequency=440)
// Our chroma[0] = C
// Rotation applied: ourChroma[(i + 9) % 12] = HPCP[i]
import { EssentiaWASM } from 'essentia.js/dist/essentia-wasm.es.js'
import Essentia from 'essentia.js/dist/essentia.js-core.es.js'
let instance = null
// Call once at startup — safe to call multiple times (returns cached instance).
export async function initEssentia() {
if (instance) return instance
console.log('[Essentia] loading WASM…')
await EssentiaWASM['ready']
instance = new Essentia(EssentiaWASM)
console.log('[Essentia] ready —', instance.version)
return instance
}
export function getEssentia() { return instance }
const NOISE_FLOOR_DB = -65
// Compute 12-bin HPCP from Web Audio frequency-domain data.
//
// freqData — Float32Array from AnalyserNode.getFloatFrequencyData() (dB values)
// sampleRate — AudioContext.sampleRate
//
// Returns Float32Array(12) with same [C, C#, D, …, B] ordering as computeChroma().
export function computeHPCP(essentia, freqData, sampleRate) {
const N = freqData.length
// Convert dB → linear magnitude. Bins below noise floor stay 0.
const magSpectrum = new Float32Array(N)
for (let i = 0; i < N; i++) {
if (freqData[i] > NOISE_FLOOR_DB) {
magSpectrum[i] = Math.pow(10, freqData[i] / 20)
}
}
const specVec = essentia.arrayToVector(magSpectrum)
// SpectralPeaks derives bin → Hz as: freq = binIndex * sampleRate / (2*(N-1))
// which matches Web Audio's FFT bin spacing (sampleRate / fftSize).
const peaks = essentia.SpectralPeaks(
specVec,
0, // magnitudeThreshold — noise already zeroed above
4000, // maxFrequency (Hz)
60, // maxPeaks
40, // minFrequency (Hz)
'byMagnitude',
sampleRate
)
specVec.delete()
// If no peaks found (silence / below noise floor), return zeros.
if (peaks.frequencies.size() === 0) {
peaks.frequencies.delete()
peaks.magnitudes.delete()
return new Float32Array(12)
}
const hpcpResult = essentia.HPCP(
peaks.frequencies,
peaks.magnitudes,
false, // bandPreset
500, // bandSplitFrequency (unused when bandPreset=false)
8, // harmonics
4000, // maxFrequency (Hz)
false, // maxShifted
40, // minFrequency (Hz)
false, // nonLinear
'unitMax', // normalized
440, // referenceFrequency (A4 = 440 Hz → HPCP[0] = A)
sampleRate,
12, // size (bins per octave)
'squaredCosine', // weightType
0.5 // windowSize (octaves)
)
peaks.frequencies.delete()
peaks.magnitudes.delete()
// Rotate A-origin → C-origin to match our chroma convention.
// HPCP[0]=A → ourChroma[9]=A, so ourChroma[(i+9)%12] = HPCP[i]
const result = new Float32Array(12)
for (let i = 0; i < 12; i++) {
result[(i + 9) % 12] = hpcpResult.hpcp.get(i)
}
hpcpResult.hpcp.delete()
return result
}
+17 -50
View File
@@ -360,72 +360,40 @@ export function toRomanNumeral(chordName, keyRoot, keyMode) {
// ─── Repeating progression detection ───────────────────────────────────────── // ─── Repeating progression detection ─────────────────────────────────────────
// Returns true if arr is made of a shorter repeating unit (e.g. [A,B,A,B] → true)
function isPeriodicPattern(arr) {
for (let p = 1; p <= Math.floor(arr.length / 2); p++) {
if (arr.length % p !== 0) continue
const unit = arr.slice(0, p)
if (arr.every((v, i) => v === unit[i % p])) return true
}
return false
}
// Returns the lexicographically smallest rotation so the same loop always
// produces the same string regardless of where in the cycle we currently are.
function canonicalize(pattern) {
let best = pattern
for (let i = 1; i < pattern.length; i++) {
const rot = [...pattern.slice(i), ...pattern.slice(0, i)]
if (rot.join('\0') < best.join('\0')) best = rot
}
return best
}
/** /**
* detectRepeatingProgression(history) chord[] or null * detectRepeatingProgression(history) chord[] or null
* * Returns the most-recently-completed repeating pattern (length 26).
* Tests every unique subsequence of every length (not just the tail) so the * Uses non-overlapping match counting to avoid over-counting.
* result is stable regardless of where in the loop the musician currently is.
* Returns the canonical (rotation-normalised) form of the best pattern found.
*/ */
export function detectRepeatingProgression(history) { export function detectRepeatingProgression(history) {
if (!history || history.length < 6) return null if (!history || history.length < 4) return null
const win = history.slice(-32) const window = history.slice(-20)
let best = null, bestScore = 0 let best = null, bestScore = 0
for (let len = 2; len <= 6; len++) { for (let len = 2; len <= 6; len++) {
if (len * 2 > win.length) break if (len * 2 > window.length) break
const seen = new Set()
for (let start = 0; start <= win.length - len; start++) {
const candidate = win.slice(start, start + len)
const key = candidate.join('\0')
if (seen.has(key)) continue
seen.add(key)
// A pattern that is itself a repetition of something shorter will be
// found at that shorter length — skip it here to avoid inflating scores.
if (len >= 4 && isPeriodicPattern(candidate)) continue
const candidate = window.slice(-len)
let reps = 0, i = 0 let reps = 0, i = 0
while (i <= win.length - len) {
if (candidate.every((c, j) => c === win[i + j])) { reps++; i += len } while (i <= window.length - len) {
else i++ if (candidate.every((c, j) => c === window[i + j])) {
reps++
i += len // skip past match — non-overlapping
} else {
i++
}
} }
if (reps < 2) continue const score = reps * len
if (reps >= 2 && score > bestScore) {
const score = reps * len * len // square length — prevents sub-patterns from beating full loop
if (score > bestScore) {
bestScore = score bestScore = score
best = candidate best = candidate
} }
} }
}
return best ? canonicalize(best) : null return best
} }
// ─── Debug / analysis helpers ───────────────────────────────────────────────── // ─── Debug / analysis helpers ─────────────────────────────────────────────────
@@ -482,7 +450,6 @@ export function getNoteHistoryAnalysis(noteHistory) {
} }
return { return {
freq: normalized, freq: normalized,
total,
topKeys: candidates.sort((a, b) => b.score - a.score).slice(0, 5), topKeys: candidates.sort((a, b) => b.score - a.score).slice(0, 5),
} }
} }
+1 -11
View File
@@ -78,17 +78,7 @@ export function useAudioTuner() {
const startListening = async () => { const startListening = async () => {
if (isListening) return if (isListening) return
try { try {
// Read saved device selection from config (if present) const stream = await navigator.mediaDevices.getUserMedia({ audio: true })
let deviceId = null
try {
const cfg = JSON.parse(localStorage.getItem('wtf_config') || '{}')
deviceId = cfg.audioDeviceId || null
} catch (e) {
// ignore
}
const constraints = deviceId ? { audio: { deviceId: { exact: deviceId } } } : { audio: true }
console.log('Tuner requesting getUserMedia with', constraints)
const stream = await navigator.mediaDevices.getUserMedia(constraints)
const ctx = new (window.AudioContext || window.webkitAudioContext)() const ctx = new (window.AudioContext || window.webkitAudioContext)()
const analyser = ctx.createAnalyser() const analyser = ctx.createAnalyser()
analyser.fftSize = 4096 analyser.fftSize = 4096
+6 -29
View File
@@ -1,37 +1,9 @@
import { defineConfig } from 'vite' import { defineConfig } from 'vite'
import react from '@vitejs/plugin-react' import react from '@vitejs/plugin-react'
import tailwindcss from '@tailwindcss/vite' import tailwindcss from '@tailwindcss/vite'
import { VitePWA } from 'vite-plugin-pwa'
export default defineConfig({ export default defineConfig({
plugins: [ plugins: [react(), tailwindcss()],
react(),
tailwindcss(),
VitePWA({
registerType: 'autoUpdate',
includeAssets: ['apple-touch-icon.png', 'icon-192.png', 'icon-512.png'],
manifest: {
name: 'JamBuddy — WhatTheFlat',
short_name: 'JamBuddy',
description: 'Real-time key and chord detection for live jams.',
theme_color: '#0f0f0f',
background_color: '#0f0f0f',
display: 'standalone',
orientation: 'any',
start_url: './',
scope: './',
icons: [
{ src: 'icon-192.png', sizes: '192x192', type: 'image/png', purpose: 'any' },
{ src: 'icon-512.png', sizes: '512x512', type: 'image/png', purpose: 'any' },
{ src: 'icon-512-maskable.png', sizes: '512x512', type: 'image/png', purpose: 'maskable' },
],
},
workbox: {
globPatterns: ['**/*.{js,css,html,svg,png,ico,woff2,webmanifest}'],
navigateFallback: 'index.html',
},
}),
],
base: './', // relative paths so Electron can load files from disk base: './', // relative paths so Electron can load files from disk
server: { server: {
port: 5173, port: 5173,
@@ -39,4 +11,9 @@ export default defineConfig({
build: { build: {
outDir: 'dist', outDir: 'dist',
}, },
optimizeDeps: {
// Essentia uses emscripten output that esbuild can't pre-bundle reliably.
// Exclude it so Vite serves the files directly from node_modules.
exclude: ['essentia.js'],
},
}) })