Note2MIDI

A free, local-first sheet music to MIDI converter with optional AI-assisted error correction.

Note2MIDI accepts printed sheet-music PDFs and images, runs Audiveris on the user's computer, validates the resulting MusicXML, exposes every recognized note in an editable piano roll, and exports a real Standard MIDI File. No account, API key, or cloud service is needed for the normal workflow.

AI Assist is deliberately secondary. When enabled, it sends only a user-selected image crop and the corresponding structured OMR measure to Gemini. Gemini returns schema-validated suggestions; it never silently rewrites the score.

What works

• PDF, PNG, JPG/JPEG, TIFF, and BMP upload with file-signature and size validation
• Audiveris batch OMR, including multi-page documents and compressed .mxl output
• MusicXML parsing for parts, staves, measures, voices, notes, rests, chords, accidentals, tempo, time/key signatures, ties, and page boundaries
• Structural validation for pitch/MIDI range, durations, references, measure timing, signatures, and inappropriate same-pitch overlaps
• Piano-roll editing: select, drag, resize, add, delete, repitch, change octave/accidental/duration/start time, move between staves, edit chords, add/remove tracks, and change tempo
• In-browser preview playback
• Multi-track MIDI export with chords, tempo, timing, velocity, programs, and merged tie chains
• MIDI round-trip verification with @tonejs/midi before download
• Optional Gemini review of one selected region and one selected measure, with accept/reject/manual-edit workflow
• Responsive, keyboard-accessible controls and explicit loading/error/empty states

Architecture

PDF / image
    │
    ▼
local bridge (loopback only by default)
    │
    ▼
Audiveris 5.11 ──► MusicXML / MXL
    │
    ▼
validated StructuredScore
    │
    ├──► editable piano roll ──► playback
    │
    ├──► Standard MIDI File ──► parser verification ──► download
    │
    └──► optional selected crop + selected measure
                   │
                   ▼
             Gemini suggestion
                   │
                   ▼
          user accepts, rejects, or edits

The UI and local bridge are separate processes. The bridge listens on 127.0.0.1:4318, creates a private temporary directory for each scan, invokes Audiveris without a shell, reads its MusicXML, and removes the temporary input/output afterward. The browser never receives the Gemini API key from the bridge.

Requirements

• Node.js 22.13 or newer
• npm
• Audiveris 5.11.0, or a compatible recent release

The official Audiveris desktop installers are the simplest route on macOS, Windows, and Linux. They include the application runtime. Tesseract language data improves lyrics/title recognition but is not required for musical symbols.

Install and run

git clone https://github.com/Repla09/note2midi.git
cd note2midi
npm install
cp .env.example .env
npm run dev

Open http://localhost:3000. npm run dev starts both the web interface and the local bridge.

Note2MIDI searches for audiveris on PATH and the standard macOS application location. If detection fails, set the exact executable path in .env:

AUDIVERIS_COMMAND=/Applications/Audiveris.app/Contents/MacOS/Audiveris

Windows example:

AUDIVERIS_COMMAND=C:\Program Files\Audiveris\Audiveris.exe

Use the status card in the left rail to confirm the detected engine and version. Audiveris can take tens of seconds per page; that is normal for full OMR.

Production-style local run

npm run build
npm start

The app intentionally does not download or redistribute Audiveris during npm install. Its platform-specific runtime, model assets, and AGPL license deserve an explicit installation step rather than a hidden postinstall download.

Typical workflow

1. Drop a printed score PDF or image into the left rail.
2. Leave Local OMR selected and click Scan locally.
3. Review validation warnings and the recognized notes.
4. Drag or resize notes in the piano roll, double-click to add a note, or use the inspector for exact edits.
5. Press Play to audition the result.
6. Export the validated .mid file. You can also download the original recognized MusicXML.

Validation warnings do not block export; structural errors do. A short or overfull measure is intentionally reported as suspicious instead of being silently padded or rewritten.

Optional Gemini AI Assist

Local OMR remains the first step even in AI Assist mode.

The default model is gemini-3.7-flash, a current multimodal understanding model that accepts images/PDFs and structured output. The integration uses Google's current @google/genai SDK and Interactions API, with store: false and a strict JSON schema. The model is configured in one place and can be overridden with GEMINI_MODEL or in Settings.

Configure a key in either of two ways:

GEMINI_API_KEY=your-own-key
GEMINI_MODEL=gemini-3.7-flash

Or enter the key in the in-app Settings dialog. A dialog-entered key is held only in the local bridge's process memory and is forgotten when that process stops. It is never returned to the browser, written by the app, or logged.

To request a review:

1. Drag over the difficult measure in the original-score preview.
2. Select the matching structured measure in the AI panel.
3. Request analysis.
4. Review the reason and confidence for each proposed change.
5. Accept the whole suggestion, reject it, or edit the affected notes manually.

The payload contains only the selected raster crop plus pitches, starts, durations, voices, rests, and track IDs for that measure. Responses are rejected if they are malformed, reference another measure/note/track, or contain out-of-range values.

Live Gemini testing requires a user-provided key and may incur charges. The repository test suite uses a mocked transport to test schema parsing and application logic; it does not pretend to make a live API request.

Official references used for the integration:

• Gemini 3.7 Flash model
• Interactions API migration guide
• Structured outputs
• Image understanding
• Document/PDF understanding

Configuration

Variable	Default	Purpose
AUDIVERIS_COMMAND	auto-detect	Exact Audiveris executable path
OMR_TIMEOUT_MS	600000	Per-scan OMR timeout
NOTE2MIDI_BRIDGE_HOST	127.0.0.1	Local bridge bind address
NOTE2MIDI_BRIDGE_PORT	4318	Local bridge port
NOTE2MIDI_ALLOWED_ORIGINS	local origins	Comma-separated browser origins allowed to call the bridge
NEXT_PUBLIC_NOTE2MIDI_BRIDGE_URL	http://127.0.0.1:4318	Browser-visible bridge URL
GEMINI_API_KEY	unset	Optional server-side Gemini key
GEMINI_MODEL	gemini-3.7-flash	Optional model override
NEXT_PUBLIC_SITE_URL	http://localhost:3000	Canonical URL for social metadata

Do not expose the bridge to an untrusted network. If the web UI is hosted separately, explicitly add that trusted HTTPS origin to NOTE2MIDI_ALLOWED_ORIGINS while keeping the bridge on loopback.

Tests

npm run typecheck
npm run lint
npm test
npm run build

The normal suite covers:

• image and multi-page PDF upload routes
• unsupported and corrupted inputs
• MusicXML notes, rests, chords, staves, parts, voices, pages, tempo, and accidentals
• malformed MusicXML and score validation failures
• valid MIDI headers, pitches, edited durations, chords, tie chains, and multiple tracks
• strict AI response validation, invalid references, review-before-apply behavior, and correction application

The legally clean fixtures were engraved from MusicXML authored for this repository. A live test runs the same PNG and two-page PDF through an installed Audiveris binary, then exports and reparses MIDI:

AUDIVERIS_COMMAND=/path/to/audiveris npm run test:omr:live

The live integration suite is skipped during ordinary npm test because CI may not have Audiveris installed. The release was verified with Audiveris 5.11.0 on Apple Silicon; both live PNG and two-page PDF tests passed.

OMR choice and licensing

Audiveris was selected after evaluating maintained open-source options including homr and oemer. It has the strongest fit here: active releases, multi-page and polyphonic score support, a documented batch CLI, and MusicXML 4.0 export. Camera-photo-first tools are useful alternatives, but their current scope is less aligned with a desktop PDF/staff/part workflow.

Audiveris is licensed under the GNU Affero General Public License v3. Note2MIDI invokes a separately installed Audiveris executable and does not copy its source or binaries into this repository. Even though process separation may permit independent licensing, Note2MIDI is conservatively licensed as AGPL-3.0-only to keep source-availability expectations clear and distribution unambiguously compatible. See LICENSE.

Important dependency licenses:

Component	Role	License
Audiveris	Local OMR executable	AGPL-3.0
React / vinext / Express	UI and local server	MIT
@google/genai	Optional Gemini client	Apache-2.0
PDF.js	Browser PDF preview	Apache-2.0
@tonejs/midi	MIDI writer/parser	MIT
fast-xml-parser	MusicXML parsing	MIT
JSZip	MXL extraction	MIT option selected
Zod	Runtime schemas	MIT

Audiveris references: repository, 5.11.0 release, CLI guide, and license.

Known limitations

• OMR is probabilistic. Dense orchestral scores, handwritten notation, tablature, contemporary notation, skewed photographs, and poor scans may need substantial correction.
• The internal editor preserves the core note/timing/staff data needed for MIDI, but it is not a full notation engraver. Lyrics, articulations, dynamics, ornaments, tuplets, pedal marks, and layout are not editable.
• One global tempo is exported. Mid-score tempo maps are not currently retained.
• Recognition confidence is not uniformly exported by Audiveris, so suspicious measures are primarily detected through structural timing checks.
• Preview playback uses lightweight Web Audio oscillators, not sampled instruments.
• Gemini suggestions can be wrong. They are advisory and must be reviewed by the user.

Privacy and disclaimers

• Normal scanning is local. Source files are written only to a private temporary folder and removed after the request.
• AI Assist sends the selected crop and selected measure to Google's Gemini API using the user's key. Gemini use is subject to Google's applicable terms, privacy policies, quotas, and pricing.
• Transcription accuracy is not guaranteed. Always review results before relying on them.
• Users are responsible for having the right to process uploaded material and for their own API usage and costs.
• Note2MIDI is provided without warranty under the terms of the AGPLv3.

Contributing

See CONTRIBUTING.md. Bug reports should include the operating system, Audiveris version, input type, and sanitized bridge error. Do not attach copyrighted scores unless you have permission to redistribute them.

License

Copyright (C) 2026 Note2MIDI contributors.

Note2MIDI is free software licensed under the GNU Affero General Public License v3.0 only.