Audio to MIDI converter

Drop a song, a piano or guitar recording, or a sung melody. Get MIDI, MusicXML and ABC files, see the notes, and hear what was recognized.

Drop a song or a recording here
or click to choose, or paste with Ctrl+V

Nothing is uploaded. Everything runs in this tab with @spotify/basic-pitch, @audio/mir, @audio/beat, @audio/midi, @audio/musicxml, @audio/midi-render.

Polyphonic audio to MIDI transcription, on your device

Most audio to MIDI converters upload your file to a server. This page runs Spotify’s Basic Pitch in the browser: a neural network of 16,782 parameters from Spotify’s Audio Intelligence Lab, trained on recordings of piano, guitar, voice, synthesizers and other instruments.1 Every 11.6 ms it reads a constant-Q spectrum with three bins per semitone, stacks each bin with its harmonics, and marks for each of the 88 piano keys where a note starts and where a note sounds. A peak in the onset map starts a note and the note map holds it until it fades; strong stretches of the note map with no onset become notes too.1,2 The model weighs 0.9 MB and runs on your graphics chip with TensorFlow.js.

Tempo comes from @audio/beat and the key from chroma frames with @audio/mir-key, as in the Key and BPM finder. @audio/midi writes the Standard MIDI File, @audio/musicxml the MusicXML 4.05 and ABC 2.16 scores, and @audio/midi-render plays the result back with a General MIDI synth so you can check it by ear before opening it anywhere else.

Song to MIDI: what comes out

One MIDI track with every pitched note Basic Pitch heard, voice included, from A0 to C8. It does not tell instruments apart and has no drum output. Its authors note it works best on one instrument at a time:2 a solo piano, guitar or voice comes out cleaner than a full mix, which gives a draft to edit. To transcribe only the tune of a song, isolate the vocal with the vocal remover first.

Even a clean recording yields some extra notes. On this page’s test chords, C major then F major, one second each, it finds all six notes within 25 ms of their onsets, plus seven more: four overtones written as quieter notes an octave or more above, and A4, C5 and the F5 overtone struck again at 1.65 s. Spotify’s Python release writes the same thirteen notes from the same file.

Melody mode

The second mode is a classical tracker: @audio/mir-transcribe follows pitch continuity across @audio/mir-multif0 frames (Klapuri’s 2006 multiple-F0 estimation3) and keeps one note at a time. Use it for a whistled or sung tune, a solo line or a bassline when chords are not wanted.

MIDI and sheet music

The MIDI file keeps every note as heard: its own start and length, unquantized, at the detected tempo, with Basic Pitch’s note activation as velocity, as its own MIDI export does. Pitch bends are not written. The MusicXML and ABC scores are quantized to the grid you pick and spelled in the key. For polyphony they keep every note: notes that start and end together form a chord, and a note held under a moving line goes to a voice of its own on the same staff, each voice filling every bar.5 Notation programs show up to four voices on a staff; a dense transcription can have more. To split a piano part into two hands, import the MIDI file into MuseScore, whose MIDI import does that.4

Reading the report

Notes found, the most notes sounding at once, the tempo and key used (auto-detected or the value set above), the pitch range, how long Basic Pitch took and on which processor, and the first bars of the ABC transcription. The piano roll draws one row per semitone, C rows named, stronger notes brighter. A confidence percentage on an auto-detected key, or a low-confidence note on the tempo, is a signal to set that value manually.

References

  1. R. M. Bittner, J. J. Bosch, D. Rubinstein, G. Meseguer-Brocal, S. Ewert, “A Lightweight Instrument-Agnostic Model for Polyphonic Note Transcription and Multipitch Estimation,” Proc. IEEE ICASSP 2022: 16,782 parameters; CQT with 3 bins per semitone, ≈11 ms hop, 7 harmonics and 1 sub-harmonic stacked; onset and note outputs per semitone; training data in Table 1 (GuitarSet, MAESTRO, Slakh, iKala, MedleyDB).
  2. Spotify, basic-pitch-ts (@spotify/basic-pitch 1.0.1, Apache License 2.0): “Basic pitch works best on one instrument at a time.” Defaults used here, from basic_pitch/constants.py: onset threshold 0.5, frame threshold 0.3, minimum note 127.7 ms, 22,050 Hz input, 256-sample hop (11.6 ms), 88 keys from 27.5 Hz.
  3. A. Klapuri, “Multiple fundamental frequency estimation by summing harmonic amplitudes,” Proc. ISMIR 2006.
  4. MuseScore Handbook 3, “MIDI import”: Max. voices, Split staff.
  5. W3C Music Notation Community Group, MusicXML 4.0.
  6. The abc music standard 2.1.

Questions

Can it convert a whole song to MIDI?

Yes, in Polyphonic mode. Basic Pitch writes every pitched note it hears, voice included, into one MIDI track. It does not split the instruments apart and has no drum output, and its authors note it works best on one instrument at a time: a solo piano or guitar comes out cleaner than a dense mix. For only the tune of a song, isolate the vocal with the vocal remover on this site, then use Melody mode.

Polyphonic or Melody mode?

Polyphonic, the default, runs Basic Pitch, a small neural network that hears several notes at once: chords, piano, guitar, a band. Melody mode follows one pitch at a time with a multiple-F0 estimator (Klapuri 2006) and suits a whistled or sung tune, a solo line or a bassline when you want exactly one note at each moment.

Can I convert my voice to MIDI?

Yes. Record yourself singing, humming or whistling and drop the file here. Basic Pitch was trained with vocal recordings among its data; Melody mode keeps strictly one note at a time. Vibrato and slides land on the nearest semitone: the MIDI file has no pitch bends.

What do I do with the MIDI, MusicXML and ABC files?

Import the .mid into any DAW or notation program: it holds every note with its own start, length and velocity, at the detected tempo. Open the .musicxml in MuseScore, Finale or Sibelius for engraved sheet music. The .abc file is plain text: paste it into any ABC renderer (abcjs, EasyABC). In Polyphonic mode both scores keep every note, notes that overlap written as voices on one staff; to split a piano part into two hands, import the .mid into MuseScore, whose MIDI import does that.

Why is the tempo wrong?

Tempo comes from the @audio/beat onset and beat tracker, which can lock onto half or double the real tempo, or drift on rubato playing. Type a tempo in manually if the note grid looks off.

Why is the key wrong?

Key comes from matching chroma (pitch-class) energy against major and minor key profiles over the first two minutes. Modal or ambiguous music can report the relative major or minor, which shares every note. Pick a key manually to force the spelling.

Is my audio uploaded anywhere?

No. The file is decoded and transcribed in your browser. Basic Pitch runs with TensorFlow.js on your graphics chip through WebGL, or on the CPU without it; its model (0.9 MB) and TensorFlow.js (about 1.3 MB) download once from public CDNs, jsDelivr and esm.sh. Tempo, key, MIDI, MusicXML and ABC are computed and written locally.

More audio utilities