KVR Audio

mfa · Post by **mfa** » Sat Dec 03, 2016 9:33 pm

Hello,

i'm looking for a solution for detecting chords from audio. I'm aware that this is a very difficult problem in the general case. What i'm looking for specifically, is the ability to tell whether a given set of notes i.e. a chord is present in a given audio input.

This would be used in a music learning app to detect when the user plays the correct chord and when not. Some mobile apps, e.g. Überchord and Yousician, to name a couple, are already doing this very well. I assume this problem is significantly easier to solve, than for example general polyphonic audio to midi or automatic music transcribing. The ability would be very useful even after loosening the requirements further to, say, to be able to answer yes/no for a limited set of chords or just the basic majors/minors and so on.

There's been a lot of academic work on the subject, that i will read as much as i can, but there's a risk that the solutions will just go over my head or that there's simply not enough hours in a day. Please share with me if you have any insight on this technology and methods. I'll appreciate all advice. Although i didn't come here for shopping, i would of course be very interested to know if anyone of you already have code for this and would be willing to discuss licensing.

Thanks for your time!

Urs · Post by **Urs** » Sat Dec 03, 2016 9:56 pm

I guess actually detecting chords isn't rocket science, as long as the timing accuracy can be neglected. Also, you probably don't need to know which instrument plays which note.

Check out Photsounder Spiral. Maybe when you look at its output you get an idea about how to appraoch this.

http://photosounder.com/spiral/

mfa · Post by **mfa** » Sat Dec 03, 2016 10:19 pm

Wow, that's a cool plugin! Thanks!

Yes, i suppose a sufficiently high resolution dft could go a long way, especially on acoustic instruments, where the ovetones fall off naturally. The spiral - kind of modulo 12 note - view is indeed inspiring and sort of solves it. Thanks a lot Urs!

earlevel · Post by **earlevel** » Sun Dec 04, 2016 8:10 pm

mfa wrote:Yes, i suppose a sufficiently high resolution dft could go a long way...

If you have time, take the free course, "Audio Signal Processing for Music Applications", or at least go through the videos to see the parts about the harmonic model (and play with it via sms-tools). Or just go directly to sms-tools and figure it out.

Yes, basically hi-res FFT as you said, but there's a trick (fitting a parabola to find the center of a harmonic, between "bins"), that lets a not-so-hi-res transform do the job. So, you have a bunch of frequencies, group them into tones based on harmonic relationships, and now you have your independent notes. From there you can just take that info, or change the notes independently and resynthesizes.

A quick search, this discusses the parabolic interpolation:
https://www.coursera.org/learn/audio-si ... -detection

mfa · Post by **mfa** » Wed Dec 07, 2016 8:31 pm

Wow, that's a cool trick! I had know idea that we can compute the peak frequency with greater precision like this. And yes, i signed up for the course, even though the lesson you linked pretty much gave the formula and explained the parabolic interpolation. This is great stuff i'm already matching overtones to the computed peak frequencies and it looks absolutely fantastic.

Acoustic instruments seem to have pretty much "character" in how the overtones resonate and fall off, but i'm pretty optimistic in simply being able to match the peak locations/frequencies to identify the chord, even if there's a half dead note or two.

Thanks a lot Earlevel!

Detecting if audio contains a given chord or not - polyphonic pitch recognition