MusiKraken, a MIDI controller construction kit app

[Snarp]

Hello everyone!

I am the developer of MusiKraken, an app for iOS and Android (for both tablets and phones).

With the app, you can make music with your hands (Keyboard and Chords Pad, or Hand Tracking on iOS), your face (Face Tracking), your voice (Microphone pitch and amplitude detection) or by rotating the device (Accelerometer). And you can control multiple instruments simultaneously by splitting chords or sending the MIDI events to multiple channels.

It is basically a MIDI Controller Construction Kit, so you can combine the modules however you want and adjust the setup to the instrument(s) you want to control with it. You can connect to a computer running a DAW via WiFi or USB cable and use your mobile device as MIDI input, or host Audio Units on iOS, or send the MIDI data to other apps that generate sound.

The Keyboard control now also has MIDI Polyphonic Expression (MPE) support, and the Android version has almost the same features as the iOS version (except Hand Tracking and the TrueDepth module. Hand Tracking will hopefully come in a later version).

Here is a short overview of some of the modules (but I have already added a few new features since then ):
https://youtu.be/ihbBFYxidGA

You can get it here:
iOS:
https://apps.apple.com/app/musikraken/id1538781007

Android:
https://play.google.com/store/apps/deta ... Controller

[Snarp]

I released a new update for MusiKraken on iOS (version 1.13):

-Hand Tracking now also supports detection of two hands, so the gestures can be performed with two hands simultaneously. And if your device has a TrueDepth sensor, you can directly access it from the Hand Tracking module. The advantage of this is that it will get the distance of the middle of your hand instead of the closest distance to whatever is in front of the camera. And you can use the distance of two hands (your hands, or with another person)!
The disadvantage is that you will only get a depth value if you hand is actually detected. Which worked nicely with my hands so far, but I haven't tested that many other hands yet. And one difficulty when tracking two hands is that both hands need to be visible. So if you move one hand closer to the camera, make sure that it doesn't cover the second hand. The easiest way to do this is to make the range smaller and keep enough distance to the device (it tracks the hands up to one meter of distance).

-You can now also use the Gyroscope to make music. While the Accelerometer uses the acceleration in 3 axes as data, the Gyroscope uses the device rotation.

-In the MIDI Input or MIDI Output module, you can now open the Bluetooth MIDI configuration settings of iOS. So you can scan and connect with other Bluetooth MIDI devices or advertize as a Bluetooth MIDI device, so that other devices can connect with it. So from now on you can now connect via WiFi, cable and Bluetooth.

MusiKraken iOS:
https://apps.apple.com/app/musikraken/id1538781007

MusiKraken Android (this will get the gyroscope and bluetooth update probably in the next version):
https://play.google.com/store/apps/deta ... Controller

http://www.musikraken.com

https://youtu.be/unn4Qj8xzqs

[Tj Shredder]

Great idea, got it immediately and immediately have requests and bugs to report...
The chords are reversed, that is unintuitive, the upper rows should be the higher ones, as they are higher. When switching the octave in the chords I get stuck notes. But its the least interesting module anyway...
The face and hand tracking is cool, I would add a video display, to actually see what the device sees, maybe as extra device. That way its easier to learn what the values mean...
It does not seem to be able to distinguish left and right hand, that could be useful for the future of the app.
The keys module needs more layouts. For example a LinnStrument layout, a hex layout layout and it needs to support MPE somehow. Even if we don’t get aftertouch. It would be cool to create your own layout (I would want a LinnStrument with tritone tuning)
Anyway thanks for creating this and also thanks for publishing it in the state it is at the moment...

[Snarp]

The face and hand tracking is cool, I would add a video display, to actually see what the device sees, maybe as extra device. That way its easier to learn what the values mean...
It does not seem to be able to distinguish left and right hand, that could be useful for the future of the app.
The keys module needs more layouts.

Thanks for the feedback! The video display will come soon, I will need it for the full-body tracking (works on both iOS and Android already) that I haven't added to the release yet because for that you will need to see where on the image your hands and feet are...
And there will probably also soon be more key layouts. I am constantly trying out new ideas, so if I like one of them I will add it to the app...

[Snarp]

I released a new version of MusiKraken that has a new control: The Touchpad! (not a very creative name, I know ).

The Touchpad is one giant multitouch surface. You can use multiple touch movements to generate MIDI events. Slide up-down and sideways with one or multiple fingers, spread your fingers to change the x- and y-difference values or use pressure or touch radius if your device supports it. This way you can control up to 5 independent parameters simultaneously. Per hand...

The Touchpad is also one giant button. Connect anything that generates notes or chords to its noteIn port and the Touchpad will play the active notes if you touch it.

And to make this more interesting, I also added a few arpeggiator patterns to it, so if you activate one of these patterns, it will play single notes based on the pattern and the button presses.

And then there is the "Note tapping" setting. In normal buttons, if there is already a finger down on the button it will not play another note until you release it. If Note Tapping is enabled, it will play a new note on every new touch. This way you can play faster by tapping on the Touchpad (at least I have a lot of fun every time I play with this...).

You can see a short demonstration of what you can do with the Touchpad here:
https://youtu.be/JO5eRUiLcc0

[Snarp]

I now added the probably least useful module for serious music making to MusiKraken. But you can jump around and make music at the same time, so it is a lot of fun and useful to control MIDI expression values. I present to you, the BODY POSE TRACKER.

The app tracks the positions and movements of your hands, feet and head and, as always with Musikraken, you can define how these values should be converted to MIDI events. The hand and feet positions are defined relative to the shoulders and hips, and to the estimated size of the body on the camera image. This way it should not matter how large you are or how far away you are from the camera.

The head position is defined relative to the image frame. I wanted to have at least one parameter that is different depending on where you stand, and the head is easy to move sideways or up and down.

And you can make music together with someone else! The app can track two persons simultaneously!

So the Body Pose Tracker can be very useful if you always wanted to dance and make music at the same time (my kids love it). Or you can also use it to conduct your music: Move your hands up and down or sideways to change the volume, dynamics or any other parameter of your music.

And I added a video screen so you can see what the camera sees and tracks (you can activate it in the general settings. Click on the settings button in the editor to enable it).

The Body Pose Tracker is only available in the iOS version for now, but I will probably add something very similar to this soon to Android as well.

I will now work on more serious features for the next update. Probably...

https://youtu.be/E8Qeu2Wr4SA

[3dog]

My first use case was fairly easy to set up. I used the chord pads to control SwAM mono instruments. I need to be able to assign a midi note to a chord pad so I can control the chord pads with a wind controller. Any plans on using midi learn to do this? Any plans for guitar voicings on the pads?