Velocity Sensing for Tablets and Touch Screens - Crazy Idea

[Breeze]

Personally I like this idea! I don't see why you couldn't use some variation of existing technology like this virtual projected keyboard to do it. You could completely separate the "kbd" from the screen this way, and use a larger kbd.

The problem I see with using the cameras with the iPad is that the changing light of the iPad itself would interfere with the detection; and IMO the camera FOV would have to be skimming the top of the screen, perpendicular to it for this approach to work at all.

Orpheon used varying amounts of skin on the iPad screen to vary velocity, which was pretty good (got slapped in the fingers by Apple for using reserved code... not sure if it was re-implemented...). Detecting velocity has also been tried using the iPad's motion sensor (GarageBand does it this way) with mixed results. It's definitely a missing component for proper sound engine control on the iPad.

[BertKoor]

There's something inherently wrong in using tablets as keyboards: no physical feedback! How many times have you encountered pressing the backspace key (region) on your phone repeatedly and you end up with a sting of L or M characters? And does that also happen with physical keyboards? Point made...

[Kalamata Kid]

Personally I like this idea! I don't see why you couldn't use some variation of existing technology like this virtual projected keyboard to do it. You could completely separate the "kbd" from the screen this way, and use a larger kbd..

In the OP I did mention "A paper with a keyboard grid will do as well. Let the software figure out everything." True your idea is better though more expensive.

The problem I see with using the cameras with the iPad is that the changing light of the iPad itself would interfere with the detection; and IMO the camera FOV would have to be skimming the top of the screen, perpendicular to it for this approach to work at all...

What you say may be true. Initially I pointed out to put dots on the fingernails so to make tracking easier for the computer. It may take four cameras to get this to work better.

Orpheon used varying amounts of skin on the iPad screen to vary velocity, which was pretty good (got slapped in the fingers by Apple for using reserved code... not sure if it was re-implemented...). Detecting velocity has also been tried using the iPad's motion sensor (GarageBand does it this way) with mixed results. It's definitely a missing component for proper sound engine control on the iPad.

Velocity sensitivity is an important component in music creation so any attempt to accomplish this is ok by me.

[Kalamata Kid]

There's something inherently wrong in using tablets as keyboards: no physical feedback! How many times have you encountered pressing the backspace key (region) on your phone repeatedly and you end up with a sting of L or M characters? And does that also happen with physical keyboards? Point made...

I also thought the same and still do to a lesser degree but when velocity sensitivity finally arrives for the iPad I will be celebrating.

I have never had an issue with the problem you faced with your phone. Perhaps my wired connection with iConnectMIDI2+ makes things solid.

[PsYcHo SaMuRai]

There's something inherently wrong in using tablets as keyboards: no physical feedback! How many times have you encountered pressing the backspace key (region) on your phone repeatedly and you end up with a sting of L or M characters? And does that also happen with physical keyboards? Point made...

Exactly.


Is your camera device going to have it's own battery or will it plug into a port to suck the tablet's battery dry? And please don't tell me you'll just leave it plugged in to a charger. Be sure that this thing can be unplugged easily and remain portable...and that it comes with a pen or stickers for the dots on fingers. Preferably hypo-allergenic so no one gets a rash.

Again, once you pay for development of the hardware, coders for the programming and marketing how will this be profitable? Why would someone plug this battery sucking, potentially clunky, buggy and-not-as-good-as the hundreds of choices of midi controllers that are already on the market?

I can see future tablets that may have a resistive touchscreen over a capacitive touchscreen or using a flexible glass, capacitive touchscreen where the amount of flex could possibly determine velocity but as a peripheral device it only sounds cool after a few puffs on da hooka...I'm guessing

Anyways, kudos to you for still being cool and level headed after I've been bustin' your balls about your idea. You, sir, are a patient man. I would have told me to STFU after the 2nd post, lol.

[Kalamata Kid]

I have always been interested in alternative midi keyboards. Tablets and touch screens have made this easier to attain.

There are legions on musicians that use the conventional midi keyboards. I am not promoting a the abolition of these keyboards. My aim here is to promote an idea that may help the few that enjoy making music directly on the tablet.

The photo below shows me playing the Arpeggionome Pro. All I am saying is to give velocity a chance.

[UltraJv]

I cant help feeling Ms Kinekt will do some if not all of this.

[Kalamata Kid]

I cant help feeling Ms Kinekt will do some if not all of this.

Wow! thanks!
http://www.microsoft.com/en-us/kinectfo ... tures.aspx
Developers can write applications for this and hope someone reading this thread or anyone for that matter will do so. I would if I had the skill to do it. Perhaps PsYcHo SaMuRai will volunteer.

Edit:
I just read the fine print that Kinect is for Windows 8 applications. This is not bad news as a Windows touchscreen can also run any DAW makes it the better than any tablet. Only problem is there are very few apps for Windows 8. But if velocity sensing becomes a reality all the developers will devote their creative genius to making Windows apps with velocity capability.

[PsYcHo SaMuRai]

I cant help feeling Ms Kinekt will do some if not all of this.

Wow! thanks!

Developers can write applications for this and hope someone reading this thread or anyone for that matter will do so. I would if I had the skill to do it. Perhaps PsYcHo SaMuRai will do it.

I am currently in school for software engineering but since the cons I've mentioned haven't been addressed, why would I do it? Are you going to pay me enough money to do it? I guarantee my salary alone would push your product costs up.
Again, in future devices, build it in. As a peripheral, no.

[VariKusBrainZ]

So the alternative to a velocity sensitive keyboard with aftertouch is to paint black dots on your fingernails?
Seems like a step backwards and forcing a square into a circular hole, to coin some phrases.
ie your forcing a solution that is isnt ideal

[VariKusBrainZ]

Surely mapping finger surface area depressed on the screen is a better and already developed way to imitate velocity

[Kalamata Kid]

So the alternative to a velocity sensitive keyboard with aftertouch is to paint black dots on your fingernails?
Seems like a step backwards and forcing a square into a circular hole, to coin some phrases.
ie your forcing a solution that is isnt ideal

Agreed it is not ideal. I stated earlier that this may be a way to achieve velocity sensitivity on the iPad until Apple decides to incorporate it in the iPad as a standard feature.

No USB for iPad but there is accessories s that will do it. No wired midi? well we have the accessory iConnectMIDI2+. Accessories often are less than ideal but they perform the desired function.

[Kalamata Kid]

Surely mapping finger surface area depressed on the screen is a better and already developed way to imitate velocity

I have not tried this myself. It would be interested to hear users opinions on how well they like it. My Proposal using stereo camera will hopefully sense real velocity.

I am open to other ways of sending velocity to the app. Some of these have been discussed earlier in other threads and some are rather interesting. But in the end most people will prefer real velocity sensitivity and if it is built in the better.

[Kalamata Kid]

So the alternative to a velocity sensitive keyboard with aftertouch is to paint black dots on your fingernails?

Let me reiterate, the black dots on the fingernails are for tracking purposes. I assumed that it will be easier for a computer to track dots than to try to identify and track fingers. People nowadays wear tattoos so putting dots on the fingers should not deter anyone except the one with dotophobia. If it happens that the additional processing required to identify and track the velocity of fingers and thumbs does not slow down the computer then do not put dots on the fingers and thumbs . Really that simple.

[PsYcHo SaMuRai]

My Proposal using stereo camera will hopefully sense real velocity.

I am open to other ways of sending velocity to the app. Some of these have been discussed earlier in other threads and some are rather interesting.

Get a midi interface and use a controller with velocity sensitivity.