Velocity Sensing for Tablets and Touch Screens - Crazy Idea

[Kalamata Kid]

One way to detect velocity on a Tablet: Suppose you have a keyboard layout on your tablet that does not detect finger velocity. Here is a proposal to do just that:

Attach two cameras to your tablet so that the cameras can see the surface. Perhaps a continuous lens camera will work better? Or use three or four cameras? Get the cameras and the computer to calibrate the 3D space and the 2D key layout to produce a stereo image of the immediate space.

Put dots on your fingernails. (for the early versions) and a dot in the middle of your thumb print to aid the computer to discern the finger activity. As you play the cameras will be able to detect the velocity of your fingers and thumb as they strike the 2D keys on the tablet.

It can also detect-after touch gestures or after-touch release velocity as the fingers are lifted.

This will work on tablets, Windows touch screen monitors, midi keyboards. This can be used on Acoustic pianos to detect the fingers and convert to midi data. One step further and one does not even need a tablet or the like. A paper with a keyboard grid will do as well. Let the software figure out everything.

Is this feasible? Will there be any significant time lag? Is there a product already out that can be modified to do this? Or is this just a crazy idea?

[PsYcHo SaMuRai]

Get a midi interface and use a controller with velocity sensitivity.
The after touch release velocity isn't a usual playing technique so it would be a tough feature to sell.

[Kalamata Kid]

Get a midi interface and use a controller with velocity sensitivity.
The after touch release velocity isn't a usual playing technique so it would be a tough feature to sell.

How does that help to enable a tablet to have velocity sensitivity or detection?

Let me try again. Let's say I have a keyboard app on an iPad and now the way things are I can play by tapping my fingers on the iPad surface. I am suggesting a method that can detect finger velocity.

[PsYcHo SaMuRai]

I'm saying that you can already connect a device that has velocity sensitivity.
http://www.ikmultimedia.com/products/irigkeys/

I understand what you're saying but I don't see it happening.
Now, if you had a 3D VISUAL interface AND 3D control of that interface, then I could see huge possibilities for music and many other applications.
As it is, what you're proposing is already accomplished using midi controllers minus the release velocity but I don't know of any instrument with that feature so again, it isn't a natural playing technique that the majority of musicians would feel the need to have.
I don't believe a theremin or laser harp count as there isn't any initial "touch" per se. You don't use the playing techniques of playing the piano, guitar, drums, etc. and apply it to an interface like a theremin or laser harp.
Maybe, you could develop this to go with a theremin app but again, not sure if there is a big enough market for it, cost/ benefit ratio is low.

[PsYcHo SaMuRai]

There is an option for what your saying and that's to have a resistive strip sandwiched between two pieces of foam that you could add to an area on a tablet. The harder you press, the more resistance is created.
Then, over this resistive strip you could add 2 switches under each "button area". The faster you push the "button area" the higher the velocity and Voila! You have velocity with after touch sensitivity...even release sensitivity!
All that's needed is to group the "button areas" into a couple contrasting colors to separate the natural and sharp "button areas" and lay them from left to right in a low to high note configuration. What would really make it even more expressive is to add 2 rotary knobs mounted on an axis with one that is spring loaded so that it always returned to center and the other "axis mounted rotary knob" able to travel about 3/4 of the way around yet stay wherever you move it to. Maybe add some resistive sliders and rotary knobs to control other parameters in real time.
Finally, you could develop a standardized protocol that would allow this device to connect and communicate with other devices.

A lil' sumthin sumthin I programmed in AutoCad;

[Kalamata Kid]

Eventfully tablets will have velocity and/or pressure sensitivity. My proposal is a temporary solution until the ios and android manufactures get their act together.

There are musicians that play directly on the tablet tapping with their fingers to create music. Tablet apps have a variety of new types of keyboard layouts and adding the velocity component to music creation is a desired feature.

What I am proposing in the OP is doable now. It is just a matter of how well it can be done on tablets and will it be better than the currently employed methods. I believe yes it will be better.

[PsYcHo SaMuRai]

Refining your idea and giving it some slick polish I came up with the idea to have a device that you could hold or wear. This device could have dots on it that are positioned in a way that the camera could see when the dots get smaller or larger, ie., closer or further away. It could even measure tilts if there were 3 dots on it.
All you need is your standard tablet camera and this wearable device like a wristband or...A RING!

I introduce...Ringi!

[Spiritos]

I had a wild similair idea once using an external microphone (or even the built-in mic of a tablet):

A mic can pick up volume or sound pressure. What if you have a tablet lying on it's back with a mic directed at it (or several for more exact positioning of the volume/pressure used ). In theory it would be able to pick up the velocity with which you tap the tablet's surface.
Difficulty would be configuring the mics to determine the exact position of where the pressure is applied though but that's for the soundtechies to figure out

However it should be easy to write an algorhythm to translate the volume/soundpressure into velocity input/output.

Kind of a rough idea but apps like 'Impaktor' already use volume-input of the built-in mic.

[Aural Chaos]

Jeez, we don't have enough promotion from IK already??

[Kalamata Kid]

I am not proposing anything like the Theremin or ethereal music creation. In the OP I proposed that the finger actually taps on the surface of the tablet.

The cameras capture the velocity of the dot on the fingernail and when the finger taps the tablet the location or midi key number is determined. This will make any tablet velocity sensitive for all the fingers and thumbs.

[PsYcHo SaMuRai]

It can also detect-after touch gestures or after-touch release velocity as the fingers are lifted.

Is this feasible? Will there be any significant time lag? Is there a product already out that can be modified to do this? Or is this just a crazy idea?

I responded with the idea of just plugging in a midi keyboard because;
1. It's already developed so there's a wide range of options.
2. You're asking to turn a multi-purpose machine into a purpose-built one which brings us back to just using a synth, midi keyboard or something like the iRing...eh em, I mean Ringi.
I also responded with Ringi and midi because it's a developed and available product/protocol.
I understand your want for velocity sensitivity but you also mentioned "release velocity" as a an added benefit which I don't see as practical except in the case of a theremin-type instrument application.

I thought you were spit-balling an idea and in response I thought of how practical and marketable this idea would be. Without a broader application, not very.

One way to detect velocity on a Tablet: Suppose you have a keyboard layout on your tablet that does not detect finger velocity. Here is a proposal to do just that:

Attach two cameras to your tablet so that the cameras can see the surface. Perhaps a continuous lens camera will work better? Or use three or four cameras? Get the cameras and the computer to calibrate the 3D space and the 2D key layout to produce a stereo image of the immediate space.

Put dots on your fingernails. (for the early versions) and a dot in the middle of your thumb print to aid the computer to discern the finger activity. As you play the cameras will be able to detect the velocity of your fingers and thumb as they strike the 2D keys on the tablet.

Simply put, I see what your proposing as a step in the right direction as far as it being a piece to a better UI in general but not as a stand alone product to achieve velocity sensitivity and release velocity for current virtual keyboards.
It seems to me that what you are describing with your cameras and dots-on-fingers approach is an interface above and on the tablet screen to achieve velocity sensitivity by using the screen itself to sense the key press and cameras for the area above the screen to sense velocity, correct?
I'm saying, that would be an overly complicated method for achieving velocity sensitivity and release velocity alone since midi controllers are available that already achieve this (minus release velocity which I stated as impractical on an instrument other than the theremin).
However, the idea of this interface I find practical if the device also had a 3D visual experience as well.
Think of the UI for JARVIS in Iron Man. Then, you could truly have a "virtual instrument". A 3D model of a guitar, drums, etc. but using this type of UI for far more than just a velocity sensing virtual keyboard. Such as saving all mankind from imminent doom.

[Kalamata Kid]

Many musicians that use a tablet as a midi controller would like the tablet to also have velocity sensitively. Eventually the tablets will also be velocity sensitive not just for midi but other music and non-music applications. My proposed idea can be used until this happens.

I for one us the iPad as a midi controller (see my signature) connected to my computer with the iConnectMIDI2+ http://www.iconnectivity.com/iConnectMIDI2plus . Most if not all musicians that use the iPad as a midi controller will like the apps to detect velocity.

Some apps have some means of correlating a gesture with velocity but for me this is as of now not well implemented. There is much room for improvement here and perhaps someday soon this will be adequate. Read some of the threads here at KVR and elsewhere seeking ways to have velocity detection on the tablets. I am offering one way that this can be accomplished.

My proposal using to cameras and the parallax principal can detect the finger motion and determine it’s velocity and location. The tap on the tablet determines the exact location or the midi key number. The musician will be able to tap on tablet on music apps that are velocity ready and enabled.

[PsYcHo SaMuRai]

There may be a very small market of people that would prefer a touch screen keyboard over the real thing but I think that's the minority.
Would you open an Office document on your computer and write a paper, spreadsheet, email, etc. on an iPad's touchscreen connected to that computer? I know when I have to write a paper or long email on my iPad for convenience and portability, I use my keyboard bluetooth case. If I have any serious paper writing or emailing to do and I have access to my laptop, I just use my computer instead of the iPad.
I wouldn't get the iConnect to use the iPad on-screen keyboard for controlling a plug-in on my DAW except as a basic control surface at a workstation computer or laptop. I use the computing power/ apps on the iPad but control it and any DAW plug-ins with a midi controller. iConnect to link 2 iPads, a laptop and desktop all controlled by 1 or 2 midi controllers. Done deal with off-the-shelf components.

[Kalamata Kid]

When tablets have built in velocity sensitivity most if not all music apps will be have it implemented. Only a few musicians will disable the velocity sensitivity. Before this time how may will be willing to pay for velocity detection/sensitivity for tablets will depend on the price.

Some upcoming developer may hit upon a software solution to mimic velocity detection that may make the hardware/software solution less desirable and even pointless. Music app developers already have shown their creativity in designing new ways of music creation. As they say we’ve seen nothing yet. Their creative genius will make things happen in un-expected ways.

The inconvenience of dragging along an additional device to detect velocity will inhibit usage but for the studio musician like myself this is not an issue. All my studio midi gear has velocity sensing and so do all my DAW’s and VSTi’s. Soon (hopeful very soon) this will be common in tablets and music apps.

Some enterprising individual or company may not be willing to make a device like mine or something similar when there will soon (hopefully soon) be tablets with built is velocity sensitively. Then off course will the tablet manufactures have musicians in mind to give the sensitivity 128 levels, or will it be just two or three levels?

[Kalamata Kid]

The photo shows my paper Mockup of my velocity detection proposal. See OP

Granted a very crude mock-up.

The two orange dots represent the two cameras viewing the tablet area and the fingers. The fingers have dots for easier velocity and location tracking. Some calibration will be required for the setup.

3D tracking with two cameras of people and cars is now common place. Tracking of dots on fingernails should relatively easy, even if we are including 10 dots. I suppose that a small computer can accomplish this. If I am wrong here that the deal about my proposed idea about velocity detection is off.

So it would be great if Seline (the current app) and the iPad both were velocity sensitive. I suppose some people will prefer not to have the velocity sensitivity so they can disable it. I for one prefer velocity sensitivity weather playing on a midi keyboard or on a music app on the iPad.