Chris Cowdery

Although I have a deep interest in all ways that we interact with technology, one my favorite niche areas of HCI is Multitouch research. Ever since I saw Jeff Han give his TED Talk on Multitouch technology, I was hooked.

This summer I wanted to make my own Multitouch interface as a side project, but I wanted to add a new twist to it – I wanted to make it work in thin air, without any physical contact. I saw some very interesting projects on touchable holography , and I’d also just finished watching District 9 (my favorite part was admittedly some of the cool UIs that the aliens had), and I couldn’t stop but think… how cool would it be to have something like that??

I started sketching a bunch of prototypes in my spare time, and I finally came up with one inspired by some of the fantastic work of Johnny Chung Lee. The prototype used a matrix of infrared LEDs, positioned on the top of a display, which would reflect infrared light back to a user (standing an arbitrary location away from the display). The user would be wearing gloves with reflective tape on each finger, which would in turn reflect the infrared light back at two infrared cameras (read: wiimotes), positioned at either side of the display.

When the user first starts the system, they must calibrate it before they use it. A quick sample is done of the area to see if there are any areas emitting infrared light and if so, to remove them from the sampling area so they aren’t treated as input from you. The system plays a tone and the user then creates a brightness threshold by presenting all ten fingers at a point which functions as a virtual wall – it’s very similar to a mime’s wall performance.

Anything that is registered as brighter than the average of all ten points counts as a push, and for these reasons we can also detect virtual pressure based on how bright the light is being reflected when compared to the initial threshold. But we’re not done yet! We still need to know where the screen is, so after creating the virtual wall the system prompts the user to point at four crosshairs, positioned at the four respective corners of the screen.

CV algorithms are then used to detect if there is a ‘press’ and if so, how many and where. Using pre-existing libraries for processing mutlitouch input, we are able to process the information read from the cameras and use this information to recognize various gestures, such as pinching and a finger coming in and out of the threshold.

The current version now has ‘darth vader gloves’ which have IR LEDs wired right into the glove. This not only helps cut down on power consumption, but it also gives more relevant readings in the cameras as lots of noise would be generated by the IR light rays hitting various parts of the reflective tape. The current prototype doesn’t support any more than 4 points acting in a multitouch gesture either at this point, but I plan to expand on this soon, school permitting

Youtube videos, photos, howtos, specs and the like will hopefully find their way up here soon, coursework permitting