Attendees of the latest TED conference got a look at a futuristic device called SpaceTop where the user views a 3-D workspace through a transparent display, manipulating the on-screen elements with just their hands. It won't be on shelves any time soon, but it does pique the imagination.
The demonstration was by Jinha Lee, an MIT graduate student who undertook an internship at Microsoft's Applied Science group to develop a 3-D desktop. Part of that project was creating a new method for users to interact with the computer.
Mice and touchscreens, of course, really only let one interact along two dimensions — you can scroll left, right, up, and down, but going in and out must be done with 2-D gestures like pinches, whereas in real life you would just move your hand forward or back.
Lee's work focused on creating an interface though which one could move one's hand naturally and interact with familiar elements like windows and documents. The result is still rather rough, but it's promising.
The transparent display is equipped with a camera that tracks the user's head and adjusts the perspective on the 3-D desktop "under" it. Meanwhile, a second camera watches the user's hands and determines their position in three dimensions.
While it's still limited to a few demo applications, it's fascinating to watch: Grabbing a file from a stack is as easy as plucking it out, and when you need to type, just drop your hands onto the built-in keyboard.
Among the many challenges of developing such a system is that there is essentially no software written for it. Every operating system with which an average user might be familiar is built around the idea of a flat display navigated by a mouse or touchscreen. So Lee's had to be built from scratch.
As such, it's still very much just a concept or work in progress. His original work was done in late 2011, and the "core design and framework" were presented at Microsoft's TechForum in 2012. But this week's TED demonstration was much farther along and the first time it has been shown so publicly.
Whether this kind of interaction becomes commonplace or not (devices like the Oculus Rift and Leap Motion also present alternative input methods and are farther along in development), it's still a thrill to see something like this that seems to be science fiction made fact.
Devin Coldewey is a contributing writer for NBC News Digital. His personal website is coldewey.cc.
