WebCenter User Experience and Interaction From iPads to Xbox

More documents

Recommendations

Info

How does this Work There are currently no browser APIs available for the Kinect; however, there are two ways I have managed to integrate the Kinect with the browser. 1. WebSocket Push 1. Plug the Kinect into your PC or Mac. 2. Install the Kinect drivers OpenNI and NITE 3. From device manager you will see you Kinect is now recognized. 4. Setup a WebSocket server (I use nodeJS) (If you prefer, you can setup nodeJS to run directly on the clients machine within a FireFox XUL Extension - a tutorial can be found here so an external server is not required.) http://rawkes.com/blog/2011/12/05/running-node.js-from-within-a-firefox-xul-extension a.) WebSocket Server plugin for WebLogic is soon to be released. 5. I wrote a small C++ app that passes the information from the Kinect to the socket server. 6. On the webpage I use the socket.io.client JS Lib that opens a connection to the WebSocket server. 7. I pass an array X,Y,Z co-ordinates from my C++ app connected to the Kinect through the WebSocket gateway that transfers the information to the browser in realtime allowing me to create my own gestures and integrations with HTML5 elements like canvas. 2. DepthJS DepthJS in action interacting with The New York Times Website Students at the Massachusetts Institute of Technology have gone further, inventing DepthJS, a browser extension (currently Chrome & Safari) that allows the Microsoft Kinect to talk to any web page. It provides the low-level raw access to the Kinect as well as high-level hand gesture events to simplify development. These allow the Kinect to recognise hand and finger motions, allowing users to surf the internet and ''handle'' computer files. And, unlike Tom Cruise's character in Minority Report, no gloves are required. © 2012. Fishbowl Solutions, Inc.
1. Plug the Kinect into your PC or Mac. 2. Install the Kinect drivers OpenNI and NITE 3. Setup Browser a.) Chrome, Firefox install the FireBreath plugin + DepthJS b.) Safari install the DepthJS extension c.) No support currently for IE 4. Include the depthjs lib on your page. 5. A demo can be seen here with a few sample events – https://github.com/doug/depthjs/blob/master/developer-api/BasicDemo.html Kinect for Windows If you know C++, Windows provides a great resource for their open SDK allowing you to integrate with the Kinect Voice Recognition engine and more – You can find information on this here – http://www.microsoft.com/en-us/kinectforwindows/develop/ Conclusion For the last 30 years the mouse and keyboard have been the main input devices for interacting with desktop interfaces. While other technology such as graphics cards, processors, and network infrastructures have significantly evolved during this period, it is only within the last couple of years that we have stepped out of the confinement of mouse-and-keyboard interactions and have begun to look at how other input devices can help improve our interaction with data. Touch integrations and spatially aware devices that were not even conceivable a decade ago will now let us push user experience to the next level. © 2012. Fishbowl Solutions, Inc.
Page 1 and 2: WebCenter User Experience and Inter
Page 3 and 4: Advertisements, Tags, and highlight
Page 5 and 6: Example Xbox Metro UI with Voice En
Page 7 and 8: var flashvars = {speechServer : "ht
Page 9 and 10: } // Called if something bad happen
Page 11 and 12: ADF Example: AdfDhtmlPanelSplitterP
Page 13: • Example: element.addEventListen

WebCenter User Experience and Interaction From iPads to Xbox

Create successful ePaper yourself

Delete template?

Save as template?