15.01.2015 Views

A Short History of Tabletop Research, Technologies and Products

A Short History of Tabletop Research, Technologies and Products

A Short History of Tabletop Research, Technologies and Products

SHOW MORE
SHOW LESS

You also want an ePaper? Increase the reach of your titles

YUMPU automatically turns print PDFs into web optimized ePapers that Google loves.

A <strong>Short</strong> <strong>History</strong> <strong>of</strong> <strong>Tabletop</strong> <strong>Research</strong>,<br />

<strong>Technologies</strong> <strong>and</strong> <strong>Products</strong><br />

Christian Müller-Tomfelde <strong>and</strong> Morten Fjeld<br />

July 2011


Overview<br />

• Introduction<br />

• What are tabletops<br />

• <strong>Research</strong> domains <strong>and</strong> communities<br />

• Review <strong>of</strong><br />

• L<strong>and</strong>mark research<br />

• Enabling technologies<br />

• Commercially available products<br />

• Analysis<br />

• Three key transitions<br />

• Synoptic l<strong>and</strong>scape<br />

• Springer book in the HCI Series:<br />

<strong>Tabletop</strong>s - Horizontal interactive Displays<br />

CSIRO - Christian Müller-Tomfelde - A <strong>Short</strong> <strong>History</strong> <strong>of</strong> <strong>Tabletop</strong> <strong>Research</strong>, <strong>Technologies</strong> <strong>and</strong> <strong>Products</strong> 2


What are <strong>Tabletop</strong>s<br />

• Computer interface st<strong>and</strong>s in the tradition <strong>of</strong> earlier<br />

terms, such as desktops <strong>and</strong> laptops.<br />

• The term was first used in 2001<br />

• <strong>Tabletop</strong>s are suitable as<br />

• Interfaces for small groups, with a<br />

• Common shared orientation-free workspace <strong>and</strong> the<br />

• Affordance <strong>of</strong> placing objects on them.<br />

• Allowing new forms <strong>of</strong><br />

• h<strong>and</strong> <strong>and</strong> gesture interaction,<br />

• tangible interaction, <strong>and</strong><br />

• novel interactive visualizations.<br />

CSIRO - Christian Müller-Tomfelde - A <strong>Short</strong> <strong>History</strong> <strong>of</strong> <strong>Tabletop</strong> <strong>Research</strong>, <strong>Technologies</strong> <strong>and</strong> <strong>Products</strong> 3


<strong>Tabletop</strong>s <strong>Research</strong> Motivation<br />

• <strong>Tabletop</strong> interfaces rely on users’ mental models <strong>of</strong><br />

traditional tables.<br />

• Users combine the advantages <strong>of</strong> the physical<br />

environment with the possibilities <strong>of</strong> the digital<br />

information space - hybrid environment.<br />

• Underst<strong>and</strong>ing a tabletop as a mere surface with<br />

electronic sheets <strong>of</strong> paper is an inadequate.<br />

• <strong>Research</strong>ers <strong>and</strong> application designers are tempted<br />

to transcend this traditional user perception<br />

• Take advantage <strong>of</strong> the possibilities that make<br />

interactive tabletops unique <strong>and</strong> fascinating.<br />

CSIRO - Christian Müller-Tomfelde - A <strong>Short</strong> <strong>History</strong> <strong>of</strong> <strong>Tabletop</strong> <strong>Research</strong>, <strong>Technologies</strong> <strong>and</strong> <strong>Products</strong> 4


<strong>Tabletop</strong> <strong>Research</strong> Domains<br />

CSCW<br />

UbiComp<br />

HCI<br />

CSIRO - Christian Müller-Tomfelde - A <strong>Short</strong> <strong>History</strong> <strong>of</strong> <strong>Tabletop</strong> <strong>Research</strong>, <strong>Technologies</strong> <strong>and</strong> <strong>Products</strong> 5


Top-ten <strong>Tabletop</strong> L<strong>and</strong>mark Publications<br />

Based on Google Scholar, August 2009 (http://www.harzing.com/pop.htm)<br />

CSIRO - Christian Müller-Tomfelde - A <strong>Short</strong> <strong>History</strong> <strong>of</strong> <strong>Tabletop</strong> <strong>Research</strong>, <strong>Technologies</strong> <strong>and</strong> <strong>Products</strong> 6


DigitalDesk (1992)<br />

• Pierre Wellner’s version <strong>of</strong><br />

an augmented desktop for<br />

individuals<br />

• Data projection <strong>and</strong> image<br />

capturing<br />

• Computer vision based<br />

finger tracking<br />

• Digital calculator<br />

applications<br />

Wellner P (1993) Interacting with paper on the DigitalDesk.<br />

Communications <strong>of</strong> the ACM 36(7):87–96<br />

CSIRO - Christian Müller-Tomfelde - A <strong>Short</strong> <strong>History</strong> <strong>of</strong> <strong>Tabletop</strong> <strong>Research</strong>, <strong>Technologies</strong> <strong>and</strong> <strong>Products</strong> 7


Active Desk (1995)<br />

• Prototype developed as part <strong>of</strong><br />

the Ontario Telepresence Project<br />

• Back projection <strong>and</strong> cable-based<br />

tracking system<br />

• Deliberate design as an<br />

“Electronic drafting table” with a<br />

30 degrees tilt<br />

• Introduction <strong>of</strong> “graspable<br />

interfaces” or later Tangible User<br />

Interfaces<br />

Fitzmaurice GW, Ishii H, Buxton WAS (1995) Bricks: laying the<br />

foundations for graspable user interfaces. In: Proc CHI ’95, pp 442–449<br />

CSIRO - Christian Müller-Tomfelde - A <strong>Short</strong> <strong>History</strong> <strong>of</strong> <strong>Tabletop</strong> <strong>Research</strong>, <strong>Technologies</strong> <strong>and</strong> <strong>Products</strong> 8


DiamondTouch (2001)<br />

• First multi-touch <strong>and</strong> multi-user<br />

interaction technology<br />

• Developed in the MERLabs<br />

• Often used in collaboration study<br />

• Commercialized by Circle Twelve<br />

in 2009<br />

• Draw-back: top projection <strong>and</strong><br />

“capacitively coupled antennas”<br />

Dietz P, Leigh D (2001) DiamondTouch: A multi-user touch technology. In:<br />

Proc. UIST ’01, pp 219–226<br />

CSIRO - Christian Müller-Tomfelde - A <strong>Short</strong> <strong>History</strong> <strong>of</strong> <strong>Tabletop</strong> <strong>Research</strong>, <strong>Technologies</strong> <strong>and</strong> <strong>Products</strong> 9


Conference Publication Activity<br />

Index based on Lucene indexing <strong>and</strong> retrieval engine<br />

CSIRO - Christian Müller-Tomfelde - A <strong>Short</strong> <strong>History</strong> <strong>of</strong> <strong>Tabletop</strong> <strong>Research</strong>, <strong>Technologies</strong> <strong>and</strong> <strong>Products</strong> 10


Enabling <strong>Technologies</strong><br />

• Enabling technologies for tabletop systems can be<br />

identified in the areas <strong>of</strong><br />

touch, display, <strong>and</strong> s<strong>of</strong>tware technologies<br />

• Early research systems used a large variety <strong>of</strong><br />

general-purpose technology<br />

• technologies were further refined <strong>and</strong><br />

• adopted from other domains.<br />

• Key performance criteria <strong>of</strong> touch technologies are:<br />

• applicability to large display space (> 40 in or 102 cm)<br />

• reliable real-time multi-user touch capability<br />

• low spatial requirements when integrating direct-touch<br />

devices into displays<br />

CSIRO - Christian Müller-Tomfelde - A <strong>Short</strong> <strong>History</strong> <strong>of</strong> <strong>Tabletop</strong> <strong>Research</strong>, <strong>Technologies</strong> <strong>and</strong> <strong>Products</strong> 11


Example: The InteracTable<br />

• part <strong>of</strong> the i-LAND<br />

project in 1999<br />

• a single touch<br />

interactive<br />

• double-mirrored rear<br />

projection<br />

• 1024 by 768 pixel<br />

display<br />

• Second version with a<br />

plasma display panel<br />

2001<br />

CSIRO - Christian Müller-Tomfelde - A <strong>Short</strong> <strong>History</strong> <strong>of</strong> <strong>Tabletop</strong> <strong>Research</strong>, <strong>Technologies</strong> <strong>and</strong> <strong>Products</strong> 12


Display technologies comparison<br />

Godlewski J, Obarowska M (2007) Organic light emitting devices. Opto-<br />

Electronics Review 15(4):179––183<br />

CSIRO - Christian Müller-Tomfelde - A <strong>Short</strong> <strong>History</strong> <strong>of</strong> <strong>Tabletop</strong> <strong>Research</strong>, <strong>Technologies</strong> <strong>and</strong> <strong>Products</strong> 13


OLED <strong>Technologies</strong> (organic light-emitting diode)<br />

http://www.holstcentre.com/Home/Partneringin<strong>Research</strong>/SharedPrograms/TechnologyIntegration/Displays.aspx<br />

http://en.wikipedia.org/wiki/File:OLED_EarlyProduct.JPG<br />

http://www.nbcnews.com/technology/gadgetbox/lg-finally-selling-first-large-screen-oled-hdtv-10-000-1C7776739<br />

http://upload.wikimedia.org/wikipedia/commons/1/12/Ecran_oled_flexible.jpg<br />

Müller-Tomfelde <strong>and</strong> Fjeld - A <strong>Short</strong> <strong>History</strong> <strong>of</strong> <strong>Tabletop</strong> <strong>Research</strong>, <strong>Technologies</strong> <strong>and</strong> <strong>Products</strong> 14


<strong>Tabletop</strong> S<strong>of</strong>tware<br />

• Flexible display s<strong>of</strong>tware<br />

• Rotating display areas on tabletops<br />

• UI customization for direct-touch <strong>and</strong> gestures<br />

• Groupware widgets<br />

• Multi-input <strong>and</strong> multi-user events<br />

• Integration into the operating system<br />

• Reliable multi-event processing<br />

• Support <strong>of</strong> native multi-input <strong>and</strong> legacy applications<br />

• Multi-touch OS extensions<br />

• Windows 7<br />

• On the wish list: “Ubuntu MT Edition”<br />

CSIRO - Christian Müller-Tomfelde - A <strong>Short</strong> <strong>History</strong> <strong>of</strong> <strong>Tabletop</strong> <strong>Research</strong>, <strong>Technologies</strong> <strong>and</strong> <strong>Products</strong> 15


Commercial <strong>Tabletop</strong> <strong>Products</strong><br />

• Example can be found in the past ten to fifteen<br />

years, but they have not yet reached the mass<br />

market.<br />

• Early efforts <strong>and</strong> initiatives<br />

• by small enterprises in cooperation<br />

• with research organizations or initiatives.<br />

• Only recently have larger companies engaged<br />

themselves with interactive tabletops <strong>and</strong> developed<br />

products for the<br />

• entertainment,<br />

• education, <strong>and</strong><br />

• domestic domains.<br />

CSIRO - Christian Müller-Tomfelde - A <strong>Short</strong> <strong>History</strong> <strong>of</strong> <strong>Tabletop</strong> <strong>Research</strong>, <strong>Technologies</strong> <strong>and</strong> <strong>Products</strong> 16


Example: Active Desk<br />

Ontario Telepresence Project (1992-1994)<br />

• OTP was a Canadian research program for<br />

academic <strong>and</strong> industry studies into<br />

• “sociological issues associated with the deployment <strong>of</strong><br />

computer <strong>and</strong> video supported cooperative work<br />

systems.”<br />

• Among other results, the Active Desk was designed<br />

jointly by the OTP <strong>and</strong> a Toronto based company<br />

called Arnott Design Group.<br />

• Out <strong>of</strong> the Arnott Design Group’s work grew another<br />

Toronto based company called Input Technology<br />

Inc. (ITI).<br />

• metaDESK by Ullmer <strong>and</strong> Ishii built upon ITI’s<br />

Visionmaker product.<br />

CSIRO - Christian Müller-Tomfelde - A <strong>Short</strong> <strong>History</strong> <strong>of</strong> <strong>Tabletop</strong> <strong>Research</strong>, <strong>Technologies</strong> <strong>and</strong> <strong>Products</strong> 17


Other Examples<br />

• In 2001 a Germany-based company Wilkhahn<br />

started to distribute the InteracTable,<br />

• An <strong>of</strong>fice table based on<br />

SMART Boards technologies<br />

• Outcome <strong>of</strong> a joint research <strong>and</strong><br />

industry collaboration in 1999.<br />

• Mitsubishi DiamondTouch<br />

• Prototype in 2001, used <strong>of</strong>ten in academic research<br />

• Commercial product in 2009 (Circle Twelve)<br />

• “Turn-key” solutions (2007)<br />

• The SMART Table Model 230i<br />

• Micros<strong>of</strong>t Surface<br />

CSIRO - Christian Müller-Tomfelde - A <strong>Short</strong> <strong>History</strong> <strong>of</strong> <strong>Tabletop</strong> <strong>Research</strong>, <strong>Technologies</strong> <strong>and</strong> <strong>Products</strong> 18


Further Commercial <strong>Products</strong><br />

• reacTable* (2005)<br />

• tangible tabletop interface to<br />

• control synthesizer s<strong>of</strong>tware for<br />

live performances<br />

• PerceptivePixel (2005)<br />

• Jeff Han presented a multi-touch<br />

technology that was very<br />

inspiring <strong>and</strong> heavily<br />

broadcasted in 2006.<br />

• Multi-Touch Collaboration Wall<br />

for use in broadcasting <strong>and</strong><br />

medical visualization.<br />

CSIRO - Christian Müller-Tomfelde - A <strong>Short</strong> <strong>History</strong> <strong>of</strong> <strong>Tabletop</strong> <strong>Research</strong>, <strong>Technologies</strong> <strong>and</strong> <strong>Products</strong> 19


Further Commercial <strong>Products</strong> con’d<br />

• MultiTouch (2008)<br />

• The Helsinki-based spin <strong>of</strong>f company,<br />

Helsinki Institute for Information Technology,<br />

• first commercially available multi-touch<br />

device integrated into a large LCD.<br />

• FlatFrog (2008)<br />

• A Swedish company introduced a multitouch<br />

device based on a patented<br />

technology.<br />

• Close integration into LCD may allow the<br />

construction <strong>of</strong> a significantly less bulky<br />

device.<br />

• http://flatfrog.com/media/FlatFrog_Multitouch_3200_EN_120912.pdf<br />

CSIRO - Christian Müller-Tomfelde - A <strong>Short</strong> <strong>History</strong> <strong>of</strong> <strong>Tabletop</strong> <strong>Research</strong>, <strong>Technologies</strong> <strong>and</strong> <strong>Products</strong> 20


Recent Commercial Product<br />

• Micros<strong>of</strong>t Surface 2<br />

• Surface 2.0 Debut at the 2011<br />

Consumer Electronic Show<br />

• In cooperation with<br />

Samsung SUR40<br />

• Key Features:<br />

• IR sensor integrated<br />

into the LCD PixelSense<br />

• 40 inch diagonal<br />

• 1920x 1080 pixels<br />

• Depth <strong>of</strong> 4 inch<br />

Müller-Tomfelde <strong>and</strong> Fjeld - A <strong>Short</strong> <strong>History</strong> <strong>of</strong> <strong>Tabletop</strong> <strong>Research</strong>, <strong>Technologies</strong> <strong>and</strong> <strong>Products</strong> 21


Analysis: Key Transitions<br />

• We underst<strong>and</strong> a transition as an abstract shift<br />

• From the prevailing approach or paradigm<br />

• To a new one.<br />

• We identify <strong>and</strong> trace three such key transitions.<br />

1. From lab prototypes (1998)<br />

to real world collaborative applications<br />

2. From single touch (2001)<br />

to multi-touch <strong>and</strong> tangibility<br />

3. From projection (2008)<br />

to direct display technology<br />

CSIRO - Christian Müller-Tomfelde - A <strong>Short</strong> <strong>History</strong> <strong>of</strong> <strong>Tabletop</strong> <strong>Research</strong>, <strong>Technologies</strong> <strong>and</strong> <strong>Products</strong> 22


Analysis: Synoptic L<strong>and</strong>scape<br />

• We synthesize historical information into a synoptic<br />

l<strong>and</strong>scape including<br />

• research highlights,<br />

• enabling technologies,<br />

• prototypes, <strong>and</strong> products.<br />

• On top <strong>of</strong> this l<strong>and</strong>scape, we point out <strong>and</strong> trace<br />

• innovations <strong>and</strong><br />

• l<strong>and</strong>mark research<br />

• as they stimulated <strong>and</strong> triggered three key<br />

transitions in research <strong>and</strong> technology.<br />

CSIRO - Christian Müller-Tomfelde - A <strong>Short</strong> <strong>History</strong> <strong>of</strong> <strong>Tabletop</strong> <strong>Research</strong>, <strong>Technologies</strong> <strong>and</strong> <strong>Products</strong> 23


Synoptic L<strong>and</strong>scape<br />

CSIRO. Insert presentation title, do not remove CSIRO from start <strong>of</strong> footer


Synoptic L<strong>and</strong>scape<br />

CSIRO. Insert presentation title, do not remove CSIRO from start <strong>of</strong> footer


Synoptic L<strong>and</strong>scape<br />

CSIRO. Insert presentation title, do not remove CSIRO from start <strong>of</strong> footer


Synoptic L<strong>and</strong>scape<br />

CSIRO. Insert presentation title, do not remove CSIRO from start <strong>of</strong> footer


Synoptic L<strong>and</strong>scape<br />

CSIRO. Insert presentation title, do not remove CSIRO from start <strong>of</strong> footer


<strong>Tabletop</strong> Hype cycle<br />

Müller-Tomfelde <strong>and</strong> Fjeld - A <strong>Short</strong> <strong>History</strong> <strong>of</strong> <strong>Tabletop</strong> <strong>Research</strong>, <strong>Technologies</strong> <strong>and</strong> <strong>Products</strong> 29


<strong>Tabletop</strong> Book at Springer HCI Series<br />

• Facts about the book<br />

• 18 chapter contributions<br />

• Over 50 international authors<br />

• Foreword from Norbert Streitz:<br />

• From Furniture to Interaction Substrate<br />

• Over 450 pages with various figures<br />

• Book has three main parts:<br />

• Under,<br />

• On <strong>and</strong> Above, <strong>and</strong><br />

• Around <strong>and</strong> Beyond <strong>Tabletop</strong>s<br />

CSIRO - Christian Müller-Tomfelde - A <strong>Short</strong> <strong>History</strong> <strong>of</strong> <strong>Tabletop</strong> <strong>Research</strong>, <strong>Technologies</strong> <strong>and</strong> <strong>Products</strong> 30


Book Part 1: Under <strong>Tabletop</strong>s<br />

• Hardware/ S<strong>of</strong>tware <strong>and</strong> Computer Science<br />

• The positioning <strong>of</strong> the chapters topics under<br />

tabletops may not be taken too literally.<br />

• In a boarder sense the chapters <strong>of</strong> this part refer to<br />

topics “hidden from <strong>and</strong> invisible to the user” <strong>and</strong> to<br />

core tabletop technology.<br />

• The part introduces<br />

• basic technical topics <strong>and</strong><br />

• provides a system classification as well as<br />

• more specific aspects such as high-resolution <strong>and</strong><br />

object recognition.<br />

CSIRO - Christian Müller-Tomfelde - A <strong>Short</strong> <strong>History</strong> <strong>of</strong> <strong>Tabletop</strong> <strong>Research</strong>, <strong>Technologies</strong> <strong>and</strong> <strong>Products</strong> 31


Book Part 2: On <strong>and</strong> Above <strong>Tabletop</strong>s<br />

• Human Computer Interaction<br />

• On tabletops refers to interactions on the surface,<br />

namely by<br />

• direct touch input, <strong>and</strong><br />

• gestures,<br />

• mediating tangible objects.<br />

• The part introduces<br />

• Tangibility for interactive surfaces<br />

• Interaction styles <strong>and</strong> basic tabletop actions<br />

• Fundamental limitations <strong>of</strong> direct touch interaction<br />

• 3D tabletop systems.<br />

CSIRO - Christian Müller-Tomfelde - A <strong>Short</strong> <strong>History</strong> <strong>of</strong> <strong>Tabletop</strong> <strong>Research</strong>, <strong>Technologies</strong> <strong>and</strong> <strong>Products</strong> 32


Book Part 3: Around <strong>and</strong> Beyond <strong>Tabletop</strong>s<br />

• Computer Supported Cooperative Work<br />

• The typical group around a tabletop setting is<br />

• Small in size,<br />

• Around a common workspace <strong>and</strong> can also become<br />

• Distributed in scenarios <strong>of</strong> beyond tabletops.<br />

• The part introduces<br />

• Interaction techniques for groups <strong>and</strong> file access<br />

• The concept <strong>of</strong> spatial territories<br />

• <strong>Tabletop</strong> systems in work <strong>and</strong> public environments<br />

• Shared remote tabletops <strong>and</strong> distributed assemblies<br />

CSIRO - Christian Müller-Tomfelde - A <strong>Short</strong> <strong>History</strong> <strong>of</strong> <strong>Tabletop</strong> <strong>Research</strong>, <strong>Technologies</strong> <strong>and</strong> <strong>Products</strong> 33


ICT-Centre<br />

Information Engineering Laboratory<br />

Christian Müller-Tomfelde<br />

<strong>Research</strong> Scientist<br />

Phone: 02 9372 4552<br />

Email: Christian.Mueller-Tomfelde@csiro.au<br />

Web: www.ict.csiro.au<br />

Thank you<br />

Contact Us<br />

Phone: 1300 363 400 or +61 3 9545 2176<br />

Email: enquiries@csiro.au Web: www.csiro.au

Hooray! Your file is uploaded and ready to be published.

Saved successfully!

Ooh no, something went wrong!