Sabine Coquillart Greg Welch. Guido Brunnett. Editors. Virtual Realities. Dagstuhl Seminar SpringerWienNewYork

W

•

Sabine Coquillart G Greg Welch

•

Guido Brunnett

Editors

Virtual Realities Dagstuhl Seminar 2008

SpringerWienNewYork

Editors Dr. Sabine Coquillart INRIA Rhône-Alpes ZIRST avenue de I’Europe 655 38334 St.Ismier France [email protected] Prof. Dr. Guido Brunnett TU Chemnitz Fak. Informatik Professur für Graphische Datenverabeitung und Visualisierung Straße der Nationen 62 09107 Chemnitz Germany [email protected]

Dr. Greg Welch University of North Carolina Dept. Computer Science Sitterson Hall 27599-3175 Chapel Hill North Carolina Campus Box 3175 USA [email protected]

This work is subject to copyright. All rights are reserved, whether the whole or part of the material is concerned, specifically those of translation, reprinting, re-use of illustrations, broadcasting, reproduction by photocopying machines or similar means, and storage in data banks. Product Liability: The publisher can give no guarantee for all the information contained in this book. The use of registered names, trademarks, etc. in this publication does not imply, even in the absence of a specific statement, that such names are exempt from the relevant protective laws and regulations and therefore free for general use. © 2011 Springer-Verlag/Wien Printed in Germany SpringerWienNewYork is a part of Springer Science+Business Media springer.at Cover: WMXDesign GmbH, Heidelberg, Germany Typesetting: SPI Publisher Services, India Printed on acid-free and chlorine-free bleached paper SPIN: 12530366 With 78 Figures

ISBN 978-3-211-99177-0 e-ISBN 978-3-211-99178-7 DOI 10.1007/978-3-211-99178-7 SpringerWienNewYork

Preface

During the week of June 1–6, 2008, the Schloss Dagstuhl – Leibniz Center for Informatics held a first-of-its-kind seminar in the area of Virtual Reality. Being the first seminar in this area, both the organizers and the participants were not completely sure what to expect from this event beforehand. In retrospect, we rate this Dagstuhl seminar as a great success: it was the ideal venue to define and discuss key topics and to initiate new collaborations.

What is Virtual Reality? The definition of Virtual Reality (VR) depends on whom you ask and in what context. For the purpose of planning this seminar, we defined it as a multidisciplinary area of research aimed at interactive human–computer-mediated simulations of artificial environments. This definition captures applications ranging from simulation, to training, to scientific visualization, and to even entertainment. An important aspect of such VR-based systems is the stimulation of the human senses – typically sight, sound, and touch – such that a user feels a sense of presence (or immersion) in the virtual environment. Different applications require different levels of presence, with corresponding levels of realism, sensory immersion, and spatiotemporal interactive fidelity. These requirements lead to research ranging from human perception, to psychological aspects, to physical simulation, and to VR technologies including tracking, displays, user interfaces, rendering, and modeling.

Who Attended the Seminar? We succeeded in bringing together a good mix of leading researchers and promising young scientists. In total, 50 researchers from 11 different countries participated in the seminar; nine of them were women. The attendees were mostly affiliated with universities, with a few coming from research institutes (e.g., MPI, Fraunhofer) or industry. v

vi

Preface

What Transpired During the Seminar? The format of the seminar sessions changed during the week. Based on the responses from the participants, the organizers tried to adapt to evolving needs for more discussions or lectures. One idea that turned out to be very fruitful was to concentrate the discussion toward the end of each session, i.e., the talks were presented without the usual question part. Instead at the end of the session, all speakers took a seat in front of the audience and answered questions in panel style. This format allowed questions to bridge between the talks, and stirred very lively discussions. These discussions were considered to be so productive that the lengths of the talks had to be reduced in order to free up time. On Wednesday morning, the entire group met for discussion of the “grand challenges” facing VR. Based on the results of this session, different topics were defined for further discussion in subsequent parallel sessions. On Thursday afternoon, the group split up to discuss the following themes: Latency, Augmented Reality, Experience Design, Virtual Humans, and Perception. As these themes were selected by popular vote from a larger list, it is an indication that these topics were considered some of the most important in VR.

Major VR Themes Discussed Latency is recognized by VR researchers as a topic that is important but underreported in the field. The group identified the main sources of latency: tracking and other input devices, interface buffering, network delays, device driver and operating system overheads, application simulation time, rendering (software and hardware), and display devices. In a typical system, the end-to-end latency is the sum of these. It is this end-to-end timing that was identified as the key measurement for comparing different systems, and for understanding the impact of varying levels of latency on task performance. There is also a need to measure the latencies in the contributing components. The discussion of existing methods for measuring and reporting end-to-end latency made clear that these methods will have to be straightforward to use if they are to be adopted widely by the VR researchers and practitioners. Two variants were proposed: one for HMDs and the other for screen/projector-based systems, and the group agreed to further develop these for dissemination. More generally, it was agreed that there is an urgent need for a ‘field guide to latency’. In Augmented Reality, the observer’s view of a real scene is enhanced by virtual objects. Examples include to blend in repair instructions into the field of view of a car mechanic, or to project MRI data onto a patient during a surgery. Obviously, AR requires accurate tracking of the user. In moving environments, it might be necessary to combine different kind of tracking devices in order to avoid situations, where the tracked markers are lost. To make use of the full potential of AR, it is important

Preface

vii

to install AR infrastructure not only in laboratories but also in usual environments (e.g., offices). In this context, the concept of AR-ready buildings was discussed. The group that worked on Experience Design focused on the creative use of existing VR equipment to create compelling experiences. This extremely fruitful discussion took advantage from the different views of engineers, psychologists, and practitioners in multimedia design. The discussion made clear that not only the technical possibilities but also the script of the presentation has a strong effect on the experience made by the user. Visitors of virtual worlds are usually more impressed when technical explanations (how it is done) are not given in advance. Depending on the application, it can also be advantageous to leave certain details to the imagination of the observer. Also, the element of surprise is an effective means for getting the observer involved in the virtual environment. An extremely important research topic in VR is that of virtual humans. Similar to the Turing test of AI, the goal is here to design the appearance of virtual humans and their interaction with the user so life-like that emotions are invoked similar to those in interhuman interactions. One application presented in Dagstuhl was a training environment for medical doctors. To improve their interaction with patients, a virtual environment has been created in which difficult situations between doctors and patients can be simulated. In the perception group, it was discussed how recent results from perceptual psychology could be used for the design of intuitive 3D human computer interfaces. Another issue of this group was the problem of simulation sickness, and the question of how the length of time humans can be exposed to virtual environments can be extended.

This Book This book comprises a collection of position papers, documentation of breakout discussions, or extended research papers related to VR. The position papers include topics such as constrained 3D user interfaces, social gaming and learning applications, and future VR software platforms. The breakout discussion papers included the next generation of Augmented Reality systems, and the importance of experiential fidelity. The extended research papers include topics ranging from fundamental methods to compelling applications. All of the submissions were peer-reviewed for appropriateness and quality. TU Chemnitz, DE INRIA Rhˆone-Alpes, FR University of North Carolina, Chapel Hill, US

Guido Brunnett Sabine Coquillart Greg Welch

•

Contents

1

Proposals for Future Virtual Environment Software Platforms . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . Anthony Steed

1

2

Augmented Reality 2.0 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . 13 Dieter Schmalstieg, Tobias Langlotz, and Mark Billinghurst

3

Experiential Fidelity: Leveraging the Mind to Improve the VR Experience . . .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . 39 Steffi Beckhaus and Robert W. Lindeman

4

Social Gaming and Learning Applications: A Driving Force for the Future of Virtual and Augmented Reality? .. . . . . . . .. . . . . . . 51 Ralf D¨orner, Benjamin Lok, and Wolfgang Broll

5

[Virtual C 1] * Reality . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . 77 Steffi Beckhaus

6

Action Capture: A VR-Based Method for Character Animation . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . 97 Bernhard Jung, Heni Ben Amor, Guido Heumer, and Arnd Vitzthum

7

Cloth Simulation Based Motion Capture of Dressed Humans . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . .123 Nils Hasler, Bodo Rosenhahn, and Hans-Peter Seidel

8

Remote 3D Medical Consultation .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . .139 Greg Welch, Diane H. Sonnenwald, Henry Fuchs, Bruce Cairns, M.D., Ketan Mayer-Patel, Ruigang Yang, Andrei State, Herman Towles, Adrian Ilie, Srinivas Krishnan, and Hanna M. S¨oderholm

ix

x

Contents

9

SEE MORE: Improving the Usage of Large Display Environments . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . .161 Achim Ebert, Hans Hagen, Torsten Bierz, Matthias Deller, Peter-Scott Olech, Daniel Steffen, and Sebastian Thelen

10 Inner Sphere Trees and Their Application to Collision Detection.. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . .181 Rene Weller and Gabriel Zachmann 11 The Value of Constraints for 3D User Interfaces . . . . . . . . . . . . . . . . . .. . . . . . .203 Wolfgang Stuerzlinger and Chadwick A. Wingrave 12 Evaluation of a Scalable In-Situ Visualization System Approach in a Parallelized Computational Fluid Dynamics Application .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . .225 Sebastian Manten, Michael Vetter, and Stephan Olbrich Author Index. . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . .239 Subject Index . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . .249

Contributors

Heni Ben Amor VR and Multimedia Group, Institute of Informatics, TU Bergakademie Freiberg, Freiberg, Germany, [email protected] Steffi Beckhaus im.ve, Department of Informatics, University of Hamburg, Vogt-K¨olln-Str. 30, 22767 Hamburg, Germany, [email protected] Torsten Bierz University of Kaiserslautern, D-67663 Kaiserslautern, Germany, [email protected] Mark Billinghurst The HIT Lab NZ, University of Canterbury, Christchurch, New Zealand, [email protected] Wolfgang Broll Ilmenau University of Technology, D 98693 Ilmenau, Germany, [email protected] Bruce Cairns The Department of Surgery, The University of North Carolina at Chapel Hill, Campus Box 7228, Chapel Hill, North Carolina 27599-7228, USA, bruce [email protected] Matthias Deller German Research Center for Artificial Intelligence (DFKI), D-67663 Kaiserslautern, Germany, [email protected] Ralf D¨orner RheinMain University of Applied Sciences, D 65197 Wiesbaden, Germany, [email protected] Achim Ebert University of Kaiserslautern, D-67663 Kaiserslautern, Germany, [email protected] Henry Fuchs The Department of Computer Science, The University of North Carolina at Chapel Hill, Campus Box 3175, Chapel Hill, North Carolina 27599-3175, USA, [email protected] Hans Hagen University of Kaiserslautern, D-67663 Kaiserslautern, Germany, [email protected] Nils Hasler Weta Digital, Wellington, New Zealand, [email protected] and MPI Informatik, Saarbr¨ucken, Germany

xi

xii

Contributors

Guido Heumer VR and Multimedia Group, Institute of Informatics, TU Bergakademie Freiberg, Freiberg, Germany, [email protected] Adrian Ilie The Department of Computer Science, The University of North Carolina at Chapel Hill, Campus Box 3175, Chapel Hill, North Carolina 27599-3175, USA, [email protected] Bernhard Jung VR and Multimedia Group, Institute of Informatics, TU Bergakademie Freiberg, Freiberg, Germany, [email protected], http://vr.tu-freiberg.de Srinivas Krishnan The Department of Computer Science, The University of North Carolina at Chapel Hill, Campus Box 3175, Chapel Hill, North Carolina 27599-3175, USA, [email protected] Tobias Langlotz Institute for Computer Graphics and Vision, Graz University of Technology, Graz, Austria, [email protected] Robert W. Lindeman HIVE Lab, Department of Computer Science, Worcester Polytechnic Institute, 100 Institute Rd., Worcester, MA 01609, USA, [email protected] Benjamin Lok University of Florida, 32611-6120, Gainesville, FL USA, [email protected] Sebastian Manten Center of Information and Media Technology, Heinrich Heine University Duesseldorf, 40225, Duesseldorf, Germany, [email protected] Ketan Mayer-Patel The Department of Computer Science, The University of North Carolina at Chapel Hill, Campus Box 3175, Chapel Hill, North Carolina 27599-3175, USA, [email protected] Stephan Olbrich Regional Computer Center and Department of Computer Science, Scientific Visualization and Parallel Processing, University of Hamburg, 20146 Hamburg, Germany, [email protected] Peter-Scott Olech University of Kaiserslautern, D-67663 Kaiserslautern, Germany, [email protected] Bodo Rosenhahn Institut f¨ur Informationsverarbeitung, Hannover University, Hannover, Germany, [email protected] Dieter Schmalstieg Institute for Computer Graphics and Vision, Graz University of Technology, Graz, Austria, [email protected] Hanna M. S¨oderholm The Swedish School of Library and Information Science, G¨oteborg University and the University College of Bor˚as, 50190 Bor˚as, Sweden, [email protected] Diane H. Sonnenwald School of Information and Library Studies, University College Dublin, Belfeild, Dublin 4, Ireland, [email protected]

Contributors

xiii

Hans-Peter Seidel MPI Informatik, Saarbr¨ucken, Germany, [email protected] Andrei State The Department of Computer Science, The University of North Carolina at Chapel Hill, Campus Box 3175, Chapel Hill, North Carolina 27599-3175, USA, [email protected] Anthony Steed Department of Computer Science, University College London, London, UK, [email protected] Daniel Steffen German Research Center for Artificial Intelligence (DFKI), D-67663 Kaiserslautern, Germany, [email protected] Wolfgang Stuerzlinger York University, Toronto, Canada, http://www.cse.yorku. ca/wolfgang, [email protected] Sebastian Thelen University of Kaiserslautern, D-67663 Kaiserslautern, Germany, s [email protected] Herman Towles The Department of Computer Science, The University of North Carolina at Chapel Hill, Campus Box 3175, Chapel Hill, North Carolina 27599-3175, USA, [email protected] Michael Vetter Department of Computer Science, Scientific Visualization and Parallel Processing, University of Hamburg, D-20146 Hamburg, Germany, [email protected] Arnd Vitzthum VR and Multimedia Group, Institute of Informatics, TU Bergakademie Freiberg, Freiberg, Germany, [email protected] Greg Welch The Department of Computer Science, The University of North Carolina at Chapel Hill, Campus Box 3175, Chapel Hill, North Carolina 27599-3175, USA, [email protected] Rene Weller Department of Computer Science, Clausthal University, D-38678 Clausthal-Zellerfeld, Germany, [email protected] Chadwick A. Wingrave University of Central Florida, Orlando, FL, USA, [email protected] Ruigang Yang The Department of Computer Science, The University of Kentucky, 232 Hardymon Building, Lexington, KY 40506-0195, USA, [email protected] Gabriel Zachmann Department of Computer Science, Clausthal University, D-38678 Clausthal-Zellerfeld, Germany, [email protected]