Augmented Reality: Principles and Practice

Augmented Reality: Principles and Practice

by Dieter Schmalstieg, Tobias Höllerer
Released June 2016
Publisher(s): Addison-Wesley Professional
ISBN: 9780133153217

Read it now on the O’Reilly learning platform with a 10-day free trial.

O’Reilly members get unlimited access to books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.

Start your free trial

Book description

Today’s Comprehensive and Authoritative Guide to Augmented Reality

By overlaying computer-generated information on the real world, augmented reality (AR) amplifies human perception and cognition in remarkable ways. Working in this fast-growing field requires knowledge of multiple disciplines, including computer vision, computer graphics, and human-computer interaction. Augmented Reality: Principles and Practice integrates all this knowledge into a single-source reference, presenting today’s most significant work with scrupulous accuracy. Pioneering researchers Dieter Schmalstieg and Tobias Höllerer carefully balance principles and practice, illuminating AR from technical, methodological, and user perspectives.

Coverage includes

  • Displays: head-mounted, handheld, projective, auditory, and haptic

  • Tracking/sensing, including physical principles, sensor fusion, and real-time computer vision

  • Calibration/registration, ensuring repeatable, accurate, coherent behavior

  • Seamless blending of real and virtual objects

  • Visualization to enhance intuitive understanding

  • Interaction–from situated browsing to full 3D interaction

  • Modeling new geometric content

  • Authoring AR presentations and databases

  • Architecting AR systems with real-time, multimedia, and distributed elements

    • This guide is indispensable for anyone interested in AR, including developers, engineers, students, instructors, researchers, and serious hobbyists.

    Table of contents

    1. About This E-Book
    2. Title Page
    3. Copyright Page
    4. Dedication Page
    5. Contents
    6. Preface
      1. Enter: Augmented Reality
      2. Why a Book on Augmented Reality?
      3. What’s in the Book?
      4. How to Use the Book and the Related Material
      5. Companion Website
    7. Acknowledgments
    8. About the Authors
    9. Chapter 1. Introduction to Augmented Reality
      1. Definition and Scope
      2. A Brief History of Augmented Reality
      3. Examples
        1. Industry and Construction
        2. Maintenance and Training
        3. Medical
        4. Personal Information Display
        5. Navigation
        6. Television
        7. Advertising and Commerce
        8. Games
      4. Related Fields
        1. Mixed Reality Continuum
        2. Virtual Reality
        3. Ubiquitous Computing
      5. Summary
    10. Chapter 2. Displays
      1. Multimodal Displays
        1. Audio Displays
        2. Haptic, Tactile, and Tangible Displays
        3. Olfactory and Gustatory Displays
      2. Visual Perception
      3. Requirements and Characteristics
        1. Method of Augmentation
        2. Ocularity and Stereoscopy
        3. Focus
        4. Occlusion
        5. Resolution and Refresh Rate
        6. Field of View
        7. Viewpoint Offset
        8. Brightness and Contrast
        9. Distortions and Aberrations
        10. Latency
        11. Ergonomics
        12. Social Acceptance
      4. Spatial Display Model
      5. Visual Displays
        1. Near-Eye Displays
        2. Handheld Displays
        3. Stationary Displays
        4. Projected Displays
      6. Summary
    11. Chapter 3. Tracking
      1. Tracking, Calibration, and Registration
      2. Coordinate Systems
        1. Model Transformation
        2. View Transformation
        3. Projective Transformation
        4. Frames of Reference
      3. Characteristics of Tracking Technology
        1. Physical Phenomena
        2. Measurement Principle
        3. Measured Geometric Property
        4. Sensor Arrangement
        5. Signal Sources
        6. Degrees of Freedom
        7. Measurement Coordinates
        8. Spatial Sensor Arrangement
        9. Workspace Coverage
        10. Measurement Error
        11. Temporal Characteristics
      4. Stationary Tracking Systems
        1. Mechanical Tracking
        2. Electromagnetic Tracking
        3. Ultrasonic Tracking
      5. Mobile Sensors
        1. Global Positioning System
        2. Wireless Networks
        3. Magnetometer
        4. Gyroscope
        5. Linear Accelerometer
        6. Odometer
      6. Optical Tracking
        1. Model-Based versus Model-Free Tracking
        2. Illumination
        3. Markers versus Natural Features
        4. Target Identification
      7. Sensor Fusion
        1. Complementary Sensor Fusion
        2. Competitive Sensor Fusion
        3. Cooperative Sensor Fusion
      8. Summary
    12. Chapter 4. Computer Vision for Augmented Reality
      1. Marker Tracking
        1. Camera Representation
        2. Marker Detection
        3. Pose Estimation from Homography
        4. Pose Refinement
      2. Multiple-Camera Infrared Tracking
        1. Blob Detection
        2. Establishing Point Correspondences
        3. Triangulation from Two Cameras
        4. Triangulation from More Than Two Cameras
        5. Matching Targets Consisting of Spherical Markers
        6. Absolute Orientation
      3. Natural Feature Tracking by Detection
        1. Interest Point Detection
        2. Descriptor Creation
        3. Descriptor Matching
        4. Perspective-n-Point Pose
        5. Robust Pose Estimation
      4. Incremental Tracking
        1. Active Search
        2. Kanade-Lucas-Tomasi Tracking
        3. Zero-Normalized Cross-Correlation
        4. Hierarchical Search
        5. Combined Detection and Tracking
      5. Simultaneous Localization and Mapping
        1. Five-Point Algorithm for Essential Matrix
        2. Bundle Adjustment
        3. Parallel Tracking and Mapping
        4. Relocalization and Loop Closure
        5. Dense Mapping
      6. Outdoor Tracking
        1. Scalable Visual Matching
        2. Prior Information from Sensors
        3. Prior Information from Geometry
        4. Simultaneous Tracking, Mapping, and Localization
      7. Summary
    13. Chapter 5. Calibration and Registration
      1. Camera Calibration
        1. Internal Camera Parameters
        2. Correcting Lens Distortion
      2. Display Calibration
        1. Single Point Active Alignment Method
        2. Head-Mounted Display Calibration Using a Pointing Device
        3. Hand–Eye Calibration
      3. Registration
        1. Geometric Measurement Distortions
        2. Error Propagation
        3. Latency
        4. Filtering and Prediction
      4. Summary
    14. Chapter 6. Visual Coherence
      1. Registration
      2. Occlusion
        1. Occlusion Refinement
        2. Probabilistic Occlusion
        3. Model-Free Occlusion
      3. Photometric Registration
        1. Image-Based Lighting
        2. Light Probes
        3. Offline Light Capturing
        4. Photometric Registration from Static Images
        5. Photometric Registration from Specular Reflections
        6. Photometric Registration from Diffuse Reflections
        7. Photometric Registration from Shadows
        8. Outdoor Photometric Registration
        9. Reconstructing Explicit Light Sources
      4. Common Illumination
        1. Differential Rendering
        2. Real-Time Global Illumination
        3. Shadows
        4. Diffuse Global Illumination
        5. Specular Global Illumination
      5. Diminished Reality
        1. Determination of the Region of Interest
        2. Observation and Modeling of the Hidden Area
        3. Removal of the Region of Interest
        4. Projector-Based Diminished Reality
      6. Camera Simulation
        1. Lens Distortion
        2. Blur
        3. Noise
        4. Vignetting
        5. Chromatic Aberrations
        6. Bayer Pattern Artifacts
        7. Tone Mapping Artifacts
      7. Stylized Augmented Reality
      8. Summary
    15. Chapter 7. Situated Visualization
      1. Challenges
        1. Data Overload
        2. User Interaction
        3. Registration Errors
        4. Visual Interference
        5. Temporal Coherence
      2. Visualization Registration
        1. Locally Registered Situated Visualization
        2. Globally Registered Situated Visualization
        3. Registration Uncertainty
      3. Annotations and Labeling
        1. Labeling Fundamentals
        2. Optimization Techniques
        3. Temporal Coherence
        4. Image-Guided Placement
        5. Legibility
      4. X-Ray Visualization
        1. Ghostings from Object Space
        2. Ghostings from Image Space
        3. Implementation with G-Buffers
      5. Spatial Manipulation
        1. Explosion Diagrams
        2. Space Distortion
      6. Information Filtering
        1. Knowledge-Based Filter
        2. Spatial Filter
        3. Combined Knowledge-Based and Spatial Filter
      7. Summary
    16. Chapter 8. Interaction
      1. Output Modalities
        1. Augmentation Placement
        2. Agile Displays
        3. Magic Lenses
      2. Input Modalities
        1. Tracking and Manipulation of Rigid Objects
        2. Body Tracking
        3. Gestures
        4. Touch
        5. Physically Based Interfaces
      3. Tangible Interfaces
        1. Tangibles on Surfaces
        2. Tangibles with Generic Shape
        3. Tangibles with Distinct Shapes
        4. Transparent Tangibles
      4. Virtual User Interfaces on Real Surfaces
      5. Augmented Paper
      6. Multi-view Interfaces
        1. Multi-display Focus+Context
        2. Shared Space
        3. Multiple Locales
        4. Cross-View Interaction
      7. Haptic Interaction
      8. Multimodal Interaction
      9. Conversational Agents
      10. Summary
    17. Chapter 9. Modeling and Annotation
      1. Specifying Geometry
        1. Points
        2. Planes
        3. Volumes
      2. Specifying Appearance
      3. Semi-automatic Reconstruction
      4. Free-Form Modeling
      5. Annotation
      6. Summary
    18. Chapter 10. Authoring
      1. Requirements of AR Authoring
        1. Real-World Interfaces
        2. Hardware Abstraction
        3. Authoring Workflow
      2. Elements of Authoring
        1. Actors
        2. Story
        3. Stages
        4. Interactions
        5. Setup
      3. Stand-Alone Authoring Solutions
        1. Desktop Authoring
        2. Authoring by Performance
      4. Plug-In Approaches
      5. Web Technology
      6. Summary
    19. Chapter 11. Navigation
      1. Foundations of Human Navigation
      2. Exploration and Discovery
      3. Route Visualization
      4. Viewpoint Guidance
        1. Guidance Toward a Target Object
        2. Guidance Toward a Target Viewpoint
      5. Multiple Perspectives
        1. Simultaneous Multiple Perspectives
        2. Transitional Interfaces
      6. Summary
    20. Chapter 12. Collaboration
      1. Properties of Collaboration Systems
      2. Co-located Collaboration
        1. Individual Displays and Views
        2. Gaze Awareness
        3. Agile Collaboration in Shared Space
      3. Remote Collaboration
        1. Video Sharing
        2. Video Sharing with Virtual Objects
        3. Video Sharing with Geometric Reconstruction
        4. Pointing and Gestures
        5. Remote Collaboration with Agile Users
      4. Summary
    21. Chapter 13. Software Architectures
      1. AR Application Requirements
        1. Environment Control and Scene Dynamics
        2. Display Space
        3. Real–Virtual Consistency
        4. Semantic Knowledge
        5. Physical Space
      2. Software Engineering Requirements
        1. Platform Abstraction
        2. User Interface Abstraction
        3. Reusability and Extensibility
        4. Distributed Computing
        5. Decoupled Simulation
      3. Distributed Object Systems
        1. Object Management
        2. Case Study: SHEEP
      4. Dataflow
        1. Dataflow Graphs
        2. Multimodal Interaction
        3. Threads and Scheduling
        4. Case Study: Wearable Augmented Reality Setup
      5. Scene Graphs
        1. Fundamentals of Scene Graphs
        2. Dependency Graph
        3. Scene Graph Integration
        4. Distributed Shared Scene Graph
      6. Developer Support
        1. Parameter Configuration
        2. Declarative Scripting
        3. Case Study: Augmented Reality Tour Guide
        4. Procedural Scripting
        5. Mixed-Language Programming
        6. Runtime Reconfiguration
        7. Choosing an AR Platform
      7. Summary
    22. Chapter 14. The Future
      1. What May Drive Business Cases
        1. Professional Users
        2. Consumers
      2. An AR Developer’s Wish List
        1. Low-Level Camera API
        2. Multiple Cameras
        3. Wide-Field-of-View Cameras
        4. Sensors
        5. Unified Memory
        6. Parallel Programming on the Mobile GPU
        7. Better Displays
      3. Taking AR Outdoors
        1. Uncooperative Users
        2. Limited Device Capabilities
        3. Localization Success Rate
      4. Interfacing with Smart Objects
      5. Confluence of Virtual Reality and Augmented Reality
      6. Augmented Humans
      7. AR as a Dramatic Medium
      8. AR as a Social Computing Platform
      9. Summary
    23. References
    24. Index
    25. Code Snippets

    Product information

    • Title: Augmented Reality: Principles and Practice
    • Author(s): Dieter Schmalstieg, Tobias Höllerer
    • Release date: June 2016
    • Publisher(s): Addison-Wesley Professional
    • ISBN: 9780133153217