Principles of Game Audio and Sound Design

Table of contents

1. Cover
2. Half Title
3. Title Page
4. Copyright Page
5. BRIEF CONTENTS
6. DETAILED CONTENTS
7. 1 Introduction
   1. 1 The Genesis of Audio in Games
   2. 2 From Sample Playback to Procedural Audio
   3. 3 How to Use This Book
      1. What This Book Is
         1. What This Book Isn’t
8. 2 The Role of Audio in Interactive and Immersive Environments
   1. 1 Inform, Entertain, Immerse
      1. 1 Inform: How, What
         1. a Geometry/Environment: Spatial Awareness
         2. b Distance
         3. c Location
         4. d User Feedback and Game Mechanics
      2. 2 Entertain
         1. a Sound Design
         2. b Music and the Mix
      3. 3 Defining Immersion
   2. 2 Challenges of Game Audio
      1. 1 Implementation
      2. 2 Repetition and Fatigue Avoidance
      3. 3 Interactive Elements and Prototyping
      4. 4 Physics
      5. 5 Environmental Sound Design and Modeling
      6. 6 Mixing
      7. 7 Asset Management and Organization
9. 3 The Game Engine Paradigm
   1. 1 What Is a Game Engine
      1. The Unity3D Project Structure
      2. 1 Level Basics 101
         1. a 2D, 3D and Cartesian Coordinates
         2. b World Geometry
         3. c Lighting
         4. d Character Controllers
         5. e Cameras
      3. 2 Elements of a Level
         1. a Everything Is an Object
         2. b Transform
         3. c Sprites
         4. d Meshes
         5. e Models
         6. f Textures
         7. g Shaders
         8. h Materials
         9. i Terrain
         10. j Skyboxes
         11. k Particles Systems
         12. l Colliders
         13. m Triggers/Trigger Zones
         14. n Lighting
         15. o Audio
         16. p Prefabs
   2. 2 Sub Systems
      1. 1 Animation
      2. 2 Input
      3. 3 Physics
         1. Rigidbodies and Collision Detection
         2. Physics Materials
         3. Triggers
         4. Raycasting
      4. 4 Audio
      5. 5 Linear Animation
      6. 6 Additional Sub Systems
10. 4 The Audio Engine and Spatial Audio
    1. 1 Listeners, Audio Clips and Audio Sources
       1. 1 The Audio Listener
          1. Audio Clips
          2. Audio Sources
       2. 2 Audio Source Parameters
       3. 3 Attenuation Shapes and Distance
          1. a Spherical Spreading
          2. b Sound Cones – Directional Audio Sources
          3. c Square/Cube
          4. d Volumetric Sound Sources
          5. e 2D, 3D or 2.5D Audio?
       4. 4 Features of Unity’s Audio Engine
          1. a Audio Filters
          2. b Audio Effects
          3. c Audio Mixers
    2. 2 Audio Localization and Distance Cues
       1. 1 Distance Cues
          1. a Loudness
          2. b Dry to Reflected Sound Ratio
          3. c Low Pass Filtering With Distance
          4. d Spatial Width
       2. 2 Localization Cues
          1. a Localization on the Horizontal Plane
          2. b Localization on the Vertical Plane
       3. 3 Implementing 3D Audio
          1. a Object-based Audio and Binaural Renderings
          2. b Working With HRTFs
          3. c Multichannel Audio and Ambisonics
       4. 4 Optimizing Sound Design for Spatialization
          1. a Putting It all Together
          2. b Working With 2D and Multichannel Audio
          3. c Working With Ambisonics
          4. d Working With Object-Based Audio
11. 5 Sound Design – The Art of Effectively Communicating With Sound
    1. 1 The Art of Sound Design
       1. 1 A Brief History of Sound Design
       2. 2 Sound Design – Basic Considerations
          1. a Effective Sound Design
          2. b Sound Design Guidelines
       3. 3 Getting the Right Tools
          1. a Equalization
          2. b Dynamic Range
          3. c Reverberation
          4. d Harmonic Processors
          5. e Metering Tools
          6. f Utilities
       4. 4 Microphones
          1. a Microphone Choice: Dynamic vs. Condensers
          2. b Mic Placement
       5. 5 Sound Design – Before You Start
          1. a Always Use High Quality Material
          2. b Don’t Get Too Attached
          3. c Build and Learn
          4. d Listen for the Expected and the Unexpected
          5. e Layers
          6. f Be Organized
          7. g Communicate
          8. h Experiment, Experiment, Experiment
    2. 2 Basic Techniques
       1. 1 Layering/Mixing
       2. 2 Pitch Shifting
          1. a Playback Speed Modulation
          2. b Granular Synthesis
          3. c Fast Fourier Transform-Based Algorithms
       3. 3 Distortion
          1. a Saturation
          2. b Overdrive
          3. c Distortion
          4. d Bit Crushing
       4. 4 Compression
          1. a Blending Through Bus Compression
          2. b Transient Control
          3. c Inflation
       5. 5 Equalization/Filtering
          1. a Equalization for Sound Design
          2. b Resonance Simulation
       6. 6 Harmonic Generators/Aural Exciters
       7. 7 Granular Synthesis and Granulation of Sampled Sounds
          1. a Granular Synthesis Terminology
          2. b Sound Design Applications of Granular Synthesis
       8. 8 DSP Classics
          1. a Ring Modulation/Amplitude Modulation
          2. b Comb Filtering/Resonators
       9. 9 Reverberation
          1. a Indoors vs. Open Air
          2. b Reverb Parameters
          3. c Reverberation for Environmental Modeling
          4. d Reverberation as a Dramatic Tool
       10. 10 Convolution
           1. a Optimization
           2. b Speaker and Electronic Circuit Emulation
           3. c Filtering/Very Small Space Emulation
           4. d Hybrid Tones
       11. 11 Time-Based Modulation FX
           1. a Chorus
           2. b Flanger
           3. c Phasers
           4. d Tremolo
       12. 12 Foley Recording
12. 6 Practical Sound Design
    1. 1 Setting Up a Sound Design Session and Signal Flow
       1. 1 Signal Flow
          1. a Input
          2. b Inserts
          3. c Pre-Fader Send
          4. d Volume Fader
          5. e Metering: Pre-Fader vs. Post Fader
          6. f Post-Fader Send
          7. g Output
       2. 2 Working With Video
          1. a Know Your Frame Rate
       3. 3 Clipping Is Easy – Mind the Signal Path
          1. Use the Dynamic Range
       4. 4 Setting Up a Basic Session for Linear Mixes and Cut Scenes
          1. a Music, Dialog and Sound Effects
          2. b Inserts vs. Effects Loops for Reverberation
          3. c Setting Up the Mix Session
          4. d Master Output and Sub Maste
          5. e Submixes and Effects Loops
          6. f Further Enhancements
    2. 2 Practical Sound Design and Prototyping
       1. 1 Guns
          1. a One Shot vs. Loops
          2. b General Considerations
          3. c Designing a Gunshot
       2. 2 Prototyping Vehicles
          1. a Specifications
          2. b Selecting Your Material
          3. c Processing and Preparing Your Material
          4. d Building a Prototype
       3. 3 Creature Sounds
          1. a Primary vs. Secondary Sounds
          2. b Emotional Span
          3. c Working With Vocal Recordings
          4. d Working With Animal Samples
          5. e Working With Non-Human or Animal Samples
       4. 4 An Adaptive Crowd Engine Prototype in MaxMSP
13. 7 Coding for Game Audio
    1. 1 Why Learn to Code?
       1. 1 Syntax and Logic
       2. 2 Algorithms
       3. 3 Basic Object-Oriented Programming Concepts
          1. a Procedural vs. Object-Oriented
          2. b Encapsulation and Inheritance
    2. 2 An Intro to C#: Syntax and Basics
       1. 1 Our First Script
       2. 2 Variables, Constants, Data Types Operators, Arrays and Lists
          1. a Data Types
          2. b Variables
          3. c Arrays
          4. d Lists
          5. e Access Modifiers
       3. 3 Accessing a Function From Another Class
    3. 3 Playing Audio in Unity
       1. 1 Our First Audio Script
       2. 2 Play() vs. PlayOneShot()
       3. 3 Using Triggers
       4. 4 Sample Randomization
       5. 5 Detecting Keyboard Events
       6. 6 Audio-Specific Issues
          1. a Timing – Frame Rate vs. Absolute Time
          2. b Linear vs. Logarithmic Amplitude
14. 8 Implementing Audio: Common Scenarios
    1. 1 Before You Start: Preparing Your Assets
    2. 2 Ambiences and Loops
       1. 1 Creating Ambiences and Loops
          1. a Seamless Loops
          2. b Creating a Simple Loop – Looping Techniques
          3. c Creating Variations
       2. 2 Implementing Our Loops in a Unity Level
          1. a Challenges
          2. b Spatial Distribution
          3. c Working With the Time Property to Avoid Phasing Issues
       3. 3 Random Emitters
          1. a A Simple Random Emitter Algorithm
          2. b Coroutines
       4. 4 Ambiences, Putting It All Together
       5. 5 Sample Concatenation
          1. a Creating Variations With Footsteps Samples
          2. b Case 1: Swapping Audio Clips
          3. c Case 2: Using PlayScheduled()
       6. 6 Collisions
          1. a Detecting Collision
          2. b Velocity-based Sample Selection
       7. 7 Raycasting and Smart Audio Sources
          1. a Implementing Occlusion With Raycasting
          2. b Avoiding the Pebble Effect
       8. 8 Animation Events
       9. 9 Audio Fades
       10. 10 Distance Crossfades
       11. 11 Working With Prefabs
           1. a Creating a Smart Intermittent Emitter Prefab With Occlusion
           2. b Instantiating a Prefab From Scripting
           3. c Destroying an Object Instantiated From a Prefab
           4. d Instantiating Audio Emitters at Random Locations in 3D
15. 9 Environmental Modeling
    1. 1 What Is Environmental Modeling?
       1. 1 Reverberation
          1. a Pre-Computed vs. Real Time Computation
          2. b Absorption Coefficients
          3. c Environmental Modeling With Reverberation in Unity
          4. d Unity’s Reverberation Parameters
       2. 2 Best Practices for Environmental Modeling
          1. a Late vs. Early Reflections
          2. b Reflections Level
          3. c Density and Diffusion
          4. d High Frequencies vs. Low Frequencies
       3. 3 Reverb Zones, Effects Loops and Audio Reverb Filters
          1. a Reverb Zones
          2. b Adding Reverb as an Effect Loop Using the Mixer
          3. c Audio Reverb Filters
    2. 2 Distance Modeling
       1. 1 Filtering as a Product of Distance
          1. a Adding a Low Pass Filter That Will Modulate its Cutoff Frequency Based on Distance
          2. b Width Perception as Product of Distance
          3. c Dry to Wet Ratio as a Product of Distance
          4. d Distance Simulation: Putting It All Together
    3. 3 Additional Factors
       1. 1 Occlusion, Obstruction, Exclusion
          1. a Occlusion
          2. b Obstruction
          3. c Exclusion
       2. 2 Distance Crossfades
       3. 3 Doppler Effect
16. 10 Procedural Audio: Beyond Samples
    1. 1 Introduction, Benefits and Drawbacks
       1. 1 What Is Procedural Audio?
          1. a Procedural Audio, Pros and Cons
          2. b Approaches to Procedural Audio
    2. 2 Practical Procedural Audio: A Wind Machine and a Sword Collision Model
       1. 1 A Wind Machine in MaxMSP With Subtractive Synthesis
          1. Making the Model Flexible
       2. 2 A Sword Maker in MaxMSP With Linear Modal Synthesis
          1. Spectral Analysis
          2. Modeling the Impulse
          3. Modeling the Resonances
          4. Making the Model Flexible
17. 11 Adaptive Mixing
    1. 1 What’s in a Mix? Inform and Entertain (Again)
       1. 1 Mix Considerations
       2. 2 Music, Dialogue and Sound Effects
       3. 3 Planning and Pre-Production
          1. a SubMixing
          2. b Routing
          3. c Dynamic Range
          4. d Passive vs. Active Mix Events
    2. 2 The Unity Audio Mixer
       1. 1 Adding Groups to the Unity Mixer
       2. 2 The Audio Group Inspector
       3. 3 Working With Views and Colors in the Unity Mixer
          1. Creating Views in Unity
       4. 4 Adding Effects to Groups in Unity
       5. 5 Inserts vs. Effect Loops
       6. 6 Setting Up an Effect Loop for Reverberation in Unity Using Send and Receive
          1. Note on Adjusting Levels During Gameplay
       7. 7 Ducking in Unity
          1. Setting Up a Ducking Compressor in Unity
    3. 3 Snapshots, Automation and Game States
       1. 1 Working With Snapshots
       2. 2 Recalling Snapshots via Scripting
       3. 3 Editing Mixer and Plugin Parameters via Scripting
       4. 4 Exposing a Parameter: Controlling a Volume Slider
    4. 4 Good Practices
       1. Mix Levels
18. 12 Audio Data Reduction
    1. 1 Digital Audio: A Quick Review
       1. 1 Pulse Code Modulation
       2. 2 File Size Calculation
    2. 2 Data Reduction Strategies
       1. 1 Speech vs. Generic Audio
       2. 2 Bit Rates
       3. 3 Perceptual Coding
          1. The Trade-Off
       4. 4 Common File Formats
          1. a MP3
          2. b Advanced Audio Coding
          3. c Ogg Vorbis
          4. d AC-3 Dolby Digital
          5. e Adaptive Differential Pulse Code Modulation
    3. 3 Data Reduction Good Practices
    4. 4 Data Reduction Options in Unity
       1. 1 File Options
       2. 2 Load Type
       3. 3 Compression Formats Options
          1. Sample Rate Setting
19. Index