Programming Interactivity, 2nd Edition

Programming Interactivity, 2nd Edition

by Joshua Noble
Released January 2012
Publisher(s): O'Reilly Media, Inc.
ISBN: 9781449311445

Read it now on the O’Reilly learning platform with a 10-day free trial.

O’Reilly members get unlimited access to books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.

Buy on Amazon Buy on ebooks.com
Start your free trial

Book description

Ready to create rich interactive experiences with your artwork, designs, or prototypes? This is the ideal place to start. With this hands-on guide, you’ll explore several themes in interactive art and design—including 3D graphics, sound, physical interaction, computer vision, and geolocation—and learn the basic programming and electronics concepts you need to implement them. No previous experience is necessary.

You’ll get a complete introduction to three free tools created specifically for artists and designers: the Processing programming language, the Arduino microcontroller, and the openFrameworks toolkit. You’ll also find working code samples you can use right away, along with the background and technical information you need to design, program, and build your own projects.

  • Learn cutting-edge techniques for interaction design from leading artists and designers
  • Let users provide input through buttons, dials, and other physical controls
  • Produce graphics and animation, including 3D images with OpenGL
  • Use sounds to interact with users by providing feedback, input, or an element they can control
  • Work with motors, servos, and appliances to provide physical feedback
  • Turn a user’s gestures and movements into meaningful input, using Open CV

Publisher resources

View/Submit Errata

Table of contents

  1. Programming Interactivity
  2. A Note Regarding Supplemental Files
  3. Preface
    1. What Is—and Isn’t—in This Book
      1. What’s In
      2. What’s Not In
    2. Conventions Used in This Book
    3. Companion Website
    4. Using Code Examples
    5. Safari® Books Online
    6. How to Contact Us
    7. Acknowledgments
  4. 1. Introducing Interaction Design
    1. What This Book Is For
    2. Programming for Interactivity
      1. The Nature of Interaction
      2. Messages and Interaction
      3. Interfaces and Interaction
      4. Languages of Interaction
    3. Design and Interaction
    4. Art and Interaction
    5. Data Exchange and Exploration
    6. Working Process
  5. 2. Programming Basics
    1. Why You’ll Read This Chapter More Than Once
    2. The Nature of Code
    3. Variables
      1. Simple Types
        1. int
        2. Signed versus unsigned
        3. float
        4. char
        5. bool or boolean
        6. string
        7. byte
        8. long
      2. Arrays
      3. Casting
      4. Operators
    4. Control Statements
      1. if/then
      2. for Loop
      3. while Loop
      4. continue
      5. break
    5. Functions
      1. Defining a Function
      2. Passing Parameters to a Function
      3. Some Suggestions on Writing Functions
      4. Overloading Functions
    6. Objects and Properties
    7. Scope
    8. Review
  6. 3. Processing
    1. Downloading and Installing Processing
    2. Exploring the Processing IDE
    3. The Basics of a Processing Application
      1. The setup() Method
        1. The size() method
        2. The frameRate() method
        3. The print() method
      2. The draw() Method
    4. The Basics of Drawing with Processing
      1. The rect(), ellipse(), and line() Methods
      2. RGB Versus Hexadecimal
      3. The fill() Method
      4. The background() Method
      5. The line() Method
      6. The stroke() and strokeWeight() Methods
      7. The curve() Method
      8. The vertex() and curveVertex() Methods
    5. Capturing Simple User Interaction
      1. The mouseX and mouseY Variables
      2. The mousePressed() Method
      3. The mouseReleased() and mouseDragged() Methods
      4. The keyPressed and key Variables
    6. Importing Libraries
      1. Downloading Libraries
    7. Loading External Data into Processing
      1. Loading and Displaying Images
        1. The PImage class
        2. The loadImage() method
        3. The image() method
      2. Displaying Videos in the Processing Environment
      3. Using the Movie Class
      4. Reading and Writing Files
        1. The loadStrings() method
        2. The saveStrings() method
    8. Running and Debugging Applications
    9. Exporting Processing Applications
    10. Conclusion
    11. Review
  7. 4. Arduino
    1. Starting with Arduino
      1. Installing the IDE
        1. Mac OS X
        2. Windows
        3. Linux
      2. Configuring the IDE
    2. Touring an Arduino Board
      1. The Controller
      2. Uno Versus Mini Versus Mega
        1. What’s a pin?
        2. Digital versus analog pins
        3. Pins on the Mini
        4. Pins on the Mega
    3. Touring the Arduino IDE
    4. The Basics of an Arduino Application
      1. The setup Statement
      2. The loop Method
    5. Features of the Arduino Language
      1. Constants
      2. Methods
      3. Arrays
      4. Strings
    6. How to Connect Things to Your Board
      1. Resistors
    7. Hello World
    8. Debugging Your Application
    9. Importing Libraries
    10. Running Your Code
      1. Running Your Board Without a USB Connection
    11. Review
  8. 5. Programming Revisited
    1. Object-Oriented Programming
    2. Classes
      1. The Basics of a Class
      2. Class Rules
    3. Public and Private Properties
    4. Inheritance
    5. Processing: Classes and Files
    6. C++: Classes and Files
      1. .cpp and .h
      2. A Simple C++ Application
    7. Pointers and References
      1. Pointer
      2. Reference
      3. When to Use Pointers
      4. When to Use References
      5. Some Rules for Pointers
      6. Some Rules for References
      7. Pointers and Arrays
      8. When Are You Going to Use This?
    8. Review
  9. 6. openFrameworks
    1. Your IDE and Computer
      1. Windows
        1. Visual Studio
        2. Code::Blocks for Windows
      2. Mac OS X
      3. Linux
        1. Code::Blocks for Linux
        2. Using makefiles
    2. Taking Another Quick Tour of C++
      1. Basic Variable Types
      2. Arrays
      3. Methods
      4. Classes and Objects in C++
    3. Getting Started with oF
    4. Touring an oF Application
      1. Methods
      2. Variables
    5. Creating “Hello, World”
    6. Drawing in 2-D
      1. Setting Drawing Modes
      2. Drawing Polygons
    7. Displaying Video Files and Images
      1. Images
      2. Video
    8. Compiling an oF Program
      1. Compiling in Xcode
      2. Compiling in Code::Blocks
    9. Debugging an oF Application
      1. Using the cout Statement
      2. Using the GNU Debugger
      3. Using the Debugger in Xcode
      4. Using the Debugger in Code::Blocks
    10. Importing Libraries
      1. ofxOpenCv
      2. ofxVectorGraphics
      3. ofxAssimpModelLoader
      4. ofxNetwork
      5. ofxOsc
    11. Review
  10. 7. Physical Input
    1. Interacting with Physical Controls
    2. Thinking About Kinetics
    3. Getting Gear for This Chapter
    4. Controlling Controls
      1. The Button As an Electrical Object
      2. The Button As an Interactive Object
      3. Reading the Value of a Button
    5. Turning Knobs
      1. The Dial As an Interactive Object
      2. Potentiometers
    6. Using Lights
      1. Wiring an LED
    7. Detecting Touch and Vibration
      1. Reading a Piezo Sensor
      2. Getting Piezo Sensors
    8. Detecting Proximity
      1. Proximity Through Capacitance
    9. Detecting Motion
      1. PIR Motion Sensor
    10. Reading Distance
      1. Reading Input from an Infrared Sensor
    11. Understanding Binary Numbers
      1. Binary Numbers
      2. Bits and Bit Operations
      3. Why Do You Need to Know Any of This?
    12. Communicating with Other Applications
    13. Sending Messages from the Arduino
      1. openFrameworks
    14. Detecting Forces and Tilt
    15. Introducing I2C
      1. Gyroscopes
    16. What’s Next
    17. Review
  11. 8. Programming Graphics
    1. The Screen and Graphics
    2. Seeing Is Thinking, Looking Is Reading
    3. Math, Graphics, and Coordinate Systems
    4. Drawing Strategies
      1. Use Loops to Draw
      2. Use Vectors to Draw
      3. Draw Only What You Need
      4. Use Sprites
    5. Transformation Matrices
    6. Creating Motion
      1. Shaping the Gaze
      2. Setting the Mood
      3. Creating Tweens
    7. Using Vectors
    8. Using Graphical Controls
      1. ControlP5 Library
      2. Event Handling
    9. Importing and Exporting Graphics
      1. Using PostScript in Processing
      2. Using PostScript Files in oF
    10. What’s Next
    11. Review
  12. 9. Bitmaps and Pixels
    1. Using Pixels As Data
    2. Using Pixels and Bitmaps As Input
    3. Providing Feedback with Bitmaps
    4. Looping Through Pixels
      1. ofPixels
    5. Manipulating Bitmaps
      1. Manipulating Color Bytes
      2. Using Convolution in Full Color
    6. Analyzing Bitmaps in oF
      1. Analyzing Color
      2. Analyzing Brightness
      3. Detecting Motion
    7. Using Edge Detection
    8. Using Pixel Data
    9. Using Textures
      1. Textures in oF
      2. Textures in Processing
    10. Saving a Bitmap
    11. What’s Next
    12. Review
  13. 10. Sound and Audio
    1. Sound As Feedback
    2. Sound and Interaction
    3. How Sound Works on a Computer
    4. Audio in Processing
      1. Instantiating the Minim Library
      2. Generating Sounds with Minim
      3. Filtering Sounds with Minim
    5. Sound in openFrameworks
    6. openFrameworks and the FMOD Ex Library
    7. Maximilian
    8. Physical Manipulation of Sound with Arduino
    9. A Quick Note on PWM
    10. Creating Interactions with Sound
    11. Further Resources
    12. Review
  14. 11. Arduino and Feedback
    1. Using Motors
      1. DC Motors
      2. Stepper Motors
      3. Motor Shields
    2. Smart Feedback
    3. Using Servos
      1. Connecting a Servo
      2. Communicating with the Servo
      3. Wiring a Servo
    4. Using Household Currents
    5. Working with Appliances
    6. Introducing the LilyPad Board
    7. Using Vibration
      1. Nano, Fio, and Mini
    8. Using an LED Matrix
      1. Using the LEDControl Library
    9. Using the SPI Protocol
    10. Serial LED Matrix
    11. Using LCDs
    12. Using Solenoids for Movement
    13. What’s Next
    14. Review
  15. 12. Protocols and Communication
    1. Communicating over Networks
    2. Using XML
    3. Understanding Networks and the Internet
      1. Network Organization
      2. Network Identification
      3. Network Data Flow
    4. Handling Network Communication in Processing
      1. Client Class
      2. Server Class
      3. Sharing Data Across Applications
    5. Understanding Protocols in Networking
    6. Using the ofxNetwork Add-on
    7. Creating Networks with the Arduino
      1. Initializing the Ethernet Library
      2. Creating a Client Connection
      3. Creating a Server Connection
    8. Wireless Internet on the Arduino
    9. Communicating with Bluetooth
      1. Using Bluetooth in Processing
        1. Installing Bluetooth on Linux and Windows
        2. Installing Bluetooth on Mac OS X
      2. Using the bluetoothDesktop Library
    10. Communicating Using MIDI
    11. Review
  16. 13. Graphics and OpenGL
    1. What Does 3-D Have to Do with Interaction?
    2. Understanding 3-D
    3. What Is OpenGL?
    4. Working with 3-D in Processing
      1. OpenGL in Processing
      2. Lighting in Processing
      3. Controlling the Viewer’s Perspective
    5. Making Custom Shapes in Processing
    6. Using Coordinates and Transforms in Processing
      1. Transformations
    7. 3-D in openFrameworks
      1. Drawing in 3-D
      2. Transformations in openFrameworks
    8. Lighting in OpenGL
    9. Blending Modes in OpenGL
    10. Creating 3-D Objects in oF
    11. Using Textures and Shading in Processing
    12. Using Another Way of Shading
      1. What Does GLSL Look Like?
      2. Vertex Shaders
      3. Geometry Shader
      4. Fragment Shader
      5. Variables Inside Shaders
    13. Using ofShader
    14. Using Shaders in Processing
    15. What to Do Next
    16. Review
  17. 14. Motion and Gestures
    1. Computer Vision
      1. Interfaces Without Controls
      2. Example CV Projects
    2. OpenCV
    3. Using Blobs and Tracking
      1. Starting with ofxOpenCV
    4. Detecting Features with oF
    5. Using OpenCV in Processing
    6. Feature Tracking in Processing
    7. Using Blob Tracking with Physics
    8. Exploring Further in OpenCV
    9. Detecting Gestures
      1. Using ezGestures in Processing
      2. Using Gestures in oF
    10. Capturing iOS gestures with oF
    11. Touch with oF
      1. Tuio
      2. reacTIVision
      3. CCV
    12. What’s Next
    13. Using the Microsoft Kinect
      1. Processing
      2. openFrameworks
    14. Review
  18. 15. Movement and Location
    1. Using Movement As and In Interaction
    2. Using Software-Based Serial Ports
    3. Understanding and Using GPS
    4. Storing Data
    5. Logging GPS Data to an Arduino
      1. Using the breadcrumbs Library
      2. Implementing Hardware-Based Logging
    6. Sending GPS Data
    7. Getting Location on a Mobile Device
      1. On the iPhone
      2. On an Android Phone
    8. What to Do Next
    9. Review
  19. 16. Spaces and Environments
    1. Using Architecture and Space
    2. Sensing Environmental Data
    3. Using an XBee with Arduino
      1. Creating a Simple Test
      2. Configuring the XBee Module
      3. Addressing in the XBee
      4. XBee Library for Processing
    4. Placing Objects in 2-D
    5. Using the X10 Protocol
    6. Setting Up an RFID Sensor
    7. Reading Heat and Humidity
    8. Determine Position of an Object
    9. What’s Next
    10. Review
  20. 17. Further Resources
    1. What’s Next?
      1. Software Tools
        1. ARToolKit
        2. PureData
        3. Max/MSP
        4. vvvv
        5. Flash
      2. Construction Processes
      3. Artificial Intelligence
        1. Neural nets
        2. Pathfinding
        3. Genetic algorithms
        4. Artificial life
      4. Physics
        1. Chipmunk
        2. Box2D
        3. Other Processing physics libraries
        4. ofAddons
      5. Hardware Platforms
        1. Phidgets
        2. Robotshop Rover
        3. Fritzing
        4. AVR
        5. ARM
        6. PIC
    2. Bibliography
      1. Interaction Design
      2. Programming
      3. Hardware
      4. Art
    3. Conclusion
  21. Index
  22. About the Author
  23. Colophon
  24. Copyright

Product information

  • Title: Programming Interactivity, 2nd Edition
  • Author(s): Joshua Noble
  • Release date: January 2012
  • Publisher(s): O'Reilly Media, Inc.
  • ISBN: 9781449311445