Programming Interactivity

Programming Interactivity

by Joshua Noble
Released July 2009
Publisher(s): O'Reilly Media, Inc.
ISBN: 9780596154141

Read it now on the O’Reilly learning platform with a 10-day free trial.

O’Reilly members get unlimited access to books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.

Buy on Amazon
Start your free trial

Book description

Make cool stuff. If you're a designer or artist without a lot of programming experience, this book will teach you to work with 2D and 3D graphics, sound, physical interaction, and electronic circuitry to create all sorts of interesting and compelling experiences -- online and off.

Programming Interactivity explains programming and electrical engineering basics, and introduces three freely available tools created specifically for artists and designers:

  • Processing, a Java-based programming language and environment for building projects on the desktop, Web, or mobile phones
  • Arduino, a system that integrates a microcomputer prototyping board, IDE, and programming language for creating your own hardware and controls
  • OpenFrameworks, a coding framework simplified for designers and artists, using the powerful C++ programming language

BTW, you don't have to wait until you finish the book to actually make something. You'll get working code samples you can use right away, along with the background and technical information you need to design, program, build, and troubleshoot your own projects. The cutting edge design techniques and discussions with leading artists and designers will give you the tools and inspiration to let your imagination take flight.

Publisher resources

View/Submit Errata

Table of contents

  1. A Note Regarding Supplemental Files
  2. Preface
    1. Who This Book Is For
    2. How This Book Is Organized
    3. What Is—and Isn’t—in This Book
      1. What’s in
      2. What’s Not in
    4. Companion Website
    5. Typographical Conventions Used in This Book
    6. Using Code Examples
    7. Safari® Books Online
    8. We’d Like to Hear from You
    9. Acknowledgments
  3. I. Introductions
    1. 1. Introducing Interaction Design
      1. What This Book Is for
      2. Programming for Interactivity
        1. The Nature of Interaction
        2. Messages and Interaction
        3. Interfaces and Interaction
        4. Languages of Interaction
      3. Design and Interaction
      4. Art and Interaction
      5. Data Exchange and Exploration
      6. Working Process
    2. 2. Programming Basics
      1. Why You’ll Read This Chapter More Than Once
      2. The Nature of Code
      3. Variables
        1. Simple Types
          1. int
          2. Signed versus unsigned
          3. float
          4. char
          5. bool or boolean
          6. string
          7. byte
          8. long
        2. Arrays
        3. Casting
        4. Operators
      4. Control Statements
        1. if/then
        2. for Loop
        3. while Loop
        4. continue
        5. break
      5. Functions
        1. Defining a Function
        2. Passing Parameters to a Function
        3. Some Suggestions on Writing Functions
        4. Overloading Functions
      6. Objects and Properties
      7. Scope
      8. Review
    3. 3. Processing
      1. Downloading and Installing Processing
      2. Exploring the Processing IDE
      3. The Basics of a Processing Application
        1. The setup() Method
          1. The size() method
          2. The frameRate() method
          3. The print() method
        2. The draw() Method
      4. The Basics of Drawing with Processing
        1. The rect(), ellipse(), and line() Methods
        2. RGB Versus Hexadecimal
        3. The fill() Method
        4. The background() Method
        5. The line() Method
        6. The stroke() and strokeWeight() Methods
        7. The curve() Method
        8. The vertex() and curveVertex() Methods
      5. Capturing Simple User Interaction
        1. The mouseX and mouseY Variables
        2. The mousePressed() Method
        3. The mouseReleased() and mouseDragged() Methods
        4. The keyPressed and key Variables
      6. Importing Libraries
        1. Downloading Libraries
      7. Loading Things into Processing
        1. Loading and Displaying Images
          1. The PImage class
          2. The loadImage() method
          3. The image() method
        2. Displaying Videos in the Processing Environment
        3. Using the Movie Class
        4. Reading and Writing Files
          1. The loadStrings() method
          2. The saveStrings() method
      8. Running and Debugging Applications
      9. Exporting Processing Applications
      10. Conclusion
      11. Review
    4. 4. Arduino
      1. Starting with Arduino
        1. Installing the IDE
          1. Mac OS X
          2. Windows
          3. Linux
        2. Configuring the IDE
      2. Touring Two Arduino Boards
        1. The Controller
        2. Duemilanove Versus Mini
          1. What’s a pin?
          2. Digital versus analog pins
          3. Pins on the Mini versus Duemilanove
      3. Touring the Arduino IDE
      4. The Basics of an Arduino Application
        1. The setup Statement
        2. The loop Method
      5. Features of the Arduino Language
        1. Constants
        2. Methods
        3. Arrays
        4. Strings
      6. How to Connect Things to Your Board
      7. Hello World
      8. Debugging Your Application
      9. Importing Libraries
      10. Running Your Code
        1. Running Your Board Without a USB Connection
      11. Review
    5. 5. Programming Revisited
      1. Object-Oriented Programming
      2. Classes
        1. The Basics of a Class
        2. Class Rules
      3. Public and Private Properties
      4. Inheritance
      5. Processing: Classes and Files
      6. C++: Classes and Files
        1. .cpp and .h
        2. A Simple C++ Application
      7. Pointers and References
        1. Reference
        2. Pointer
        3. When to Use Pointers
        4. Some Rules for Pointers
        5. Pointers and Arrays
        6. When Are You Going to Use This?
      8. Review
    6. 6. openFrameworks
      1. Your IDE and Computer
        1. Windows
          1. Visual Studio
          2. Code::Blocks for Windows
        2. Mac OS X
        3. Linux
          1. Code::Blocks for Linux
          2. Using makefiles
      2. Taking Another Quick Tour of C++
        1. Basic Variable Types
        2. Arrays
        3. Methods
        4. Classes and Objects in C++
      3. Getting Started with oF
      4. Touring an oF Application
        1. Methods
        2. Variables
      5. Creating “Hello, World”
      6. Drawing in 2D
        1. Setting Drawing Modes
        2. Drawing Polygons
      7. Displaying Video Files and Images
        1. Images
        2. Video
      8. Importing Libraries
        1. ofxOpenCv
        2. ofxVectorGraphics
        3. ofxVectorMath
        4. ofxNetwork
        5. ofxOsc
      9. Compiling an oF Program
        1. Compiling in Xcode
        2. Compiling in Code::Blocks
      10. Debugging an oF Application
        1. Using the printf Statement
        2. Using the GNU Debugger
        3. Using the Debugger in Xcode
        4. Using the Debugger in Code::Blocks
      11. Review
  4. II. Themes
    1. 7. Sound and Audio
      1. Sound As Feedback
      2. Sound and Interaction
      3. How Sound Works on a Computer
      4. Audio in Processing
        1. Instantiating the Minim Library
        2. Generating Sounds with Minim
        3. Filtering Sounds with Minim
      5. Sound in openFrameworks
      6. openFrameworks and the FMOD Ex Library
      7. The Sound Object Library
      8. The Magic of the Fast Fourier Transform
      9. Physical Manipulation of Sound with Arduino
      10. A Quick Note on PWM
      11. Creating Interactions with Sound
      12. Further Resources
      13. Review
    2. 8. Physical Input
      1. Interacting with Physical Controls
      2. Thinking About Kinetics
      3. Getting Gear for This Chapter
      4. Controlling Controls
        1. The Button As an Electrical Object
        2. The Button As an Interactive Object
        3. The Button As a Value in Code
      5. Turning Knobs
        1. The Dial As an Interactive Object
        2. Potentiometers
      6. Using Lights
        1. Wiring an LED
      7. Detecting Touch and Vibration
        1. Reading a Piezo Sensor
        2. Getting Piezo Sensors
      8. Communicating with Other Applications
      9. Sending Messages from the Arduino
        1. openFrameworks
      10. Detecting Motion
        1. PIR Motion Sensor
      11. Reading Distance
        1. Reading Input from an Infrared Sensor
      12. Understanding Binary Numbers
        1. Binary Numbers
        2. Bits and Bit Operations
        3. Why Do You Need to Know Any of This?
      13. Detecting Forces and Tilt
      14. Introducing I2C
      15. What Is a Physical Interface?
      16. What’s Next
      17. Review
    3. 9. Programming Graphics
      1. The Screen and Graphics
      2. Seeing Is Thinking, Looking Is Reading
      3. Math, Graphics, and Coordinate Systems
      4. Drawing Strategies
        1. Use Loops to Draw
        2. Use Arrays to Draw
        3. Draw Only What You Need
        4. Use Sprites
      5. Processing and Transformation Matrices
      6. Creating Motion
        1. Shaping the Gaze
        2. Setting the Mood
        3. Creating Tweens
      7. Using Vectors
      8. Using Graphical Controls
        1. ControlP5 Library
        2. Event Handling
      9. Importing and Exporting Graphics
        1. Using PostScript in Processing
        2. Using PostScript Files in oF
      10. What’s Next
      11. Review
    4. 10. Bitmaps and Pixels
      1. Using Pixels As Data
      2. Using Pixels and Bitmaps As Input
      3. Providing Feedback with Bitmaps
      4. Looping Through Pixels
      5. Manipulating Bitmaps
        1. Manipulating Color Bytes
        2. Using Convolution in Full Color
      6. Analyzing Bitmaps in oF
        1. Analyzing Color
        2. Analyzing Brightness
        3. Detecting Motion
        4. Using Edge Detection
      7. Using Pixel Data
      8. Using Textures
        1. Textures in oF
        2. Textures in Processing
      9. Saving a Bitmap
      10. What’s Next
      11. Review
    5. 11. Physical Feedback
      1. Using Motors
        1. DC Motors
        2. Stepper Motors
        3. Other Options
      2. Using Servos
        1. Connecting a Servo
        2. Communicating with the Servo
        3. Wiring a Servo
      3. Using Household Currents
      4. Working with Appliances
      5. Introducing the LilyPad Board
      6. Using Vibration
      7. Using an LED Matrix
        1. Using the Matrix Library
        2. Using the LedControl Library
        3. Using the SPI Protocol
      8. Using LCDs
        1. Serial LCD
      9. Using Solenoids for Movement
      10. What’s Next
      11. Review
    6. 12. Protocols and Communication
      1. Communicating Over Networks
      2. Using XML
      3. Understanding Networks and the Internet
        1. Network Organization
        2. Network Identification
        3. Network Data Flow
      4. Handling Network Communication in Processing
        1. Client Class
        2. Server Class
        3. Sharing Data Across Applications
      5. Understanding Protocols in Networking
      6. Using ofxNetwork
      7. Creating Networks with the Arduino
        1. Initializing the Ethernet Library
        2. Creating a Client Connection
        3. Creating a Server Connection
      8. Using Carnivore to Communicate
        1. Installing the Carnivore Library
        2. Creating a Carnivore Client
      9. Communicating with Bluetooth
        1. Using Bluetooth in Processing
          1. Installing Bluetooth on Linux and Windows
          2. Installing Bluetooth on Mac OS X
        2. Using the bluetoothDesktop Library
        3. Using the Arduino Bluetooth
      10. Communicating Using MIDI
      11. Review
  5. III. Explorations
    1. 13. Graphics and OpenGL
      1. What Does 3D Have to Do with Interaction?
      2. Understanding 3D
      3. Working with 3D in Processing
        1. Lighting in Processing
        2. Controlling the Viewer’s Perspective
      4. Making Custom Shapes in Processing
      5. Using Coordinates and Transforms in Processing
      6. Working with 3D in OpenGL
        1. So, What Is OpenGL?
        2. Transformations
        3. OpenGL in Processing
        4. Open GL in openFrameworks
      7. Using Matrices and Transformations in OpenGL
      8. Using Vertices in OpenGL
      9. Drawing with Textures in oF
      10. Lighting in OpenGL
      11. Blending Modes in OpenGL
      12. Using Textures and Shading in Processing
        1. Applying Material Properties
      13. Using Another Way of Shading
        1. What Does GLSL Look Like?
        2. Vertex Shaders
        3. Fragment Shader
        4. Variables Inside Shaders
      14. Using an ofShader Addon
      15. What to Do Next
      16. Review
    2. 14. Detection and Gestures
      1. Computer Vision
        1. Interfaces Without Controls
        2. Example CV Projects
      2. OpenCV
      3. Using Blobs and Tracking
        1. Starting with ofxOpenCV
        2. Tracking Blobs with ofxOpenCV
          1. Taking a quick foray into virtual methods
          2. Using the velocity of blobs
      4. Using OpenCV in Processing
        1. Exploring Further in OpenCV
      5. Detecting Gestures
        1. Using ezGestures in Processing
        2. Using Gestures in oF
      6. Implementing Face Recognition
      7. Exploring Touch Devices with oF
        1. TouchKit
        2. Tuio
        3. Touchlib
        4. reacTIVision
      8. What’s Next
      9. Review
    3. 15. Movement and Location
      1. Using Movement As and in Interaction
      2. Using Software-Based Serial Ports
      3. Understanding and Using GPS
      4. Storing Data
      5. Logging GPS Data to an Arduino
        1. Using the Breadcrumbs Library
        2. Implementing Hardware-Based Logging
      6. Sending GPS Data
      7. Determining Location by IP Address
      8. What to Do Next
      9. Review
    4. 16. Interfaces and Controls
      1. Examining Tools, Affordances, and Aesthetics
      2. Reexamining Tilt
      3. Exploring InputShield
      4. Understanding Touch
      5. Exploring Open Source Touch Hardware
        1. Nort_/D
        2. Liquidware TouchShield
        3. Drawing to the TouchShield Screen
        4. Controlling Servos Through the TouchShield
        5. Setting Up Communication Between Arduino and TouchShield
      6. Communicating Using OSC
      7. Using the Wiimote
        1. Using the Wii Nunchuck in Arduino
        2. Tracking Wii Remote Positioning in Processing
      8. What’s Next
      9. Review
    5. 17. Spaces and Environments
      1. Using Architecture and Space
      2. Sensing Environmental Data
      3. Using an XBee with Arduino
        1. Creating a Simple Test
        2. Configuring the XBee Module
        3. Addressing in the XBee
        4. XBee Library for Processing
      4. Placing Objects in 2D
      5. Using the X10 Protocol
      6. Setting Up an RFID Sensor
      7. Reading Heat and Humidity
      8. What’s Next
      9. Review
    6. 18. Further Resources
      1. What’s Next?
        1. Software Tools
          1. ARToolKit
          2. PureData
          3. Max/MSP
          4. vvvv
          5. Flash
        2. Construction Processes
        3. Artificial Intelligence
          1. Neural nets
          2. Pathfinding
          3. Genetic algorithms
          4. Artificial life
        4. Physics
          1. Chipmunk
          2. Box2D
          3. Other Processing physics libraries
          4. ofAddons
        5. Hardware Platforms
          1. Phidgets
          2. Robotshop Rover
          3. Fritzing
          4. AVR
          5. ARM
          6. PIC
      2. Bibliography
        1. Interaction Design
        2. Programming
        3. Hardware
        4. Art
      3. Conclusion
  6. A. Circuit Diagram Symbols
  7. Programming Glossary
  8. Index
  9. Colophon
  10. Copyright

Product information

  • Title: Programming Interactivity
  • Author(s): Joshua Noble
  • Release date: July 2009
  • Publisher(s): O'Reilly Media, Inc.
  • ISBN: 9780596154141