Analyzing Data in the Internet of Things

Analyzing Data in the Internet of Things

by Ashish Thusoo
Released May 2016
Publisher(s): O'Reilly Media, Inc.
ISBN: 9781491959008

Read it now on the O’Reilly learning platform with a 10-day free trial.

O’Reilly members get unlimited access to books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.

Start your free trial

Book description

The Internet of Things (IoT) is growing fast. According to the International Data Corporation (IDC), more than 28 billion things will be connected to the Internet by 2020—from smartwatches and other wearables to smart cities, smart homes, and smart cars. This O’Reilly report dives into the IoT industry through a series of illuminating talks and case studies presented at 2015 Strata + Hadoop World Conferences in San Jose, New York, and Singapore.

Among the topics in this report, you’ll explore the use of sensors to generate predictions, using data to create predictive maintenance applications, and modeling the smart and connected city of the future with Kafka and Spark.

Case studies include:

  • Using Spark Streaming for proactive maintenance and accident prevention in railway equipment
  • Monitoring subway and expressway traffic in Singapore using telco data
  • Managing emergency vehicles through situation awareness of traffic and weather in the smart city pilot in Oulu, Finland
  • Capturing and routing device-based health data to reduce cardiovascular disease
  • Using data analytics to reduce human space flight risk in NASA’s Orion program

This report concludes with a discussion of ethics related to algorithms that control things in the IoT. You’ll explore decisions related to IoT data, as well as opportunities to influence the moral implications involved in using the IoT.

Publisher resources

View/Submit Errata

Table of contents

  1. Introduction
  2. I. Data Processing and Architecture for the IoT
  3. 1. Data Acquisition and Machine-Learning Models
    1. Modeling Machine Failure
      1. Root Cause Analysis
      2. Application Across Industries
      3. A Demonstration: Microsoft Cortana Analytics Suite
      4. Data Needed to Model Machine Failure
      5. Training a Machine-Learning Model
      6. Getting Started with Predictive Maintenance
      7. Feature Engineering Is Key
      8. Three Different Modeling Techniques
      9. Start Collecting the Right Data
  4. 2. IoT Sensor Devices and Generating Predictions
    1. Sampling Bias and Data Sparsity
    2. Minimizing the Minimization Error
    3. Constrained Throughput
    4. Implementing Deep Learning
  5. 3. Architecting a Real-Time Data Pipeline with Spark Streaming
    1. What Features Should a Smart City Have?
      1. Free Internet Access
      2. Two-Way Communication with City Officials
      3. Data Belongs to the Public
      4. Empower Cities to Hire Great Developers
    2. Designing a Real-Time Data Pipeline with the MemCity App
    3. The Real-Time Trinity
    4. Building the In-Memory Application
    5. Streamliner for IoT Applications
    6. The Lambda Architecture
  6. 4. Using Spark Streaming to Manage Sensor Data
    1. Architectural Considerations
    2. Visualizing Time-Series Data
    3. The Importance of Sliding Windows
    4. Checkpoints for Fault Tolerance
    5. Start Your Application from the Checkpoint
  7. II. Case Studies in IoT Data
  8. 5. Monitoring Traffic in Singapore Using Telco Data
    1. Understanding the Data
    2. Developing Real-Time Recommendations for Train Travel
    3. Expressway Data
  9. 6. Oulu Smart City Pilot
    1. Managing Emergency Vehicles, Weather, and Traffic
    2. Creating Situation Awareness
  10. 7. An Open Source Approach to Gathering and Analyzing Device-Sourced Health Data
    1. Generating Personal Health Data
    2. Applications for Personal Health Data
    3. The Health eHeart Project
      1. Health eHeart Challenges
  11. 8. Leverage Data Analytics to Reduce Human Space Mission Risks
    1. Over 300,000 Measurements of Data
    2. Microsecond Timestamps
    3. Identifying Patterns in the Data
    4. Our Goal: A Flexible Analytics Environment
    5. Using Real-Time and Machine Learning
    6. Analytics Using Stream and Batch Processing
  12. III. Ethics of Algorithms in IoT
  13. 9. How Are Your Morals? Ethics in Algorithms and IoT
    1. Beta Representations of Values
    2. Choosing How We Want to Be Represented

Product information

  • Title: Analyzing Data in the Internet of Things
  • Author(s): Ashish Thusoo
  • Release date: May 2016
  • Publisher(s): O'Reilly Media, Inc.
  • ISBN: 9781491959008