Architecting IoT Solutions on Azure

Architecting IoT Solutions on Azure

by Blaize Stewart
Released January 2024
Publisher(s): O'Reilly Media, Inc.
ISBN: 9781098142865

Read it now on the O’Reilly learning platform with a 10-day free trial.

O’Reilly members get unlimited access to books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.

Buy on Amazon Buy on ebooks.com
Start your free trial

Book description

How can you make sense of the complex IoT landscape? With dozens of components ranging from devices to metadata about the devices, it's easy to get lost among the possibilities. But it's not impossible if you have the right guide to help you navigate all the complexities. This practical book shows developers, architects, and IT managers how to build IoT solutions on Azure.

Author Blaize Stewart presents a comprehensive view of the IoT landscape. You'll learn about devices, device management at scale, and the tools Azure provides for building globally distributed systems. You'll also explore ways to organize data by choosing the appropriate dataflow and data storage technologies. The final chapters examine data consumption and solutions for delivering data to consumers with Azure.

Get the architectural guidance you need to create holistic solutions with devices, data, and everything in between. This book helps you:

  • Meet the demands of an IoT solution with Azure-provided functionality
  • Use Azure to create complete scalable and secure IoT systems
  • Understand how to articulate IoT architecture and solutions
  • Guide conversations around common problems that IoT applications solve
  • Select the appropriate technologies in the Azure space to build IoT applications

Publisher resources

View/Submit Errata

Table of contents

  1. Preface
    1. Who Should Read This Book
    2. Navigating This Book
    3. Conventions Used in This Book
    4. Using Code Examples
    5. O’Reilly Online Learning
    6. How to Contact Us
    7. Acknowledgments
  2. 1. The IoT Landscape
    1. Off Azure
      1. IoT Devices
      2. Edge Computing
    2. Azure
      1. IoT Messaging and Management
      2. Data Processing
      3. Data Persistence
      4. Data Presentation Layer
    3. Data Consumers
    4. Monitoring, Logging, and Security
    5. Conclusion
  3. 2. Azure-Centric IoT Devices
    1. IoT Devices as the Nexus of Three Domains
      1. Hardware
      2. Software
      3. Cloud
    2. IoT Devices and Microsoft
      1. Microsoft the Software Company
      2. Microsoft as a Hardware Company
      3. Microsoft as a Cloud Company
      4. IoT on Azure: Microsoft’s Combination of Software, Hardware, and Cloud
    3. Azure Sphere
      1. Azure Sphere Hardware
      2. Azure Sphere Software
      3. Azure Sphere Cloud Services
      4. What’s It For?
      5. What Makes It Unique?
    4. Azure MXChip
      1. MXChip Hardware
      2. MXChip Software
      3. MXChip Cloud Services
      4. What’s It For?
      5. What Makes It Unique?
    5. Kinect
      1. Kinect Hardware
      2. Kinect Software
      3. Kinect Cloud Services
      4. What’s It For?
      5. What Makes It Unique?
    6. Windows for IoT
      1. Windows IoT Software
      2. Windows for IoT Hardware
      3. Windows for IoT Cloud Services
      4. What’s It For?
      5. What Makes It Unique?
    7. Azure IoT Device SDKs
      1. Supported Languages and Platforms
      2. Real-Time Operating System (RTOS)
      3. When All Else Fails, Use the APIs
    8. Summary
  4. 3. How to Try Before You Buy, IoT Edition
    1. Thinking Through Your Software
    2. The User Experience
    3. Collecting Data
      1. Data Collected from User Inputs
      2. Data Collected from the Environment
      3. Data About the Device
      4. Data About the Software on the Device
    4. Device Simulators
      1. Accelerate Development
      2. Enable Feature Development Independent of Device Development
      3. Enable Automated Testing
      4. Device Simulator Best Practices
      5. A Word About Device Simulator Services
    5. Experiment Using Virtualization
    6. Hardware Without Dev Boards
    7. Creating a Device for the Examples
      1. Setting Up the Sample Device or the Device Simulator
      2. Dependencies
      3. Cloning the Repository
      4. Explore the Code
    8. Committing to a Dev Board
    9. Summary
  5. 4. The Device Lifecycle
    1. Device Lifecycle Management
    2. Research and Design
      1. Three Phases of R&D
      2. Hardware
      3. Software
    3. Manufacturing
    4. Shipping
    5. Claiming and Provisioning
      1. Provision Devices with an IoT Hub through the Device Provisioning Service
    6. Main Sequence
      1. Communication
      2. Twinning
      3. Device Twinning with IoT Hub
      4. Updates
    7. Deprovisioning
    8. Summary
  6. 5. Device Messaging
    1. Synchronous Versus Asynchronous Messaging
    2. Real-Time Versus Store-and-Forward Messaging
    3. Bidirectional Versus Unidirectional Communication
    4. Message Formatting
      1. Properties
      2. Body
    5. Common Protocols
      1. MQ Telemetry Transport (MQTT)
      2. Advanced Message Queuing Protocol (AMQP)
      3. Hypertext Transfer Protocol (HTTP)
    6. Device-to-Cloud (D2C) Messaging
      1. Telemetry
      2. Events
      3. File Uploads
      4. Message Routing
      5. Message Enrichments
    7. Cloud-to-Device (C2D) Messaging
      1. Commands (Direct Methods)
      2. General Messages
    8. Custom Solutions
      1. Security
      2. Device Management
      3. Message Routing
      4. Scaling
      5. Integrations with Azure
    9. Summary
  7. 6. Life on the Edge
    1. Why Use Edge Computing?
      1. Better Response Time to Events on Premises
      2. Reliable Connectivity for Critical Services
      3. Access to Critical Services Without Bandwidth Constraints
      4. Aggregations and Filtering Data to Reduce Network Traffic
      5. Compute Offloading to Save Network Bandwidth
      6. Data Localization When Dealing with Data Sovereignty and Security Issues
      7. A Disconnected Cloud That Performs with No Network Connection
    2. Container Basics
    3. Azure IoT Edge
      1. IoT Edge Modules
      2. Message Brokering
      3. Data
      4. Storage
      5. Bringing the Cloud Closer with AI
      6. Extensibility (Bring Your Own Code)
    4. Creating an IoT Edge Device and Deploying a Module
      1. Install Azure IoT Edge
      2. Deploy a Module on IoT Edge
    5. Azure Arc and Kubernetes
    6. IoT Edge or Arc with Kubernetes?
    7. Setting Up Arc with MicroK8S
      1. Set Up MicroK8S
      2. Connect MicroK8S to Azure Arc
      3. Install the Device Simulator to MicroK8S Using the Azure Portal
    8. Azure Data Box Gateway
    9. Azure Stack
      1. Azure Stack Hub
      2. Azure Stack HCI
      3. Azure Stack Edge
    10. Summary
  8. 7. Scalable Data Architecture
    1. General Principles for Data Storage
      1. Partition Your Data Appropriately
      2. Storage Is Cheap, Compute Is Expensive
      3. Prefer Eventual Consistency over Strong Consistency
      4. Separate Reads and Writes
      5. Treat Writes as Idempotent
      6. Store Data in an Optimized Form for Its Intended Use
      7. Create Retention Policies
    2. Landing Your Data
      1. Azure Blob Storage
      2. Azure Data Lake
      3. General-Purpose Blob Storage Versus Data Lake
      4. Set Up Azure Blob Storage to Land Data
      5. What About File Shares or File Sharing Services?
    3. Online Transaction Processor (OLTP) Versus Online Analytics Processor (OLAP)
    4. OLTP Solutions on Azure
      1. Cosmos DB
      2. Azure Data Explorer (ADX)
      3. Azure SQL
      4. Open Source Databases
    5. OLAP Solutions on Azure
      1. Azure Synapse Analytics
      2. HDInsight
      3. Data Lakes Versus Data Warehouses
      4. Lakehouse
    6. Hybrid Transaction Analytics Processor (HTAP)
      1. OLTP as OLAP
      2. Creating an HTAP with Cosmos DB and Azure Synapse Analytics
    7. Summary
  9. 8. Data Processing Architectures
    1. Data Storage (Again)
    2. Data Processing Cycle
      1. Data Collection
      2. Data Preparation
      3. Data Input
      4. Data Processing
      5. Output
      6. Storage
    3. Data Movement
      1. Hot Path
      2. Cold Path
      3. Warm Path
      4. Which Type of Data Path Should You Use?
    4. Data Architectures
      1. Lambda Architecture
      2. Kappa Architecture
      3. Delta Architecture
      4. Which Style of Architecture Should You Use?
    5. Summary
  10. 9. Hot Path Data Processing
    1. Messaging Platforms
    2. Hot Paths
    3. Azure Stream Analytics
    4. Azure Functions
    5. Azure Logic Apps
    6. Azure Service Bus
      1. Queues and Topics
      2. Auto-Forwarders and Subscription Filtering
      3. Topic Actions
      4. Performance Considerations
      5. Create a Service Bus with a Topic and Subscriptions
      6. Create a New Route on IoT Hub
      7. Create Some Functions Save Data
    7. Summary
  11. 10. Cold Path Data Processing
    1. Azure Data Explorer
    2. Batch Processing on Azure
    3. Azure Batch
      1. Create a Batch Account
      2. Set Up the Batch Job
    4. Azure Data Factory
      1. Create a Data Factory to Move Data
      2. Create a Source Dataset from Your Storage Account
      3. Create a Data Sink for Cosmos DB
      4. Create a Data Flow to Move Data
      5. Create a Pipeline to Move Data
      6. Start the Data Flow
    5. Summary
  12. 11. The Servicing Layer
    1. Datasets
    2. Data Formats for Datasets
    3. Push-Style Delivery
      1. Cold Path Push
      2. Hot Path Push
    4. Pull-Style Delivery
    5. Azure Data Share
      1. HTTP APIs
      2. Hybrid approaches
    6. Summary
  13. 12. Data Consumers
    1. Reporting Tools
    2. Business Intelligence Tools
      1. Power BI
      2. Connecting Power BI to Data
    3. Applications
    4. External Systems Integrations
    5. Raw Data Consumers
    6. Security and Privacy
      1. Security
      2. Data Privacy
    7. Summary
  14. 13. Monitoring and Logging
    1. Setting Goals with Service Level Agreements
    2. Monitoring Your Solution
    3. Azure Monitor
      1. Data Collection
      2. Azure Data Explorer (Log Analytics)
      3. Kusto Query Language (KQL)
      4. Monitoring and Alerting
    4. Azure Application Insights
      1. Instrumentation
      2. Application Logging
      3. User Analytics
    5. Azure Security Center
      1. Security Assessments
      2. Security Monitoring
      3. Incident Response
      4. Azure Sentinel
    6. Summary
  15. 14. IoT Security
    1. Software Vulnerabilities
    2. Malware
      1. Botnets
      2. Ransomware
    3. Data Leaks
    4. DoS
    5. Insecure Communications
    6. Device Spoofing
    7. Insecure Data
    8. Lax Access Controls
    9. Physical Threats
    10. Lax Network Security
    11. DNS Threats
    12. Man-in-the-Middle
    13. Social Engineering
    14. Advanced Persistent Threats
    15. Managing Threats with Microsoft Defender for IoT
    16. Summary
  16. 15. Further Reading
    1. Devices
    2. Edge and Containers
      1. Containers and GitOps
      2. Kubernetes on the Edge
      3. Azure IoT Edge
    3. IoT Management
    4. Data Architecture
    5. Conclusion
  17. Index
  18. About the Author

Product information

  • Title: Architecting IoT Solutions on Azure
  • Author(s): Blaize Stewart
  • Release date: January 2024
  • Publisher(s): O'Reilly Media, Inc.
  • ISBN: 9781098142865