Learn Microsoft Fabric

Learn Microsoft Fabric

by Arshad Ali, Bradley Schacht
Released February 2024
Publisher(s): Packt Publishing
ISBN: 9781835082287

Read it now on the O’Reilly learning platform with a 10-day free trial.

O’Reilly members get unlimited access to books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.

Start your free trial

Book description

Harness the power of Microsoft Fabric to develop data analytics solutions for various use cases guided by step-by-step instructions

Key Features

  • Explore Microsoft Fabric and its features through real-world examples
  • Build data analytics solutions for lakehouses, data warehouses, real-time analytics, and data science
  • Monitor, manage, and administer your Fabric platform and analytics system to ensure flexibility, performance, security, and control
  • Purchase of the print or Kindle book includes a free PDF eBook

Book Description

Discover the capabilities of Microsoft Fabric, the premier unified solution designed for the AI era, seamlessly combining data integration, OneLake, transformation, visualization, universal security, and a unified business model. This book provides an overview of Microsoft Fabric, its components, and the wider analytics landscape.

In this book, you'll explore workloads such as Data Factory, Synapse Data Engineering, data science, data warehouse, real-time analytics, and Power BI. You’ll learn how to build end-to-end lakehouse and data warehouse solutions using the medallion architecture, unlock the real-time analytics, and implement machine learning and AI models. As you progress, you’ll build expertise in monitoring workloads and administering Fabric across tenants, capacities, and workspaces. The book also guides you step by step through enhancing security and governance practices in Microsoft Fabric and implementing CI/CD workflows with Azure DevOps or GitHub. Finally, you’ll discover the power of Copilot, an AI-driven assistant that accelerates your analytics journey.

By the end of this book, you’ll have unlocked the full potential of AI-driven data analytics, gaining a comprehensive understanding of the analytics landscape and mastery over the essential concepts and principles of Microsoft Fabric.

What you will learn

  • Get acquainted with the different services available in Microsoft Fabric
  • Build end-to-end data analytics solution to scale and manage high performance
  • Integrate data from different types of data sources
  • Apply transformation with Spark, Notebook, and T-SQL
  • Understand and implement real-time stream processing and data science capabilities
  • Perform end-to-end processes for building data analytics solutions in the AI era
  • Drive insights by leveraging Power BI for reporting and visualization
  • Improve productivity with AI assistance and Copilot integration

Who this book is for

This book is for data professionals, including data analysts, data engineers, data scientists, data warehouse developers, ETL developers, business analysts, AI/ML professionals, software developers, and Chief Data Officers who want to build a future-ready data analytics solution for long-term success in the AI era. For PySpark and SQL students entering the data analytics field, this book offers a broad foundation for developing the skills to build end-to-end analytics systems for various use cases. Basic knowledge of SQL and Spark is assumed.

Table of contents

  1. Learn Microsoft Fabric
  2. Contributors
  3. About the authors
  4. About the reviewer
  5. Preface
    1. Who this book is for
    2. What this book covers
    3. To get the most out of this book
    4. Download the example code files
    5. Conventions used
    6. Get in touch
    7. Share Your Thoughts
    8. Download a free PDF copy of this book
  6. Part 1: An Introduction to Microsoft Fabric
  7. Chapter 1: Overview of Microsoft Fabric and Understanding Its Different Concepts
    1. Introduction to Microsoft Fabric
    2. Reviewing the core capabilities of Microsoft Fabric
      1. Complete analytics platform
      2. Lake-centric and open
      3. Empower every business user
      4. AI powered
    3. Unified business model with universal compute capacity
    4. Summary
  8. Chapter 2: Understanding Different Workloads and Getting Started with Microsoft Fabric
    1. Getting started with Microsoft Fabric
      1. Enabling Microsoft Fabric
      2. Checking your access to Microsoft Fabric
      3. Creating your first Fabric-enabled workspace
    2. Data Factory
      1. Pipelines
      2. Activities
      3. Connections
      4. Dataflow Gen2
      5. Loading data
    3. Data engineering
      1. Lakehouse
      2. Spark Job Definition
    4. Data Warehouse
      1. Simplifying the Data Warehouse experience
      2. Open and lake-centric
      3. Combining the lakehouse and data warehouse
      4. Loading data
      5. Querying the warehouse
    5. Data Science
      1. SynapseML
      2. MLflow integration
      3. FLAML integration for automated ML (AutoML)
      4. Data Wrangler
      5. Semantic Link
    6. Real-Time Analytics
      1. Eventstreams
      2. KQL databases
      3. KQL queryset
    7. Power BI
      1. Reports
      2. Datasets
      3. Direct Lake
    8. Summary
  9. Part 2: Building End-to-End Analytics Systems
  10. Chapter 3: Building an End-to-End Analytics System – Lakehouse
    1. Technical requirements
    2. Understanding end-to-end scenarios
      1. Understanding the end-to-end architecture
      2. Understanding sample data and data models
      3. Understanding data and transformation flow
    3. Storage
    4. Ingestion
    5. Transformation
      1. Importing notebooks
      2. Creating a shortcut (for Files): Silver zone
      3. Opening notebook and executing commands (loading to the Silver zone)
      4. Incremental data load
      5. Creating a shortcut (for Tables): Gold zone
      6. Creating business aggregates for the Gold zone
    6. Analyze
      1. Power BI
      2. SQL endpoint
    7. Orchestrate data ingestion and transformation flow and schedule notebooks and pipelines
    8. Data meshes in Fabric – a primer
    9. Summary
  11. Chapter 4: Building an End-to-End Analytics System – Data Warehouse
    1. Understanding the end-to-end scenario
      1. Data and transformation flow
    2. Creating a data warehouse
      1. Creating tables in a data warehouse
    3. Loading data
      1. Loading data using the copy activity in Data Factory
      2. Loading data using T-SQL
    4. Data transformation using T-SQL
    5. Orchestrating ETL operations with Data Factory pipelines
    6. Analyzing data with Power BI
    7. Summary
  12. Chapter 5: Building an End-to-End Analytics System – Real-Time Analytics
    1. Understanding the end-to-end scenario
    2. Creating a Kusto Query Language (KQL) database
    3. Capturing and delivering data using eventstreams
    4. Analyzing data with KQL
    5. Reporting with Power BI
      1. Creating a new Power BI report
      2. Adding visualizations to the Power BI report
      3. Configure page refresh on the Power BI report
    6. Summary
  13. Chapter 6: Building an End-to-End Analytics System – Data Science
    1. Technical requirements
    2. End-to-end data science scenario
    3. Data and storage – creating a lakehouse and ingesting data using Apache Spark
      1. Importing notebooks
    4. Problem formulation/ideation (business understanding)
      1. Semantic Link
    5. Data acquisition, discovery, and preprocessing
      1. Data acquisition
      2. Data discovery
      3. Data preprocessing
      4. Data Wrangler
    6. Experimenting and modeling
      1. Training – version 1
      2. Training – version 2
      3. AutoML with FLAML
    7. Enriching and operationalizing
    8. Analyzing and getting insights
    9. Summary
  14. Part 3: Administration and Monitoring
  15. Chapter 7: Monitoring Overview and Monitoring Different Workloads
    1. Technical requirements
    2. Overview of monitoring capabilities in Fabric
    3. Monitoring Data Factory pipelines and dataflows
    4. Monitoring Spark jobs (data engineering and data science)
    5. Monitoring data warehouse activity
    6. Monitoring Real-Time Analytics activity
      1. Monitoring eventstreams
      2. Monitoring KQL databases
    7. Monitoring capacity usage with the Microsoft Fabric Capacity Metrics app
    8. Summary
  16. Chapter 8: Administering Fabric
    1. Enabling Microsoft Fabric in your tenant
    2. What are capacities?
      1. Managing Fabric capacities
    3. Managing Spark job configurations
      1. Starter pools
      2. Custom Spark pools
      3. Spark runtime
      4. High concurrency
      5. Automatically tracking machine learning experiments and models
      6. Spark properties/configuration
      7. Library management
      8. Auto-tune
      9. Spark utility (MSSparkUtils)
    4. Summary
  17. Part 4: Security and Developer Experience
  18. Chapter 9: Security and Governance Overview
    1. Securing the Microsoft Fabric platform
      1. Guest users
      2. Conditional access
    2. Securing Microsoft Fabric workspaces and items
      1. Workspace-level permissions
      2. Item-level permissions
    3. Understanding governance and compliance in Microsoft Fabric
      1. Domains
      2. Microsoft Purview
    4. Summary
  19. Chapter 10: Continuous Integration and Continuous Deployment (CI/CD)
    1. Technical requirements
    2. Understanding the end-to-end flow
    3. Connecting to a Git repo with Azure DevOps
    4. Working on a new feature or release
    5. Creating and executing a deployment pipeline
    6. Managing database code for a Fabric data warehouse
      1. Managing database code with the SQL Database Projects extension
    7. Summary
  20. Part 5: AI Assistance with Copilot Integration
  21. Chapter 11: Overview of AI Assistance and Copilot Integration
    1. Technical requirements
    2. What is Copilot in Fabric?
    3. Copilot in data engineering and data science
    4. Copilot in Data Factory
    5. Copilot in Power BI
      1. Creating reports with the Power BI Copilot
      2. Creating a narrative using Copilot
      3. Generating synonyms with Copilot
    6. Summary
  22. Index
    1. Why subscribe?
  23. Other Books You May Enjoy
    1. Packt is searching for authors like you
    2. Share Your Thoughts
    3. Download a free PDF copy of this book

Product information

  • Title: Learn Microsoft Fabric
  • Author(s): Arshad Ali, Bradley Schacht
  • Release date: February 2024
  • Publisher(s): Packt Publishing
  • ISBN: 9781835082287