Mastering the Lightning Network

Mastering the Lightning Network

by Andreas M. Antonopoulos, Olaoluwa Osuntokun, René Pickhardt
Released November 2021
Publisher(s): O'Reilly Media, Inc.
ISBN: 9781492054863

Read it now on the O’Reilly learning platform with a 10-day free trial.

O’Reilly members get unlimited access to books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.

Buy on Amazon Buy on ebooks.com
Start your free trial

Book description

The Lightning Network (LN) is a rapidly growing second-layer payment protocol that works on top of Bitcoin to provide near-instantaneous transactions between two parties. With this practical guide, authors Andreas M. Antonopoulos, Olaoluwa Osuntokun, and Rene Pickhardt explain how this advancement will enable the next level of scale for Bitcoin, increasing speed and privacy while reducing fees.

Ideal for developers, systems architects, investors, and entrepreneurs looking to gain a better understanding of LN, this book demonstrates why experts consider LN a critical solution to Bitcoin's scalability problem. You'll learn how LN has the potential to support far more transactions than today's financial networks.

This book examines:

  • How the Lightning Network addresses the challenge of blockchain scaling
  • The Basis of Lightning Technology (BOLT) standards documents
  • The five layers of the Lightning Network Protocol Suite
  • LN basics, including wallets, nodes, and how to operate one
  • Lightning payment channels, onion routing, and gossip protocol
  • Finding paths across payment channels to transport Bitcoin off-chain from sender to recipient

Publisher resources

View/Submit Errata

Table of contents

  1. Preface
    1. Intended Audience
    2. Conventions Used in This Book
    3. Code Examples
    4. Using Code Examples
    5. References to Companies and Products
    6. Addresses and Transactions in This Book
    7. O’Reilly Online Learning
    8. How to Contact Us
      1. Contacting Andreas
      2. Contacting René
      3. Contacting Olaoluwa Osuntokun
    9. Acknowledgments by Andreas
    10. Acknowledgments by René
    11. Acknowledgments by Olaoluwa Osuntokun
    12. Contributions
    13. Sources
  2. I. Understanding the Lightning Network
  3. 1. Introduction
    1. Lightning Network Basic Concepts
    2. Trust in Decentralized Networks
    3. Fairness Without Central Authority
      1. Trusted Protocols Without Intermediaries
      2. A Fairness Protocol in Action
      3. Security Primitives as Building Blocks
      4. Example of the Fairness Protocol
    4. Motivation for the Lightning Network
      1. Scaling Blockchains
    5. The Lightning Network’s Defining Features
    6. Lightning Network Use Cases, Users, and Their Stories
    7. Conclusion
  4. 2. Getting Started
    1. Alice’s First Lightning Wallet
    2. Lightning Nodes
    3. Lightning Explorers
    4. Lightning Wallets
      1. Testnet Bitcoin
    5. Balancing Complexity and Control
    6. Downloading and Installing a Lightning Wallet
    7. Creating a New Wallet
      1. Responsibility with Key Custody
      2. Mnemonic Words
      3. Storing the Mnemonic Safely
    8. Loading Bitcoin onto the Wallet
      1. Acquiring Bitcoin
      2. Receiving Bitcoin
    9. From Bitcoin to Lightning Network
      1. Lightning Network Channels
      2. Opening a Lightning Channel
    10. Buying a Cup of Coffee Using the Lightning Network
      1. Bob’s Cafe
      2. A Lightning Invoice
    11. Conclusion
  5. 3. How the Lightning Network Works
    1. What Is a Payment Channel?
    2. Payment Channel Basics
    3. Routing Payments Across Channels
    4. Payment Channels
      1. Multisignature Address
      2. Funding Transaction
      3. Commitment Transaction
      4. Cheating with Prior State
      5. Announcing the Channel
      6. Closing the Channel
    5. Invoices
      1. Payment Hash and Preimage
      2. Additional Metadata
    6. Delivering the Payment
      1. The Peer-to-Peer Gossip Protocol
      2. Pathfinding and Routing
    7. Source-Based Pathfinding
      1. Onion Routing
      2. Payment Forwarding Algorithm
    8. Peer-to-Peer Communication Encryption
    9. Thoughts About Trust
    10. Comparison with Bitcoin
      1. Addresses Versus Invoices, Transactions Versus Payments
      2. Selecting Outputs Versus Finding a Path
      3. Change Outputs on Bitcoin Versus No Change on Lightning
      4. Mining Fees Versus Routing Fees
      5. Varying Fees Depending on Traffic Versus Announced Fees
      6. Public Bitcoin Transactions Versus Private Lightning Payments
      7. Waiting for Confirmations Versus Instant Settlement
      8. Sending Arbitrary Amounts Versus Capacity Restrictions
      9. Incentives for Large Value Payment Versus Small Value Payments
      10. Using the Blockchain as a Ledger Versus as a Court System
      11. Offline Versus Online, Asynchronous Versus Synchronous
      12. Satoshis Versus Millisatoshis
    11. Commonality of Bitcoin and Lightning
      1. Monetary Unit
      2. Irreversibility and Finality of Payments
      3. Trust and Counterparty Risk
      4. Permissionless Operation
      5. Open Source and Open System
    12. Conclusion
  6. 4. Lightning Node Software
    1. Lightning Development Environment
      1. Using the Command Line
      2. Downloading the Book Repository
    2. Docker Containers
    3. Bitcoin Core and Regtest
      1. Building the Bitcoin Core Container
    4. The c-lightning Lightning Node Project
      1. Building c-lightning as a Docker Container
      2. Setting Up a Docker Network
      3. Running the bitcoind and c-lightning Containers
      4. Installing c-lightning from Source Code
      5. Installing Prerequisite Libraries and Packages
      6. Copying the c-lightning Source Code
      7. Compiling the c-lightning Source Code
    5. The Lightning Network Daemon Node Project
      1. The LND Docker Container
      2. Running the bitcoind and LND Containers
      3. Installing LND from Source Code
      4. Copying the LND Source Code
      5. Compiling the LND Source Code
    6. The Eclair Lightning Node Project
      1. The Eclair Docker Container
      2. Running the bitcoind and Eclair Containers
      3. Installing Eclair from Source Code
      4. Copying the Eclair Source Code
      5. Compiling the Eclair Source Code
    7. Building a Complete Network of Diverse Lightning Nodes
      1. Using docker-compose to Orchestrate Docker Containers
      2. docker-compose Configuration
      3. Starting the Example Lightning Network
      4. Opening Channels and Routing a Payment
    8. Conclusion
  7. 5. Operating a Lightning Network Node
    1. Choosing Your Platform
      1. Why Is Reliability Important for Running a Lightning Node?
      2. Types of Hardware Lightning Nodes
      3. Running in the “Cloud”
      4. Running a Node at Home
      5. What Hardware Is Required to Run a Lightning Node?
      6. Switching Server Configuration in the Cloud
    2. Using an Installer or Helper
      1. RaspiBlitz
      2. Mynode
      3. Umbrel
      4. BTCPay Server
      5. Bitcoin Node or Lightweight Lightning
      6. Operating System Choice
    3. Choose Your Lightning Node Implementation
    4. Installing a Bitcoin or Lightning Node
      1. Background Services
      2. Process Isolation
      3. Node Startup
      4. Node Configuration
      5. Network Configuration
    5. Security of Your Node
      1. Operating System Security
      2. Node Access
    6. Node and Channel Backups
      1. Hot Wallet Risk
      2. Sweeping Funds
    7. Lightning Node Uptime and Availability
      1. Tolerate Faults and Automate
      2. Monitoring Node Availability
      3. Watchtowers
    8. Channel Management
      1. Opening Outbound Channels
      2. Getting Inbound Liquidity
      3. Closing Channels
      4. Rebalancing Channels
    9. Routing Fees
    10. Node Management
      1. Ride The Lightning
      2. lndmon
      3. ThunderHub
    11. Conclusion
  8. II. The Lightning Network in Detail
  9. 6. Lightning Network Architecture
    1. The Lightning Network Protocol Suite
    2. Lightning in Detail
  10. 7. Payment Channels
    1. A Different Way of Using the Bitcoin System
    2. Bitcoin Ownership and Control
      1. Diversity of (Independent) Ownership and Multisig
      2. Joint Ownership Without Independent Control
      3. Preventing “Locked” and Un-Spendable Bitcoin
    3. Constructing a Payment Channel
      1. Node Private and Public Keys
      2. Node Network Address
      3. Node Identifiers
      4. Connecting Nodes as Direct Peers
    4. Constructing the Channel
      1. Peer Protocol for Channel Management
      2. Channel Establishment Message Flow
      3. The Funding Transaction
      4. Generating a Multisignature Address
      5. Constructing the Funding Transaction
      6. Holding Signed Transactions Without Broadcasting
      7. Refund Before Funding
      8. Constructing the Presigned Refund Transaction
      9. Chaining Transactions Without Broadcasting
      10. Solving Malleability (Segregated Witness)
      11. Broadcasting the Funding Transaction
    5. Sending Payments Across the Channel
      1. Splitting the Balance
      2. Competing Commitments
      3. Cheating with Old Commitment Transactions
      4. Revoking Old Commitment Transactions
      5. Asymmetric Commitment Transactions
      6. Delayed (Timelocked) Spending to_self
      7. Revocation Keys
    6. The Commitment Transaction
    7. Advancing the Channel State
      1. The commitment_signed Message
      2. The revoke_and_ack Message
      3. Revoking and Recommitting
      4. Cheating and Penalty in Practice
      5. The Channel Reserve: Ensuring Skin in the Game
    8. Closing the Channel (Cooperative Close)
      1. The Shutdown Message
      2. The closing_signed Message
      3. The Cooperative Close Transaction
    9. Conclusion
  11. 8. Routing on a Network of Payment Channels
    1. Routing a Payment
    2. Routing Versus Pathfinding
    3. Creating a Network of Payment Channels
    4. A Physical Example of “Routing”
    5. Fairness Protocol
      1. Implementing Atomic Trustless Multihop Payments
    6. Revisiting the Tipping Example
      1. On-Chain Versus Off-Chain Settlement of HTLCs
    7. Hash Time-Locked Contracts
      1. HTLCs in Bitcoin Script
      2. Payment Preimage and Hash Verification
      3. Extending HTLCs from Alice to Dina
      4. Back-Propagating the Secret
      5. Signature Binding: Preventing Theft of HTLCs
      6. Hash Optimization
      7. HTLC Cooperative and Timeout Failure
      8. Decrementing Timelocks
    8. Conclusion
  12. 9. Channel Operation and Payment Forwarding
    1. Local (Single Channel) Versus Routed (Multiple Channels)
    2. Forwarding Payments and Updating Commitments with HTLCs
      1. HTLC and Commitment Message Flow
    3. Forwarding Payments with HTLCs
      1. Adding an HTLC
      2. The update_add_HTLC Message
      3. HTLC in Commitment Transactions
      4. New Commitment with HTLC Output
      5. Alice Commits
      6. Bob Acknowledges New Commitment and Revokes Old One
      7. Bob Commits
    4. Multiple HTLCs
    5. HTLC Fulfillment
      1. HTLC Propagation
      2. Dina Fulfills the HTLC with Chan
      3. Bob Settles the HTLC with Alice
    6. Removing an HTLC Due to Error or Expiry
    7. Making a Local Payment
    8. Conclusion
  13. 10. Onion Routing
    1. A Physical Example Illustrating Onion Routing
      1. Selecting a Path
      2. Building the Layers
      3. Peeling the Layers
    2. Introduction to Onion Routing of HTLCs
      1. Alice Selects the Path
      2. Alice Constructs the Payloads
      3. Key Generation
    3. Wrapping the Onion Layers
      1. Fixed-Length Onions
      2. Wrapping the Onion (Outlined)
      3. Wrapping Dina’s Hop Payload
      4. Wrapping Chan’s Hop Payload
      5. Wrapping Bob’s Hop Payload
      6. The Final Onion Packet
    4. Sending the Onion
      1. The update_add_htlc Message
      2. Alice Sends the Onion to Bob
      3. Bob Checks the Onion
      4. Bob Generates Filler
      5. Bob De-Obfuscates His Hop Payload
      6. Bob Extracts the Outer HMAC for the Next Hop
      7. Bob Removes His Payload and Left-Shifts the Onion
      8. Bob Constructs the New Onion Packet
      9. Bob Verifies the HTLC Details
      10. Bob Sends the update_add_htlc to Chan
      11. Chan Forwards the Onion
      12. Dina Receives the Final Payload
    5. Returning Errors
      1. Failure Messages
    6. Keysend Spontaneous Payments
      1. Custom Onion TLV Records
      2. Sending and Receiving Keysend Payments
      3. Keysend and Custom Records in Lightning Applications
    7. Conclusion
  14. 11. Gossip and the Channel Graph
    1. Peer Discovery
      1. P2P Bootstrapping
      2. DNS Bootstrapping
      3. SRV Query Options
    2. The Channel Graph
      1. A Directed Graph
    3. Gossip Protocol Messages
      1. The node_announcement Message
      2. The channel_announcement Message
      3. The channel_update Message
    4. Ongoing Channel Graph Maintenance
    5. Conclusion
  15. 12. Pathfinding and Payment Delivery
    1. Pathfinding in the Lightning Protocol Suite
      1. Where Is the BOLT?
    2. Pathfinding: What Problem Are We Solving?
      1. Selecting the Best Path
      2. Pathfinding in Math and Computer Science
      3. Capacity, Balance, Liquidity
      4. Uncertainty of Balances
      5. Pathfinding Complexity
      6. Keeping It Simple
    3. Pathfinding and Payment Delivery Process
    4. Channel Graph Construction
      1. Liquidity Uncertainty and Probability
      2. Fees and Other Channel Metrics
    5. Finding Candidate Paths
    6. Payment Delivery (Trial-and-Error Loop)
      1. First Attempt (Path #1)
      2. Second Attempt (Path #4)
    7. Multipart Payments
      1. Using MPP
      2. Trial and Error over Multiple “Rounds”
    8. Conclusion
  16. 13. Wire Protocol: Framing and Extensibility
    1. Messaging Layer in the Lightning Protocol Suite
    2. Wire Framing
      1. High-Level Wire Framing
      2. Type Encoding
    3. Type-Length-Value Message Extensions
      1. The Protocol Buffers Message Format
      2. Forward and Backward Compatibility
    4. Type-Length-Value Format
      1. BigSize Integer Encoding
      2. TLV Encoding Constraints
      3. TLV Canonical Encoding
    5. Feature Bits and Protocol Extensibility
      1. Feature Bits as an Upgrade Discoverability Mechanism
      2. TLV for Forward and Backward Compatibility
      3. A Taxonomy of Upgrade Mechanisms
      4. Channel Construction-Level Updates
    6. Conclusion
  17. 14. Lightning’s Encrypted Message Transport
    1. Encrypted Transport in the Lightning Protocol Suite
    2. Introduction
    3. The Channel Graph as Decentralized Public Key Infrastructure
    4. Why Not TLS?
    5. The Noise Protocol Framework
    6. Lightning Encrypted Transport in Detail
      1. Noise_XK: Lightning Network’s Noise Handshake
      2. Handshake Notation and Protocol Flow
      3. High-Level Overview
      4. Handshake in Three Acts
    7. Conclusion
  18. 15. Lightning Payment Requests
    1. Invoices in the Lightning Protocol Suite
    2. Introduction
    3. Lightning Payment Requests Versus Bitcoin Addresses
    4. BOLT #11: Lightning Payment Request Serialization and Interpretation
      1. Payment Request Encoding in Practice
      2. The Human-Readable Prefix
      3. bech32 and the Data Segment
    5. Conclusion
  19. 16. Security and Privacy of the Lightning Network
    1. Why Is Privacy Important?
    2. Definitions of Privacy
    3. Process to Evaluate Privacy
    4. Anonymity Set
    5. Differences Between the Lightning Network and Bitcoin in Terms of Privacy
    6. Attacks on Lightning
      1. Observing Payment Amounts
      2. Linking Senders and Receivers
      3. Revealing Channel Balances (Probing)
      4. Denial of Service
      5. Commitment Jamming
      6. Channel Liquidity Lockup
    7. Cross-Layer De-Anonymization
      1. On-Chain Bitcoin Entity Clustering
      2. Off-Chain Lightning Node Clustering
      3. Cross-Layer Linking: Lightning Nodes and Bitcoin Entities
    8. Lightning Graph
      1. How Does the Lightning Graph Look in Reality?
    9. Centralization in the Lightning Network
    10. Economic Incentives and Graph Structure
    11. Practical Advice for Users to Protect Their Privacy
    12. Unannounced Channels
    13. Routing Considerations
      1. Accepting Channels
    14. Conclusion
    15. References and Further Reading
  20. 17. Conclusion
    1. Decentralized and Asynchronous Innovation
      1. Bitcoin Protocol and Bitcoin Script Innovation
      2. Lightning Protocol Innovation
      3. TLV Extensibility
      4. Payment Channel Construction
      5. Opt-In End-to-End Features
    2. Lightning Applications (LApps)
    3. Ready, Set, Go!
  21. A. Bitcoin Fundamentals Review
    1. Keys and Digital Signatures
      1. Private and Public Keys
      2. Hashes
      3. Digital Signatures
      4. Signature Types
    2. Bitcoin Transactions
      1. Inputs and Outputs
      2. Transaction Chains
      3. TxID: Transaction Identifiers
      4. Outpoints: Output Identifiers
    3. Bitcoin Script
      1. Running Bitcoin Script
      2. Locking and Unlocking Scripts
      3. Locking to a Public Key (Signature)
      4. Locking to a Hash (Secret)
      5. Multisignature Scripts
      6. Timelock Scripts
      7. Scripts with Multiple Conditions
      8. Using Flow Control in Scripts
  22. B. Docker Basic Installation and Use
    1. Installing Docker
    2. Basic Docker Commands
      1. Building a Container
      2. Running a Container
      3. Executing a Command in a Container
      4. Stopping and Starting a Container
      5. Deleting a Container by Name
      6. Listing Running Containers
      7. Listing Docker Images
    3. Conclusion
  23. C. Wire Protocol Messages
    1. Message Types
    2. Message Structure
      1. Connection Establishment Messages
      2. Error Communication Messages
      3. Connection Liveness
      4. Channel Funding
      5. Channel Closing
      6. Channel Operation
      7. Channel Announcement
      8. Channel Graph Syncing
  24. D. Sources and License Notices
    1. Sources
    2. BTCPay Server
    3. Lamassu Industries AG
  25. Glossary
  26. Index
  27. About the Authors

Product information

  • Title: Mastering the Lightning Network
  • Author(s): Andreas M. Antonopoulos, Olaoluwa Osuntokun, René Pickhardt
  • Release date: November 2021
  • Publisher(s): O'Reilly Media, Inc.
  • ISBN: 9781492054863