Peer-to-Peer

Peer-to-Peer

by Andy Oram
Released February 2001
Publisher(s): O'Reilly Media, Inc.
ISBN: 9781491943212

Read it now on the O’Reilly learning platform with a 10-day free trial.

O’Reilly members get unlimited access to books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.

Start your free trial

Book description

The term "peer-to-peer" has come to be applied to networks that expect end users to contribute their own files, computing time, or other resources to some shared project. Even more interesting than the systems' technical underpinnings are their socially disruptive potential: in various ways they return content, choice, and control to ordinary users.

While this book is mostly about the technical promise of peer-to-peer, we also talk about its exciting social promise. Communities have been forming on the Internet for a long time, but they have been limited by the flat interactive qualities of email and Network newsgroups. People can exchange recommendations and ideas over these media, but have great difficulty commenting on each other's postings, structuring information, performing searches, or creating summaries. If tools provided ways to organize information intelligently, and if each person could serve up his or her own data and retrieve others' data, the possibilities for collaboration would take off. Peer-to-peer technologies along with metadata could enhance almost any group of people who share an interest--technical, cultural, political, medical, you name it.

This book presents the goals that drive the developers of the best-known peer-to-peer systems, the problems they've faced, and the technical solutions they've found. Learn here the essentials of peer-to-peer from leaders of the field:

  • Nelson Minar and Marc Hedlund of target="new">Popular Power, on a history of peer-to-peer
  • Clay Shirky of acceleratorgroup, on where peer-to-peer is likely to be headed
  • Tim O'Reilly of O'Reilly & Associates, on redefining the public's perceptions
  • Dan Bricklin, cocreator of Visicalc, on harvesting information from end-users
  • David Anderson of SETI@home, on how SETI@Home created the world's largest computer
  • Jeremie Miller of Jabber, on the Internet as a collection of conversations
  • Gene Kan of Gnutella and GoneSilent.com, on lessons from Gnutella for peer-to-peer technologies
  • Adam Langley of Freenet, on Freenet's present and upcoming architecture
  • Alan Brown of Red Rover, on a deliberately low-tech content distribution system
  • Marc Waldman, Lorrie Cranor, and Avi Rubin of AT&T Labs, on the Publius project and trust in distributed systems
  • Roger Dingledine, Michael J. Freedman, andDavid Molnar of Free Haven, on resource allocation and accountability in distributed systems
  • Rael Dornfest of O'Reilly Network and Dan Brickley of ILRT/RDF Web, on metadata
  • Theodore Hong of Freenet, on performance
  • Richard Lethin of Reputation Technologies, on how reputation can be built online
  • Jon Udell ofBYTE and Nimisha Asthagiri andWalter Tuvell of Groove Networks, on security
  • Brandon Wiley of Freenet, on gateways between peer-to-peer systems


You'll find information on the latest and greatest systems as well as upcoming efforts in this book.

Publisher resources

View/Submit Errata

Table of contents

  1. Special Upgrade Offer
  2. Preface
    1. Some context and a definition
    2. How this book came into being
    3. Contents of this book
    4. Peer-to-peer web site
    5. We’d like to hear from you
  3. I. Context and Overview
    1. 1. A Network of Peers: Peer-to-Peer Models Through the History of the Internet
      1. A revisionist history of peer-to-peer (1969-1995)
        1. Usenet
        2. DNS
      2. The network model of the Internet explosion (1995-1999)
        1. The switch to client/server
        2. The breakdown of cooperation
          1. Spam: Uncooperative people
          2. The TCP rate equation: Cooperative protocols
        3. Firewalls, dynamic IP, NAT: The end of the open network
        4. Asymmetric bandwidth
      3. Observations on the current crop of peer-to-peer applications (2000)
        1. Authoring is not the same as publishing
        2. Decentralization
        3. Abusing port 80
      4. Peer-to-peer prescriptions (2001-?)
        1. Technical solutions: Return to the old Internet
        2. Social solutions: Engineer polite behavior
      5. Conclusions
    2. 2. Listening to Napster
      1. Resource-centric addressing for unstable environments
        1. Peer-to-peer is as peer-to-peer does
        2. The veil is pierced
        3. Real solutions to real problems
        4. Who’s in and who’s out?
        5. Peer-to-peer is a horseless carriage
      2. Follow the users
        1. Users reward simplicity
        2. Listen to Napster
          1. It’s the applications, stupid
          2. Decentralization is a tool, not a goal
      3. Where’s the content?
        1. PCs are the dark matter of the Internet
        2. Promiscuous computers
      4. Nothing succeeds like address, or, DNS isn’t the only game in town
        1. An explosion of protocols
      5. An economic rather than legal challenge
        1. All you can eat
        2. Yesterday’s technology at tomorrow’s prices, two days late
        3. 30 million Britney fans does not a revolution make
      6. Peer-to-peer architecture and second-class status
        1. Users as consumers, users as providers
        2. New winners and losers
    3. 3. Remaking the Peer-to-Peer Meme
      1. From business models to meme maps
        1. A success story: From free software to open source
        2. The current peer-to-peer meme map
        3. The new peer-to-peer meme map
          1. File sharing: Napster and successors
          2. Mixing centralization and decentralization: Usenet, email, and IP routing
          3. Maximizing use of far-flung resources: Distributed computation
          4. Immediate information sharing: The new instant messaging services
          5. The writable Web
          6. Web services and content syndication
          7. Peer-to-peer and devices
        4. Strategic positioning and core competencies
    4. 4. The Cornucopia of the Commons
      1. Ways to fill shared databases
        1. CDDB: A case study in how to get a manually created database
        2. Napster: Harnessing the power of personal selfishness
        3. The commons
  4. II. Projects
    1. 5. SETI@home
      1. Radio SETI
      2. How SETI@home works
      3. Trials and tribulations
      4. Human factors
      5. The world’s most powerful computer
      6. The peer-to-peer paradigm
    2. 6. Jabber: Conversational Technologies
      1. Conversations and peers
      2. Evolving toward the ideal
      3. Jabber is created
        1. The centrality of XML
        2. Pieces of the infrastructure
          1. Identity
          2. Presence
          3. Roster
        3. Architecture
          1. Protocols
          2. Browsing
          3. Conversation management
      4. Conclusion
    3. 7. Mixmaster Remailers
      1. A simple example of remailers
      2. Onion routing
      3. How Type 2 remailers differ from Type 1 remailers
      4. General discussion
    4. 8. Gnutella
      1. Gnutella in a gnutshell
      2. A brief history
        1. Gnutella’s first breath
        2. Open source to the rescue
      3. What makes Gnutella different?
        1. Gnutella works like the real world
          1. A Gnutella cocktail party
          2. A client/server cocktail party
        2. Client/server means control, and control means responsibility
        3. The client is the server is the network
        4. Distributed intelligence
        5. Different from Freenet
      4. Gnutella’s communication system
        1. Message-based, application-level routing
        2. TCP broadcast
        3. Message broadcasting
        4. Dynamic routing
        5. Lossy transmission over reliable TCP
      5. Organizing Gnutella
        1. Placing nodes on the network
      6. Gnutella’s analogues
        1. The Gnutella horizon
        2. Cellular telephony and the Gnutella network
        3. Ethernet
        4. Cultivating the Gnutella network
      7. Gnutella’s traffic problems
        1. Host caches
        2. Returning the network to its natural state
        3. Private Gnutella networks
        4. Reducing broadcasts makes a significant impact
      8. The policy debates
        1. Napster wars
        2. Anonymity and peer-to-peer
          1. Gnutella pseudoanonymity
          2. Downloads, now in the privacy of your own direct connection
          3. Anonymous Gnutella chat
        3. Next-generation peer-to-peer file-sharing technologies
      9. Gnutella’s effects
    5. 9. Freenet
      1. Requests
        1. Detail of requests
        2. The data store
      2. Keys
        1. Key types
          1. Content Hash Keys (CHKs)
          2. Keyword Signed Keys (KSKs)
          3. Signature Verification Keys (SVKs)
        2. Keys and redirects
      3. Conclusions
    6. 10. Red Rover
      1. Architecture
        1. The hub
        2. The clients
        3. The subscribers
      2. Client life cycle
      3. Putting low-tech “weaknesses” into perspective
      4. Acknowledgments
    7. 11. Publius
      1. Why censorship-resistant anonymous publishing?
        1. Publius and other systems in this book
      2. System architecture
      3. Cryptography fundamentals
        1. Encryption and decryption
        2. Secret sharing
        3. Hash functions
      4. Publius operations
        1. Publish operation
        2. Retrieve operation
        3. Update operation
        4. Delete operation
      5. Publius implementation
        1. User interface
          1. Server software
          2. Client software
      6. Publius MIME type
      7. Publius in a nutshell
    8. 12. Free Haven
      1. Privacy in data-sharing systems
        1. Reliability with anonymity
      2. Anonymity for anonymous storage
        1. Partial anonymity
      3. The design of Free Haven
        1. Elements of the system
        2. Storage
        3. Publication
        4. Retrieval
        5. Share expiration
        6. Document revocation
        7. Trading
        8. Receipts
        9. Accountability and the buddy system
        10. Communications channel
        11. Reputation system
        12. Introducers
        13. Implementation status
      4. Attacks on Free Haven
        1. Attacks on documents or the servnet
        2. Attacks on the reputation system
        3. Attacks on anonymity
      5. An analysis of anonymity
      6. Future work
      7. Conclusion
      8. Acknowledgments
  5. III. Technical Topics
    1. 13. Metadata
      1. Data about data
      2. Metadata lessons from the Web
        1. Resource description
        2. Searching
      3. Resources and relationships: A historical overview
        1. Foundations of resource description: Unique identifiers
        2. A contrasting evolution: MP3 and the metadata marketplace
      4. Conclusion
    2. 14. Performance
      1. A note on terminology
      2. Why performance matters
      3. Bandwidth barriers
      4. It’s a small, small world
        1. An excursion into graph theory
        2. The small-world model
      5. Case study 1: Freenet
        1. Initial experiments
        2. Simulating growth
        3. Simulating fault tolerance
        4. Link distribution in Freenet
        5. The impact of free riding
        6. Scalability
      6. Case study 2: Gnutella
        1. Initial experiments
        2. Fault tolerance and link distribution in Gnutella
        3. The impact of free riding
        4. Scalability
      7. Conclusions
      8. Acknowledgments
    3. 15. Trust
      1. Trust in real life, and its lessons for computer networks
      2. Trusting downloaded software
        1. Message digest functions
        2. Digital signatures
        3. Digital certificates
        4. Signature verification
        5. Open source software
        6. Sandboxing and wrappers
      3. Trust in censorship-resistant publishing systems
        1. Publius in a nutshell
        2. Risks involved in web server logging
        3. Anonymizing proxies
        4. Censorship in Publius
          1. Using the Update mechanism to censor
        5. Publius proxy volunteers
      4. Third-party trust issues in Publius
        1. Other anonymity tools
          1. SSL
          2. Mix networks
          3. Crowds
        2. Denial of service attacks
          1. Quota systems
          2. CPU-based payment schemes
          3. Anonymous e-cash payment schemes
        3. Legal and physical attacks
      5. Trust in other systems
        1. Mojo Nation and Free Haven
        2. The Eternity Service
          1. Eternity Usenet
        3. File-sharing systems
          1. Napster
          2. Gnutella
          3. Freenet
        4. Content certification
      6. Trust and search engines
        1. Distributed search engines
        2. Deniability
      7. Conclusions
    4. 16. Accountability
      1. The difficulty of accountability
        1. Special problems posed by peer-to-peer systems
        2. Peer-to-peer models and their impacts on accountability
        3. Purposes of micropayments and reputation systems
        4. Junk mail as a resource allocation problem
        5. Pseudonymity and its consequences
          1. Problems with pseudospoofing and possible defenses
          2. Reputation for sale—SOLD!
      2. Common methods for dealing with flooding and DoS attacks
        1. Caching and mirroring
        2. Active caching and mirroring
      3. Micropayment schemes
        1. Varieties of micropayments or digital cash
        2. Nonfungible micropayments
          1. Extended types of nonfungible micropayments
          2. Nonparallelizable work functions
        3. Fungible micropayments
          1. Freeloading
          2. Fungible payments for accountability
          3. Micropayment digital cash schemes
          4. Making money off others’ work
          5. Anonymous macropayment digital cash schemes
        4. The use and effectiveness of micropayments in peer-to-peer systems
          1. Identity-based payment policies
          2. General considerations in an economic analysis of micropayment design
          3. Moderating security levels: An accountability slider
      4. Reputations
        1. Early reputation systems online
          1. Codifying reputation on a wide scale: The PGP web of trust
          2. Who will moderate the moderators: Slashdot
          3. Reputations worth real money: eBay
          4. A reputation system that resists pseudospoofing: Advogato
          5. System successes and failures
        2. Scoring systems
          1. Attacks and adversaries
          2. Aspects of a scoring system
          3. Collecting ratings
          4. Bootstrapping
          5. Personalizing reputation searches
          6. Scoring algorithms
          7. Privacy and information leaks
        3. Decentralizing the scoring system
          1. Multiple trusted parties
          2. True decentralization
      5. A case study: Accountability in Free Haven
        1. Micropayments
          1. The difficulty of distributed systems: How to exchange micropayments among peers
          2. Micropayments in the Free Haven context
        2. Reputation systems
        3. Other considerations from the case study
      6. Conclusion
      7. Acknowledgments
    5. 17. Reputation
      1. Examples of using the Reputation Server
      2. Reputation domains, entities, and multidimensional reputations
      3. Identity as an element of reputation
      4. Interface to the marketplace
      5. Scoring system
      6. Reputation metrics
      7. Credibility
      8. Interdomain sharing
      9. Bootstrapping
      10. Long-term vision
      11. Central Reputation Server versus distributed Reputation Servers
      12. Summary
    6. 18. Security
      1. Groove versus email
      2. Why secure email is a failure
      3. The solution: A Groove shared space
      4. Security characteristics of a shared space
      5. Mutually-trusting shared spaces
        1. Anatomy of a mutually-trusting shared space
        2. The key to mutual trust
      6. Mutually-suspicious shared spaces
        1. Message fanout
        2. Fetching lost messages
      7. Shared space formation and trusted authentication
      8. Inviting people into shared spaces
      9. The New-Member-Added delta message
      10. Key versioning and key dependencies
      11. Central control and local autonomy
      12. Practical security for real-world collaboration
      13. Taxonomy of Groove keys
    7. 19. Interoperability Through Gateways
      1. Why unification?
        1. Why not an ÜberNetwork?
        2. Why not an ÜberClient?
        3. Why not just use XML?
      2. One network with a thousand faces
      3. Well-known networks and their roles
        1. Freenet
        2. Gnutella
        3. Mojo Nation
        4. Free Haven and Publius
      4. Problems creating gateways
        1. Problems with inserts
        2. Problems with requests
          1. Freenet
          2. Gnutella
          3. Mojo Nation
          4. Publius
          5. Free Haven
      5. Gateway implementation
      6. Existing projects
      7. Conclusion
      8. Acknowledgments
  6. 20. Afterword
    1. Precedents and parries
    2. Who gets to innovate?
    3. A clean sweep?
  7. A. Directory of Peer-to-Peer Projects
  8. B. Contributors
  9. Index
  10. Special Upgrade Offer
  11. Copyright

Product information

  • Title: Peer-to-Peer
  • Author(s): Andy Oram
  • Release date: February 2001
  • Publisher(s): O'Reilly Media, Inc.
  • ISBN: 9781491943212