Distributed Computing with Go

Distributed Computing with Go

by V.N. Nikhil Anurag
Released February 2018
Publisher(s): Packt Publishing
ISBN: 9781787125384

Read it now on the O’Reilly learning platform with a 10-day free trial.

O’Reilly members get unlimited access to books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.

Start your free trial

Book description

A tutorial leading the aspiring Go developer to full mastery of Golang's distributed features.

About This Book

  • This book provides enough concurrency theory to give you a contextual understanding of Go concurrency
  • It gives weight to synchronous and asynchronous data streams in Golang web applications
  • It makes Goroutines and Channels completely familiar and natural to Go developers

Who This Book Is For

This book is for developers who are familiar with the Golang syntax and have a good idea of how basic Go development works. It would be advantageous if you have been through a web application product cycle, although it's not necessary.

What You Will Learn

  • Gain proficiency with concurrency and parallelism in Go
  • Learn how to test your application using Go's standard library
  • Learn industry best practices with technologies such as REST, OpenAPI, Docker, and so on
  • Design and build a distributed search engine
  • Learn strategies on how to design a system for web scale

In Detail

Distributed Computing with Go gives developers with a good idea how basic Go development works the tools to fulfill the true potential of Golang development in a world of concurrent web and cloud applications. Nikhil starts out by setting up a professional Go development environment. Then you'll learn the basic concepts and practices of Golang concurrent and parallel development.

You'll find out in the new few chapters how to balance resources and data with REST and standard web approaches while keeping concurrency in mind. Most Go applications these days will run in a data center or on the cloud, which is a condition upon which the next chapter depends. There, you'll expand your skills considerably by writing a distributed document indexing system during the next two chapters. This system has to balance a large corpus of documents with considerable analytical demands.

Another use case is the way in which a web application written in Go can be consciously redesigned to take distributed features into account. The chapter is rather interesting for Go developers who have to migrate existing Go applications to computationally and memory-intensive environments. The final chapter relates to the rather onerous task of testing parallel and distributed applications, something that is not usually taught in standard computer science curricula.

Style and approach

Distributed Computing with Go takes you through a series of carefully graded tutorials, building ever more sophisticated applications.

Table of contents

  1. Title Page
  2. Copyright and Credits
    1. Distributed Computing with Go
  3. Packt Upsell
    1. Why subscribe?
    2. PacktPub.com
  4. Contributors
    1. About the author
    2. About the reviewers
    3. Packt is searching for authors like you
  5. Preface
    1. Who this book is for
    2. What this book covers
    3. To get the most out of this book
      1. Download the example code files
      2. Download the color images
      3. Conventions used
    4. Get in touch
      1. Reviews
  6. Developer Environment for Go
    1. GOROOT
    2. GOPATH
      1. src/
      2. pkg/
      3. bin/
    3. Package management
      1. go get
      2. glide
      3. go dep
    4. Structuring a project
    5. Working with book's code
    6. Containers
      1. Docker
        1. Docker versus Virtual Machine (VM)
        2. Understanding Docker
        3. Testing Docker setup
        4. Dockerfile
          1. main.go
    7. Testing in Go
      1. variadic.go
      2. variadic_test.go
        1. Running tests in variadic_test.go
      3. addInt.go
        1. addInt_test.go
        2. Running tests in addInt_test.go
      4. nil_test.go
        1. Running tests in nil_test.go
    8. Summary
  7. Understanding Goroutines
    1. Concurrency and parallelism
      1. Concurrency
        1. Code overview
        2. Serial task execution
        3. Serial task execution with goroutines
        4. Concurrent task execution
      2. Parallelism
    2. Go's runtime scheduler
      1. Goroutine
      2. OS thread or machine
      3. Context or processor
      4. Scheduling with G, M, and P
    3. Gotchas when using goroutines
      1. Single goroutine halting the complete program
      2. Goroutines aren't predictable
    4. Summary
  8. Channels and Messages
    1. Controlling parallelism
      1. Distributed work without channels
      2. Distributed work with channels
        1. What is a channel?
        2. Solving the cashier problem with goroutines
    2. Channels and data communication
      1. Messages and events
    3. Types of channels
      1. The unbuffered channel
      2. The buffered channel
      3. The unidirectional buffer
    4. Closing channels
    5. Multiplexing channels
    6. Summary
  9. The RESTful Web
    1. HTTP and sessions
      1. A brief history of HTTP
      2. HTTP sessions
    2. The REST protocol
      1. The server and client architecture
      2. The standard data format
      3. Resources
      4. Reusing the HTTP protocol
        1. GET
        2. POST
        3. PUT and PATCH
        4. DELETE
      5. Upgradable components
    3. Fundamentals of a REST server
      1. A simple web server
      2. Designing a REST API
        1. The data format
          1. The book resource
          2. GET /api/books
          3. GET /api/books/<id>
          4. POST /api/books
          5. PUT /api/books/<id>
          6. DELETE /api/books/<id>
          7. Unsuccessful requests
        2. Design decisions
        3. The REST server for books API
          1. main.go
          2. books-handler/common.go
          3. books-handler/actions.go
          4. books-handler/handler.go
    4. How to make REST calls
      1. cURL
        1. GET
        2. DELETE
        3. PUT
        4. POST
      2. Postman
      3. net/http
    5. Summary
  10. Introducing Goophr
    1. What is Goophr?
    2. Design overview
    3. OpenAPI specification
      1. Goophr Concierge API definition
        1. Goophr Librarian API definition
    4. Project structure
    5. Summary
  11. Goophr Concierge
    1. Revisiting the API definition
    2. Document feeder – the REST API endpoint
    3. Query handler – the REST API endpoint
    4. Conventions
      1. Code conventions
      2. Diagram conventions
    5. Logical flow diagrams
      1. The doc processor
      2. The doc store
      3. The index processor
      4. The line store
      5. The consolidated flow diagram
        1. Queue workers
        2. Single stores
        3. Buffered channels
    6. The Concierge source code
      1. Running tests
      2. The Concierge server
    7. Summary
  12. Goophr Librarian
    1. The standard indexing model
      1. An example – books with an index of words
    2. The inverted indexing model
      1. An example – the inverted index for words in books
    3. Ranking
    4. Revisiting the API definition
    5. The document indexer – the REST API endpoint
    6. The query resolver – the REST API endpoint
    7. Code conventions
    8. Librarian source code
      1. main.go
      2. common/helpers.go
      3. api/index.go
      4. api/query.go
    9. Testing Librarian
      1. Testing feeder.go using /api/index
        1. Testing /api/query
    10. Summary
  13. Deploying Goophr
    1. Updating Goophr Concierge
      1. Handle multiple Librarians
      2. Aggregated search results
    2. Orchestrating with docker-compose
      1. Environment variables and API ports
      2. The file server
    3. The Goophr source code
      1. librarian/main.go
      2. concierge/main.go
      3. concierge/api/query.go
      4. simple-server/Dockerfile
      5. simple-server/main.go
      6. docker-compose.yaml
      7. .env
    4. Running Goophr with docker-compose
      1. Adding documents to Goophr
    5. Searching for keywords with Goophr
      1. Search – "apple"
      2. Search – "cake"
      3. Search – "apple", "cake"
      4. Individual logs with docker-compose
    6. Authorization on a web server
      1. secure/secure.go
      2. secure/secure_test.go
      3. Test results
    7. Summary
  14. Foundations of Web Scale Architecture
    1. Scaling a web application
      1. The single server instance
      2. Separate layers for the web and database
      3. Multiple server instances
        1. The load balancer
      4. Multi-availability zones
      5. The database
        1. SQL versus NoSQL
          1. Which type of database should we use?
        2. Database replication
          1. Master-replica replication
          2. Master-master replication
          3. Failover cluster replication
    2. Monolith versus microservices
      1. Mediator design pattern
    3. Deployment options
      1. Maintainability of multiple instances
    4. Summary
  15. Other Books You May Enjoy
    1. Leave a review - let other readers know what you think

Product information

  • Title: Distributed Computing with Go
  • Author(s): V.N. Nikhil Anurag
  • Release date: February 2018
  • Publisher(s): Packt Publishing
  • ISBN: 9781787125384