Running Consul at scale: Service discovery in the cloud
Five questions for Darron Froese: Thoughts on service discovery, Consul, and the future of IT infrastructure.
Person in fog (source: Unsplash via Pixabay)
I recently sat down with Darron Froese, site reliability engineer at Datadog, to discuss service discovery and what he hopes to see on the horizon for IT infrastructure. Here are some highlights from our conversation.
What is service discovery?
My favorite definition of service discovery is from Jeff Linday’s blog:
Learn faster. Dig deeper. See farther.
Service discovery tools manage how processes and services in a cluster can find and talk to one another. It involves a directory of services, registering services in that directory, and then being able to look up and connect to services in that directory.
Why is service discovery so important?
Service discovery is so important now because we’re moving web sites and web applications beyond what a single computer or virtual machine can handle.
As sites scale up, become more popular, and need more resources, you need tools and a mechanism to handle the registering and locating of services that your application needs to function.
It’s also becoming more important in cloud and container-based environments, because a static mapping of service to IP address and port is no longer possible with those systems.
Why is Consul such a popular tool for this?
Consul is a popular tool for this because:
1.It’s opinionated and has sensible defaults—you don’t have to build absolutely everything from scratch.
2.It has very modest runtime requirements.
3.It’s backed by Hashicorp, who builds lots of very handy tools and has gone above and beyond to help us—even as a non-paying customer.
4.It’s awesome. Seriously.
What do you see as the future of IT infrastructure, say in five years?
I’m hoping that in five years we have reliable infrastructure that abstracts more of the underlying OS from being necessary to care about.
I want to launch my app and I want my app to connect to services. I don’t want to have to care what it’s running on. I don’t want to have to care about arcane incantations to make it run faster. I just want to push my app code into place and have the “system” help to scale it out.
Computers make faster decisions than me; I want them to be able to make some of the decisions that I am having to make in real time and with minimal human intervention.
You’re speaking at the Velocity Conference in New York this September. What presentations are you looking forward to attending while there?
I’m looking forward to seeing “Infrastructure as code might be literally impossible” by Joe Damato. He’s a great speaker, and I really enjoyed his last talk at Monitorama.
I also want to see “Mitigating sprawl with microservices and containerization” by Jane Arc and Susan Fowler. That should be an interesting talk.