What do you use for service discovery?
1 comments
Can someone give a really brief description of what service discovery is, and when/why one would need it?
What's wrong with simply having a config file that tells your code the URL for each service it depends on (assuming my understanding of what discovery is is valid)?
What's wrong with simply having a config file that tells your code the URL for each service it depends on (assuming my understanding of what discovery is is valid)?
Once you start thinking about immutable infrastructure and auto-scaling, service discovery becomes very important.
Let's say you have a load balancer with 4 web servers behind it, each with its own IP.
If you hit a traffic spike, you'd like your system to automatically provision another web server and add it to the load balancer with no manual intervention.
With a Service Discovery server, your load balancer could be polling the server for a list of nodes it needs to add or remove from its pool. It could also send a signal to the service discovery server that will indicate that it's dealing with a higher traffic threshold. This signal can then be captured by another Provisioning service (also polling Service Discovery) which then triggers the building of a new web server. The web server could then signal to the load balancer to add it to the pool.
Without a system like this, a person has to go in and manually update the configs on all the relevant servers. The idea behind service discovery is to prevent that.
Let's say you have a load balancer with 4 web servers behind it, each with its own IP.
If you hit a traffic spike, you'd like your system to automatically provision another web server and add it to the load balancer with no manual intervention.
With a Service Discovery server, your load balancer could be polling the server for a list of nodes it needs to add or remove from its pool. It could also send a signal to the service discovery server that will indicate that it's dealing with a higher traffic threshold. This signal can then be captured by another Provisioning service (also polling Service Discovery) which then triggers the building of a new web server. The web server could then signal to the load balancer to add it to the pool.
Without a system like this, a person has to go in and manually update the configs on all the relevant servers. The idea behind service discovery is to prevent that.
If you're using hostnames in your config file, that is a form of service discovery. One that relies on DNS. But DNS is a general solution. Are you using a public DNS server? What if it goes down? What's the minimum TTL it'll support?
Also, don't you find it a little odd that you use a hostname to avoid being bound to a specific IP, but you've still hardcoded a port? Either there's value in being decoupled from the address of the service -- which must include the port -- or there isn't.
It generally comes down to gaining more control over the process to achieve higher availability and greater flexibility. Maybe you want all authentication requests from frontend12 to be routed to auth2 for some reason. At scale, it's unruly to do these types of changes by touching config files.
Also, don't you find it a little odd that you use a hostname to avoid being bound to a specific IP, but you've still hardcoded a port? Either there's value in being decoupled from the address of the service -- which must include the port -- or there isn't.
It generally comes down to gaining more control over the process to achieve higher availability and greater flexibility. Maybe you want all authentication requests from frontend12 to be routed to auth2 for some reason. At scale, it's unruly to do these types of changes by touching config files.
That's actually what I'm doing right now, keeping everything in a config file in a directory accessible by all machines.
It's definitely worked fine so far.
I was wondering if others had other solutions.
It's definitely worked fine so far.
I was wondering if others had other solutions.
What are some of the pitfalls I should be aware of?
Should I keep things simple and roll my own using Redis backed by sentinel/clustering?