LoadBalancer: support instance selection based on specific actuator metrics

[kmandalas]

Provide more fine-grained load balancing rules based on actuator metrics like system.cpu.usage, jvm.memory.usage etc. Could be something configurable based on a specific metric and a corresponding threshold value. If this value is exceeded the balancer will prefer instances below this value. Otherwise default behavior will be applied.

Alternatively I assume this can be achieved by implementing a custom HealthIndicator but a more out-of-the-box configuration-only capability could be very useful.

[spencergibb]

#601 is along those lines. It's fairly hard to do since you would want those metrics from other instances, how do you report that the load balancer.

[kmandalas]

@spencergibb so the only solution would be to implement a custom HealthIndicator and rely on the existing HealthCheckServiceInstanceListSupplier?

[spencergibb]

that's one option, yes.

[kmandalas]

@spencergibb I think that custom HealthIndicator should be avoided though since it would show a status [DOWN] if some instance is under load exceeding some threshold and this is will not be accurate. I guess implementing a custom ReactorServiceInstanceLoadBalancer could be the only alternative at the moment.

[kmandalas]

@spencergibb & @OlgaMaciaszek I am working on a custom ReactorServiceInstanceLoadBalancer at the moment cause I want to achieve a "least_conn" behavior (similarly to NGNIX or HA_PROXY). This is useful in many cases one of which is when you need to load balance WebSocket connections from the Spring Cloud Gateway to multiple instances of WebSocket servers.

More specifically I have started creating a LeastConnLoadBalancer (alternatively a LeastConnServiceInstanceListSupplier) that will ping service instances at a configurable actuator endpoint to get number of users/connections and will choose the one with the least number. Caching with a TTL is also considered in order to avoid hitting every time the endpoint.

I will be watching the parent issue #601 but if you thing a PR would make sense when I complete it, please let me know.

[OlgaMaciaszek]

@kmandalas Definitely, if you come up with something, do submit a PR. You might also want to keep an eye on what is happening with the following issues: #675 (currently a PR in review - introduces possibilities to propagate load-balanced call data and to run a callback method after a load-balanced call has been completed; probably best to base your changes on that) and #674 (planning to work on adding in micrometer here).