Load balancer does not have available server for client: api-security

Upon login in our application on production environment, we are regularly getting Internal server error 500, message as :

“com.netflix.client.ClientException: Load balancer does not have available server for client: api-security”

Error screenshot below:

For this, infra team had increased the memory allocation to “api-security” microservice based on discussion with Mayank Goyal but after some days, it has again popped up.

It gets resolved temporarily after services restart.

What is the permanent solution for this.

6 Likes

@naveen.gupta @Vikas_Dhillon Could you please answer?

2 Likes

Lets understand this error first:

Our VRT microservices are communication with each other via Service Discovery(Eureka/Consul).

This error occurs when no service instance of api-security is registered at that moment.

Suppose a request comes from front-end to router microservice and router microservice want to communicate with api-security via Service Discovery(Eureka/Consul), But no instance of api-security is available on Service Discovery(Eureka/Consul) then this error has occurs.

Possible Root Cause:

  1. Number of request increased. due to this Service Discovery(Eureka/Consul) not able to fetch the status from given time and deregistered.
  2. api-security was down due to GC Overhead, Heap Spaces error

How to Fix:

  1. Need to scale based on request analysis or Based on Load of the system.
  2. Increase heap space.
6 Likes