Upon login in our application on production environment, we are regularly getting Internal server error 500, message as :
“com.netflix.client.ClientException: Load balancer does not have available server for client: api-security”
Error screenshot below:
For this, infra team had increased the memory allocation to “api-security” microservice based on discussion with Mayank Goyal but after some days, it has again popped up.
It gets resolved temporarily after services restart.
What is the permanent solution for this.
Lets understand this error first:
Our VRT microservices are communication with each other via Service Discovery(Eureka/Consul).
This error occurs when no service instance of api-security is registered at that moment.
Suppose a request comes from front-end to router microservice and router microservice want to communicate with api-security via Service Discovery(Eureka/Consul), But no instance of api-security is available on Service Discovery(Eureka/Consul) then this error has occurs.
Possible Root Cause:
- Number of request increased. due to this Service Discovery(Eureka/Consul) not able to fetch the status from given time and deregistered.
- api-security was down due to GC Overhead, Heap Spaces error
How to Fix:
- Need to scale based on request analysis or Based on Load of the system.
- Increase heap space.