Rest API Timeout and DNS Round Robin – Something to Be Aware Of

Java
,
1

I am calling a Rest API from Spring Boot RestTemplate having ConnectTimeout 15 sec.
But I am getting connection timeout error after 30 sec.

Why I am getting this error after 30 sec instead of 15 sec. What could be the possible reason?

2

This is what I found on internet, so please take this with a pinch of salt.

The scenario you’re describing, where you set a connection timeout of 15 seconds for a REST API call using Spring Boot’s RestTemplate, but you’re experiencing a connection timeout error after 30 seconds, could be influenced by a few factors:

  1. Request Queuing: If the server you are calling is under heavy load and the number of incoming requests is high, it might lead to queuing of incoming requests. As a result, your request might not be processed immediately, causing the connection timeout to occur after a longer period.

  2. Connection Pooling: Spring’s RestTemplate uses a connection pool to manage and reuse connections. If the maximum number of connections in the pool has been reached and no connections are available, the client might wait for a connection to become available. This waiting time could extend beyond your specified connection timeout.

  3. DNS Resolution: The time taken for DNS resolution can also contribute to the delay. If the hostname of the API needs to be resolved to an IP address and this process takes longer than expected, it can lead to an increased overall connection time.

  4. Network Latency: Network latency can play a role in the delay as well. If the physical distance between your client and the server is significant, it can contribute to the overall time taken for the connection to be established.

  5. Intermediate Network Devices: Firewalls, proxies, and other network devices between your client and the server can introduce delays. These devices might need time to process and forward the requests, adding to the overall connection time.

  6. Server-Side Processing: The server might be taking longer to respond due to its internal processing. While the client-side connection timeout is set to 15 seconds, the server-side processing time could contribute to the overall time before the error is reported.

To troubleshoot and identify the specific reason for the delay, you can consider:

  • Checking the server logs to see if there are any indications of why the response is taking longer.
  • Using network monitoring tools to analyze the network traffic and identify potential bottlenecks.
  • Reviewing the configuration of any intermediaries (e.g., load balancers, proxies) between your client and the server.

Adjusting the connection timeout to a larger value temporarily for debugging purposes might help you identify if the issue is consistently occurring after a certain period, which could provide additional insights into the problem.

1 Like
3

Let me first explain the ConnectTimeout

  • Connection timeout is used when opening a communications link to the remote resource.
  • A java.net.SocketTimeoutException is thrown if the timeout expires before the connection can be established.

@DebugHorror Point 3 DNS Resolution is the main culprit in this case.

  • It’s quite common that some larger domains will be using a DNS round robin configuration – essentially having the same domain mapped to multiple IP addresses.
  • This introduces a new challenge for a timeout against such a domain, simply because of the way HttpClient will try to connect to that domain that times out:
  • HttpClient gets the list of IP routes to that domain:
    • it tries the first one – that times out (with the timeouts we configure(15 sec))
    • it tries the second one – that also times out (15 sec)
    • and so on …

We can see the same in below screenshot:

|928px;x253px;
|928px;x253px;1247×340 78.2 KB

3 Likes
4

Excellent observation but I would digress that it is mainly because of DNS round robin. Issue could be because of multiple things.

Any comment?

1 Like
5

@DebugHorror

Here ask is for connection timeout, and its taking 2x sec (Dependent on IP list) instead of x sec.

Request Queuing : Request queue means we have established the connection and waiting for the response (read timeout could be thrown).

Connection Pooling : We get ConnectionRequestTimeout here.

Network Latency or Intermediate Network Devices :: This will not impact because I have set connection timeout for x sec. if server is not reachable within x sec then we must get the connection timeout error after x sec.

Server-Side Processing : Here we have established the connection, waiting for the response (read timeout could be thrown).

Have a look at below image. Hope you got the answer.

image
image1366×768 76 KB

3 Likes