SAP Knowledge Base Article - Public

3271514 - 503 COE_SERVICE_TEMPORARILY_UNAVAILABLE error

Symptom

You wish to understand why you are receiving a 503 error (e.g. COE_SERVICE_TEMPORARILY_UNAVAILABLE or ServiceUnavailable) and what to do to avoid it.

Environment

SAP SuccessFactors API Framework

Reproducing the Issue

There are two potential scenarios for this behaviour:

A)
The API server is an unhealthy state so when an /odata/v2 request with JSESSIONID hits it, the API server will reject the request with a 503 error and respond with a retry-after=300s. header.

B)
The API server tries to get metadata from redis cache and the cache instance is not available. In these scenarios, the metadata will not be retrieved and the API server must resolve the metadata for one API request.
In case more and more API requests for that company hit this API server, an increasing number of threads will be triggered to resolve the metadata.



See Cause below for additional details on both scenarios.

Cause


A) This is expected behaviour.
The /odata/v2 request with JSESSIONID will routed to the sticky server in the JSESSIONID. If the server is in an unhealthy state, the request will be rejected. To resolve this, please retry your integrations after 5 minutes. 
Sample error response:  {"error":{"code":"ServiceUnavailable","message":"Sorry, the server is busy and can't respond to your request. Please try again in 5 minutes."}}



B) Expected behavior, with three potential scenarios:

  1. When these requests hit one server, only the first one can succeed. Other requests will throw a 503 error which will include a retry-after header of 300 seconds 
  2. When these requests hit different servers, only the first request for each server can succeed. Other requests will throw a 503 error which will include a retry-after header of 300 seconds 
  3. In extremely rare circumstances, the root cause may be due to two parallel API calls refreshing the OData metadata simultaneously, thus causing a conflict in the API framework which would lead to error code 503 COE_SERVICE_TEMPORARILY_UNAVAILABLE. 

See Sample Error response below:

Response header: retry-after:300 (s)
Response code: 503
Response body:

{"error" :

{"code" : "COE_SERVICE_TEMPORARILY_UNAVAILABLE", *"message" : \{"lang" : "en-US", "value" : "[COE0040]The request has timed out. OData metadata is being generated for the same instance in another thread. Please try again later."}

}}

Resolution

For scenarios A) and B) 1. and B) 2. above, a retry after 300 seconds will resolve the error.

For scenario B) 3. above: This is by design as the OData metadata cache can only be refreshed/generated once at a time. So far, we have only come across this scenario for customers whose API OData Metadata had not been refreshed for a long time. To prevent similar issues from occurring in the future, our Engineering team will enhance the certificate-based API call in 2H 2023. The metadata refreshed by the technical user will be put into cache, so it will not cause any such conflicts (our internal reference is PLA-36651)

See Also

KBA 2524436 - Q32017 (1708) Enhancement & Features for API-7426 

Keywords

PTCH-39185, COE0040, 503, COE_SERVICE_TEMPORARILY_UNAVAILABLE, PLA-36651, Sorry, the server is busy and can't respond to your request. Please try again in 5 minutes, API-17601  {"error":{"code":"ServiceUnavailable","message":"Sorry, the server is busy and can't respond to your request. Please try again in 5 minutes."}}

<h2>The server encountered a temporary error and could not complete your request.<p>Please try again in 300 seconds.</h2> 

The server encountered a temporary error and could not complete your request

, KBA , LOD-SF-INT-API , API & Adhoc API Framework , LOD-SF-INT , Integrations , LOD-SF-INT-ODATA , OData API Framework , LOD-SF-INT-CE , Compound Employee API , Problem

Product

SAP SuccessFactors HXM Core all versions ; SAP SuccessFactors HXM Suite all versions