2735876 - Odata API Best Practices [Query Modified Records, Pagination - Batch Size, Timeouts, etc.,] - SAP Successfactors Odata API

• Query only modified records

Instead of querying all records, query only the records that have been modified since your last execution for integration use cases, and use server pagination to ensure stable results. For more information on server pagination, see Pagination.

• Use server pagination for integration use cases

Compared with client pagination, server pagination is faster and does not suffer data loss and duplication issues when simultaneous edits occur in the same instance. You can choose either cursor-based or snapshot-based pagination for your integration use case. See Pagination for more information.

• Do not run jobs too often to get real-time results

Repeating queries in much less than one hour will consume too much API resource, which may be throttled in the future. In extreme cases, you may even be denied of service because of frequent queries, especially ones that require a lot of backend processing.

• Use batch or $filter to get multiple records

Instead of pulling many records one at a time using key predicate, use batch or the $filter IN clause.

In OData API, a user login session is created on the server for each request. This is a resource demanding process, especially when login audit is enabled. A high volume of requests may lead to excessive usage of system resource and drastically slow down performance. Therefore, we encourage you to use batch operation or the IN clause in $filter query option whenever possible. For example, instead of using the ToDo API to query multiple users with many requests, use the TodoEntryV2 which allows you to do it in one request.

See The $filter Keyword for more information.

• Reuse login sessions

Creating login session is a resource demanding task. Instead of creating a session for each HTTP transaction or each page of paginated data, reuse login sessions.

See Enabling Session Reuse for more information.

• Do not use an API for integration that is only designed for single user

APIs designed for single users should not be used for integration purpose by iterating requests for all uses. This will create one login session for each user and significantly slow down performance. One example is the ToDo API which only allows querying data of a single user. For integration purpose, use the new TodoEntryV2 API. TodoEntryV2 allows you to query items of multiple users with the OData API Todo Export permission.

See TodoEntryV2 for more information.

• Tune your batch requests into proper sizes

The OData API can return a maximum number of 1000 records in a single page. You should tune your batch sizes to be as large as possible. For more complex transactions, you may need to decrease the size to avoid HTTP timeouts.

• Do not upsert users in single API calls (One Api call per user) when performing migration into Successfactor. 

• Avoid large $expand statements

For example, an attempt to query all JobRequesitions and then expand JobApplications with all attachments for each application is condiered too complex and large, which is error-prone and may be throttled in the future.

The recommend master-detail implementation would be:

1. Bind the master list to simple top level Job Requisition so UI5 loads in on demand via batch operations as the user scrolls.
2. When the user clicks on a job requisition, only then do you bind the detail list to candidate, filtering on the selected job requisition id.
3. Only load resume and cover letter when you click on a job candidate.

Poor performance is often a sign of misuse of APIs. As a rule of thumb, always keep your transactions simple.

• Do not query properties and expand entities you do not need or use

It is easy to build a query that does more $select and $expand than you need. This is especially easy with the Boomi connector which has UI limitations due to limitations in the Dell Boomi connector toolkit.

As a developers, you might get tired of tweaking your queries to match the content as requirements change so often. You might be tempted to just pull more than you need and release the queries to production without tuning.

The Integration Center can help avoids this. Instead of a building a query manually based on the mapping of fields, the Integration Center generates the query based on the mapping.

• Tune your client wait time to match the system wait time

Your client should be set to wait a reasonable amount of time before timing out. Complex operations can take as long as 10 minutes and our network and servers will continue to process a transaction for that long. It is best to wait for the transaction to complete rather just wasting the transaction.

You can also tune your transaction to avoid timeouts.

• Implement your retry logic properly

Retry logic can help to recover transactions that failed due to internet connectivity or backend server issues, but retry must be done with care based on the type of HTTP error.

In all cases you must "sleep" a reasonable amount of time from 1 to 5 minutes before attempting a retry. "Obsessing" by immediately retrying may cause a denial of service condition and not give a server time to recover.

Only return a maximum number 5 of times before abandoning your task. For example the Boomi connector retries only a maximum of 5 times.