2767376 - Employee Records missing in the API response while your scheduled replications are running using LastModifieOn filter / parameter - Successfactors HXM Suite

You are noticing that few records are getting missed in the response payload while making  CompoundEmployee API calls or OData calls running your processes in DELTA mode using the field LastModifiedOn as filter i.e. pick only those records which has been changed since last successful run.

One of the potential root cause would be data was not saved completely to database (DB) while API request was triggered i.e. around the time that you see in your API request for LastModifieOn filter parameter.

This usually occurs when:

• Your HR do mass imports using CSV files (Admin center > Import Employee Data or Import and Export Data)
• Your team uses EC Business rules with PostSave

The standard replications or custom replications triggers API call at the very same time when your team is doing changes via CSV mass import and/or EC Business rules with PostSave which can delay the processing of the records in SF database, the filter of LastModifiedOn of the previous run and the subsequent run would not capture the changed employee record.

We have seen this behavior of records missed in the standard replications and no errors can be seen in the Data Replication Monitor UI, middlewares Boomi/CPI or ERP side (SLG1/SRTUTIL). The employee records are updated successfully in the EC system but this information is not received in the ECP/ERP application.

This also occurs more often if your team decided to run DELTA integrations in shorter period (like 1, 2 or 5 minutes between the runs).

SAP SuccessFactors HXM Suite

• CompoundEmployee API
• OData API
• SFAPI 

There is one time gap between the last modified and DB storage in the scenarios described above for replications with APIs running the same time of the data being changed in the SF database level.

If there are any news about further improvements for this topic, they will be communicated via this KBA.

• For the situations where you use EC Business rules with PostSave, one possible solution is changing for OnSave instead, this will avoid this database delay of the field "LastModifieOn".

• For the situations where you do not have the EC Business rules and you still have missing records while performing CSV mass imports, as a workaround to avoid any data loss during API call based on middleware and/or standard content used, the recommendation is:

• scheduling additional nightly job would make sure all those records which went missing get replicated successfully.

Also, there would not be any impact on employees whose data has already been replicated successfully during standard scheduled executions.

For that, we are explaining below how to do this process for standard Integration scenarios available in EC Payroll point to point Integration, CPI or Boomi middleware. You can adapt these scenarios for your other possible integrations too (custom application or other scenarios relying with SF APIs OData and SFAPI).

Employee Central Payroll (Point-to-point Integration)

• Copy the affected PTP configuration ID
• via following IMG Activity: Personnel Management -> Integration Settings for SuccessFactors Employee Central Payroll -> Configuration of Point-to-Point Replication -> Configure Compound Employee API Query
• Or via SM34 and view cluster T77SFEC_PTP_COCL 

• Adjust the last modified date for copied PTP configuration ID.
• Go to transaction SM30
• Maintain view HRSFEC_PTP_LMOD
• Add a new entry with the copied PTP configuration ID and adjust the last modified timestamp correctly.

• Schedule a new nightly job for the copied PTP configuration ID.

DELL Boomi 

• For standard content deployed on boomi atom, you can setup another atom where you need to deploy the same version of the standard iFlow with the same configuration.
• You should schedule the deployed iFlow to execute outside business hours to pick all the changes that has been done in a day.
• Last Modified Date in the configuration should be selected accordingly.

Example:

• Go to the atom where your standard iFlow has been deployed and execution currently. Go to the deployed process and go to the "Edit Process Properties".
• Here, you would see highlighted parameter value which will be used in your next iFlow execution and this would be set as value for filter parameter "LastModifiedOn"

• For new atom where you are going to deploy same iflow again, make sure all configuration under "Environment Extension" is same as one used in standard schedule.
• Only, property that you need to set is under "Edit Process Properties" as highlighted above. So, add the property "ECERP_LAST_MODIFIED_DATE" manually and set the value as per your requirement.
• Note that format shout be in UTC only as shown above. So, before you set the value, make sure you convert the timestamp in UTC and maintain here.

SAP Cloud Platform Integration

• CPI scheduled:
• Copy the process second time from the process library (can be deployed on the same tenant) Complete the configuration same as the existing iflow (except the scheduling part). Schedule outside business hours to catch-up all the missed changes. as shown below
• Make sure new iFLow has all configuration similar to the standard schedule one apart from one parameter "USER_SET_LAST_MODIFIED_DATE_TIME" as shown below:

• CPI scheduled via ERP (lastest version BIB):
• Create a second scheduling job in ERP. For this you would have to create a second selection variant used in ECPAO_EE_ORG_REPL_QUERY with the same configuration which you are currently using for Variant while executing the Job. This way you can have 2 different Last Modified On. This is also applicable for the OM job in the program RH_SFIOM_ORG_OBJ_REPL_QUERY.
• Schedule outside business hours using the newly created variant to pick all the missed changes.
• We would not suggest to perform full load scenario using this new iFlow deployed as this would cause performance issue in SF and SAP Payroll both as number of records to be processed would be more.
• We would suggest you to fetch the changes from a dedicated time range. 

We have this behavior documented in the SFAPI CE handbook (direct link).

We also have this behavior documented in the KBA 2092850 - How to make changes to Employee Data on mass via Import - Employee Central

Records Missing, Employee Master Data Replication, Organization Assignment, CSV mass massive import, Admin center > Import Employee Data or Import and Export Data, API, SFAPI, CompoundEmployee calls, OData, SuccessFactors HCM Suite, data extraction, skipped employees record, time gap, LastModifiedOn, filter, persons not extracted , KBA , LOD-SF-INT-CE , Compound Employee API , LOD-SF-INT , Integrations , LOD-SF-INT-ODATA , OData API Framework , LOD-SF-INT-API , API & Adhoc API Framework , Problem