3580750 - Performance tips & tricks for SAP S/4HANA Migration Cockpit: Migrate Data Using Staging Tables in S/4 HANA Public Cloud Edition

• The possible options on how to improve the performance of the data migration while using the Migration Cockpit, approach "Staging Tables".
• The technical measures as well as in non-technical measures to improve the performance of the data migration.
• The performance issues while using the Migration Cockpit – Migrate your Data App, Migrate Data Using Staging Tables

SAP S/4HANA Cloud Public Edition

Note that these explanations refer to the Staging Tables / Migrate data using staging tables approach.

It is important to understand that there are non-technical and technical measures that influence the runtime of the data migration. The non-technical measures include planning and organizational aspects. Considering and applying such measures might help to reduce the system downtime during data migration projects. 

Non-technical measures

1. Check data in the source system

Verify thoroughly which data needs to be transferred to the new system and when possible, perform data cleansing. There may be outdated data (for example, old cost centers, materials used the last time x years ago) or data which has been created by mistake. For example, materials with a wrong ID, so never used after creation) or data which represents from a business perspective the same but has been created multiple times (addresses, customers, vendors...).

Data archiving might be a reasonable pre-project. if there is enough time before the migration project starts.

2. Consider a phased data migration approach

Verify if some migration objects which are modified or created not very frequently (for example cost centers) can be migrated already during system uptime. Communicate to the respective departments to not create or change these migration objects close to the productive data migration to avoid data migration issues.

3. Create a detailed cut-over plan, based on the dependencies of the migration objects

One aspect is to define in detail the sequence in which the migration objects shall be migrated. Consider that there are dependencies between most of the migration objects. The sequence of the migration objects can be found in the migration object documentation.

Do not start the migration of all objects in a migration project in parallel as there is a sequence for migrating objects (due to the dependencies between objects).

4. Using several migration projects and objects in parallel do NOT increase the performance

We recommend preventing a situation where many people are working in parallel on separate migration projects, or where one user triggers multiple activities for several projects in the same data migration context. Uploading multiple objects in parallel can flood the job queue. In our experience, it delivers better results to migrate one object with a full number of jobs, complete the data migration and then start with the migration of the next object. Do this instead of distributing the number of free jobs and start several objects with high data volumes in parallel.

Avoid running parallel processing (i.e. increase the number of data transfer jobs) using several migration projects and objects at the same time if the total number of batch jobs exceeds the available system resources. For more details about data transfer jobs for SAP S/4HANA migration cockpit, staging tables approach see KBA: 3580792 

5. Plan & check active processes during downtime

During the production migration, there should be no other active background processes (jobs) in the system - besides the processes for the migration itself. Check thoroughly which other processes are necessary. Postpone all processes which are not completely necessary.

6. At least one test in a production-like environment (no sandbox with bad performance)

Often, test cycles are executed in small sandboxes with poor performance. This makes it difficult to get good estimations and to figure out optimal settings for production as performance is not scaling linearly with all parameters. Therefore, it is recommended that at least one test should be executed in a production-like environment.

7. Further recommendations for running the migration

If experiencing performance problems, consider executing the data migration in a period when the system has a low workload.

Technical measures

Several factors may influence the performance of the system and therefore, affect the performance of the data migration (for example other processes running in parallel to data migration processes/jobs, host resources, transactional problems, SQL statement performance, security, configuration, authorization issues, etc.). Below are the most relevant parameters that may affect the general performance of the Migration Cockpit / data migration activities:

Number of data transfer jobs per migration object and migration project
System parameters for example the number of available batch and dialog processes, available memory, available CPU or network bandwidth, and latency.
The performance of the API used in the respective migration object has also an influence on the performance.
The application-specific customizing influences the performance of the target API execution as well.

Note: To analyze and solve technical performance issues related to the parameters mentioned above or other system parameters, we recommend to engage a consulting performance expert and/or a basis expert to have the database settings optimized for data migration. 

Furthermore, the following parameters of the SAP S/4HANA migration cockpit can be adjusted and optimized:

1. Define the optimal number of batch processes, data transfer jobs

The Migration Cockpits offers the possibility to parallelize the activities simulation and migration by using several data transfer jobs. Data transfer jobs are responsible for transferring the data in the staging tables to the target SAP S/4HANA Cloud Public Edition system. It is possible to adjust the maximum number of background jobs that are used for the project or a specific migration object. For more details about data transfer jobs of the Migration Cockpit – Migrate your Data App (Staging tables approach) see KBA 3580792 - Modifying Data Transfer Jobs to improve the data transfer performance of the SAP S/4HANA Migration Cockpit - Migrate Data Using Staging Tables in S/4 HANA Public Cloud

Consider that parallelization is meant for migration objects with high data volumes, complex objects with multiple staging tables, or for cases where using one job results in an unacceptable data transfer time. The number of data transfer jobs cannot be defined per activity separately. To define the number of batch jobs per migration object and the maximum number of jobs for the whole project. We recommend that to use the test migrations to test the optimal number of batch processes.

Background: The migration as such runs by using background processes (jobs), but during the procedure, also dialogue processes and update processes are called. The individual case depends on the migration objects used. Therefore, it is necessary to test the allocation of processes and adjust it in a way that is optimal for the objects running in the project.

Also, it is reasonable to distribute the jobs across migration objects with some bare minimum number of jobs per object.

SAP S/4HANA Migration Cockpit, migrate your Data - Migration Cockpit, performance improvement, migration cockpit - staging tables approach, migrate data using staging tables, performance , KBA , CA-LT-MC , S/4HANA Migration Cockpit , Problem