3791355 - Q&A For SAP Datasphere Memory related merge failures in Datasphere Object Store

Regarding to SAP Note 3787958 - Memory related merge failures in Datasphere Object Store, there are some questions regarding implementing the workaround. 

SAP Datasphere

1. What is the workaround?

• With your approval, we will create a new custom configuration ID in Datasphere -> Configuration -> Workload Management -> SAP HANA Data Lake Files for the specific space.
• SAP support will then update the memory parameters for this new custom configuration ID in the backend. This change will apply only to the new ID.
• Once we have finished configuring the new custom ID, we will let you know. You can then assign this ID to your local tables with MERGE_FILES activities and run the merge file activity again to see if the issue persists.

2. What should we do with the new custom configuration ID?

The new configuration ID (e.g., 10001) can be kept, and it is recommended to use it if you encounter the same issue with merges or other tasks using the Spark runtime.

3. Will the workaround cost more?

If the custom configuration is created based on the failed one, the adjustment should have no impact on the consumption. The fix primarily involves increasing the fraction of overhead memory for the executors, while the main executor memory remains the same.

For other cases, you will be informed about the changes and what they imply cost-wise.

4. Why can't I see the updated configuration?

The changes in the configuration can only be seen once the permanent fix is available. That is one aspect of the permanent fix to make the additional parameters visible as part of custom configurations in the future release.

Datasphere, Object Store, Memory issue , KBA , DS-DI-TAB , Table Persistency , Problem