3215139 - Replication of remote table fails due memory shortage - SAP Datasphere

• Error: {"timestamp":20XX-XX-XX XX:XX:XX.0000000,"origin":"_sys_task","message":"internal error: Error opening the cursor for the remote database No more memory available to add rows to an internal table. at federation/fedservice/FederationQueryExecutorManager.cpp:227; Statement: ,INSERT INTO...
• Error: {"timestamp":20XX-XX-XX XX:XX:XX.0000000,"origin":"_sys_task","message":"transaction rolled back by an internal error: Remote source CONNECTION is unable to prepare transaction id XXXXXXXXX at federation/fedservice/FederationContextTXN.cpp:352; Statement: ,INSERT INTO...
• Error: {"timestamp":20XX-XX-XX XX:XX:XX.0000000,"origin":"_sys_task","message":"cannot allocate enough memory: Allocation failed $REASON$ (unknown SQLScript function(s))"}
• General error;403 internal error: Error opening the cursor for the remote database No more memory available to add rows to an internal table.
• Running out of memory in DP Agent (SDI ABAPAdapter). "Agent is running very low on memory" or "Agent is shutting down".
• Error: {"timestamp":20XX-XX-XX XX:XX:XX.0000000,"origin":"_sys_task","message":"internal error: Agent is shutting down.
  at ptime/query/plan_executor/dml/hex/qe_hex_search.cc:499; Statement: ,INSERT INTO
• In source system there is memory dump:

Category Resource bottleneck
Runtime Errors TSV_TNEW_PAGE_ALLOC_FAILED

No more memory available to add rows to an internal table.

"TSV_TNEW_PAGE_ALLOC_FAILED"
"/SAPDS/SAPLRS_BASIS" bzw. /SAPDS/LRS_BASISU20
"Z_BODS_RFC_READ_TABLE2_FORM"

5 FORM         /SAPDS/SAPLRS_BASIS                 /SAPDS/LRS_BASISU20                   390
  Z_BODS_RFC_READ_TABLE2_FORM
4 FUNCTION     /SAPDS/SAPLRS_BASIS                 /SAPDS/LRS_BASISU20                    37
  /SAPDS/RFC_READ_TABLE2
3 FORM         /SAPDS/SAPLRS_BASIS                 /SAPDS/LRS_BASISU20                     1
  /SAPDS/RFC_READ_TABLE2
2 FORM         SAPMSSY1                            SAPMSSY1                              189
  REMOTE_FUNCTION_CALL
1 MODULE (PBO) SAPMSSY1                            SAPMSSY1                               35
  %_RFC_START

SAP Datasphere

• Replicating remote table from Connection type SAP ABAP, SAP ECC, SAP BW or SAP S/4HANA On-Premise which uses SDI ABAPAdapter.
• Remote table is a SAP ABAP Dictionary table.
• Running Data Flow which contains Remote table as a Source. 

Memory shortage occurs because SAP Datasphere requests DP Agent to Insert data from ABAP system. 
When ABAP Adapter Connection is not using streaming, SDI uses /SAPDS/RFC_READ_TABLE function that write the source table to an ITAB (internal table) in the ABAP system.
This can cause the system to run out of heap memory (abap/heap_area_dia or abap/heap_area_total).

As mentioned in Help Portal SAP ABAP Adapter Functionality (ABAPAdapter), the memory consumption could be very high because only limited filters can be pushed down to remote sources.

See resolution in KBA 2861324 - No more memory available to add rows to an internal table - SAP HANA Smart Data Integration 

• Document SAP Data Warehouse Cloud - First Guidance: Data Integration for ABAP Source Systems
• Document SAP Datasphere - First Guidance: Development Guidelines and Naming Conventions
• SAP Help Portal SAP HANA Smart Data Integration and SAP HANA Smart Data Quality - Field Delimited Extraction
• KBA 2861324 - No more memory available to add rows to an internal table - SAP HANA Smart Data Integration
• SAP Note 3139720 - SAP Datasphere: View Persistency aborts with an Out of Memory exception or consumes to much memory
• KBA 3215139 - Replication of remote table fails due memory shortage

Note: For ABAP managed objects (BW Objects, CDS Views), consider using replication flows to replicate the data.

source, system, HANA Adapter, ABAP Adapter, smart, data, integration, Prefetch timed out, pre fetch, time, out, pre, fetch, dwc, data warehouse cloud, ACDOCA , KBA , HAN-DP-SDI-DS , SAP Datasphere specific SDI issues , Problem