/support/notes/service/sap_logo.png){kind=logo}
SAP Knowledge Base Article - PreviewSymptom
- Resource Container Shrinks and memory pressure are occurring, where persistent memory under the 'NVM_ALLOCATED_SIZE' space is the highest consumer :
[22209]\{358011\}[38/3761858043] 2023-05-22 05:38:32.140031 i TraceContext TraceContext.cpp(01347) : UserName=SAPABAP1, ApplicationUserName=SAPSUPPORT1, ApplicationName=ABAP:SID, ApplicationSource=CL_FINS_SDM_ACDOCA_NETDT_AMDP=CP:39, Client=100, StatementHash=4c79d84fa817dd6feb18ae674e97517c, EppRootContextId=C15DB5FF84811EDDBE8FDC07A7646F09, EppTransactionId=3DF5405E6F9E0080E00646AA2D555375, EppConnectionId=646A4C25A2B656F2E10000000AA70612, EppConnectionCounter=1, EppComponentName=SID/host_SID_00, EppAction=R_SDM_ONLINE_EXEC, StatementExecutionID=844438590200945
[22209]\{358011\}[38/3761858043] 2023-05-22 05:38:32.140031 w ResMan ResourceContainerShrink.cpp(03167) : Information about shrink at 2023-05-22 05:38:32 134 Local:
Reason for shrink: Reserve for HBA, user size: 893344248
ShrinkCaller
......
IPMM short info:
GLOBAL_ALLOCATION_LIMIT (GAL) = 972.74gb (1044479148032b), SHARED_MEMORY = 20.74gb (22276599808b), CODE_SIZE = 3.56gb (3830210560b), NVM_ALLOCATED_SIZE = 774.61gb (831733710848b), OOM_RESERVATION_ALLOCATOR_SIZE = 96.14mb (100810752b)
PID=12674 (hdbindexserver), PAL=972.87gb (1044613365760b), AB=162.73gb (174735241216b), UA=0, U=150.95gb (162085162541b), FSL=0b, FMC=0b
[Some processes may have been skipped]
Total allocated memory= 971.06gb (1042668449792b)
Total used memory = 956.38gb (1026912509163b)
Sum AB = 184827928576
Sum Used = 169071987947
Heap memory fragmentation: 1
Top "M_HEAP_MEMORY" allocators (component, name, size). Ordered descending by inclusive_size_in_use.
1: System: / 153.80gb (165147820401b)
2: System: Pool 149.66gb (160705447952b)
3: System: Pool/PersistenceManager 70.84gb (76072626640b)
4: System: Pool/PersistenceManager/PersistentSpace 46.82gb (50278516832b)
5: System: Pool/PersistenceManager/PersistentSpace/DefaultPageProvider:DataPage 32gb (34367688704b)
6: Statement Execution & Intermediate Results: Pool/JoinEvaluator 29.01gb (31150471716b)
7: System: Pool/ColumnStore 27.82gb (29872568965b)
8: Statement Execution & Intermediate Results: Pool/JoinEvaluator/JEStep2 19.21gb (20635558840b)
9: Column Store Tables: Pool/PersistenceManager/UnifiedTableContainer 17.67gb (18981965152b)
10: Column Store Tables: Pool/ColumnStore/Main 13.12gb (14088848129b)
11: System: Pool/PersistenceManager/PersistentSpace/LOBPage 12.02gb (12910800896b)
12: Column Store Tables: Pool/ColumnStore/Main/Compressed 10.02gb (10760845944b)
13: Column Store Tables: Pool/ColumnStore/Main/Compressed/Indirect 10.02gb (10760831544b)
14: Statement Execution & Intermediate Results: Pool/JoinEvaluator/TranslationTable 9.54gb (10254337208b)
15: Statement Execution & Intermediate Results: Pool/ColumnStore/Transient 7.28gb (7824327804b)
16: System: Pool/RowEngine 6.09gb (6545781754b)
17: Other: Pool/ColumnStore/TableOptimization 4.10gb (4404060656b)
18: Column Store Tables: Pool/PersistenceManager/UnifiedTableContainer/MVCC 3.60gb (3868247752b)
19: Monitoring & Statistical Data: Pool/Statistics 3.33gb (3576032528b)
20: System: Pool/PersistenceManager/VarSizeEntryFreeSpaceInformation 3.06gb (3289791512b)
Top "M_HEAP_MEMORY" allocators (component, name, size). Ordered descending by exclusive_size_in_use.
1: System: Pool/PersistenceManager/PersistentSpace/DefaultPageProvider:DataPage 32gb (34367688704b)
2: Statement Execution & Intermediate Results: Pool/JoinEvaluator/JEStep2 20.04gb (21528904632b) (851.96mb (893345792b) precharged, 851.96mb (893345792b) in preparation)
3: Column Store Tables: Pool/PersistenceManager/UnifiedTableContainer 14.07gb (15113717400b)
4: System: Pool/PersistenceManager/PersistentSpace/LOBPage 12.02gb (12910800896b)
5: Column Store Tables: Pool/ColumnStore/Main/Compressed/Indirect 10.02gb (10761523224b)
6: Statement Execution & Intermediate Results: Pool/JoinEvaluator/TranslationTable 9.54gb (10254337208b)
7: Statement Execution & Intermediate Results: Pool/ColumnStore/Transient 7.28gb (7824327804b)
8: Other: Pool/ColumnStore/TableOptimization 4.10gb (4404060656b)
9: Column Store Tables: Pool/PersistenceManager/UnifiedTableContainer/MVCC 3.60gb (3868247752b)
10: System: Pool/PersistenceManager/VarSizeEntryFreeSpaceInformation 3.06gb (3289791512b)
11: Row Store Tables: Pool/RowStoreTables/CpbTree 2.73gb (2932837120b)
12: Column Store Tables: Pool/ColumnStore/Main/Rowid 2.68gb (2879124952b)
13: System: Pool/RowEngine/QueryExecution/SearchAlloc 2.65gb (2846447064b)
14: Caches: Pool/RowEngine/SQLPlan 2.33gb (2502457714b)
15: Statement Execution & Intermediate Results: StackAllocator 1.90gb (2043691008b)
16: System: AllocateOnlyAllocator-unlimited/FLA-UL<24592,1>/MemoryMapLevel3Nodes 1.70gb (1827947952b)
17: Column Store Tables: Pool/ColumnStore/Delta 1.29gb (1395456960b)
18: System: Pool/ResourceContainer 1.28gb (1377069192b)
19: Monitoring & Statistical Data: Pool/Statistics/Semaphores/CondVars 1.21gb (1308691200b)
20: System: Pool/PersistenceManager/PersistentSpace/DefaultPageProvider:Controlblock 1.17gb (1262125824b)
Process: AB=162.73gb (174735241216b), U=153.80gb (165147820401b), fragmentation=5%.
.....
- Delta Merge statistics show out of memory failures during critical and manual hard merges for large PMEM tables, in this example ACDOCA. SQL : SELECT * FROM M_DELTA_MERGE_STATISTICS WHERE SUCCESS = 'FALSE'
HOST PORT TYPE SCHEMA_NAME TABLE_NAME START_TIME EXECUTION_TIME MOTIVATION SUCCESS CS_ERROR ERROR_DESCRIPTION
vhmglsh1db01 30,440 MERGE SAPABAP1 ACDOCA 2023-05-22 05:38:11.9500000 435,017 CRITICAL FALSE 2,489 column store error: [2489] Out of memory during delta merge; table = SID::SAPABAP1:_SYS_SPLIT_ACDOCA~4 (t 111888), exception = exception 1000002: Allocation failed ; $size$=42207880; $name$=Prefix; $type$=pool; $inuse_count$=158; $allocated_size$=84544528; $failure_type$=GLOBAL_ALLOCATION_LIMIT; $failure_flag$= (early exit)
vhmglsh1db01 30,440 MERGE SAPABAP1 ACDOCA 2023-05-22 06:31:50.9300000 446,707 HARD FALSE 2,489 column store error: [2489] Out of memory during delta merge; table = SID::SAPABAP1:_SYS_SPLIT_ACDOCA~14 (t 111888), exception = exception 1000002: Allocation failed ; $size$=536870912; $name$=PageableSingle; $type$=pool; $inuse_count$=1828; $allocated_size$=1148950304; $failure_type$=GLOBAL_ALLOCATION_LIMIT; $failure_flag$=
vhmglsh1db01 30,440 MERGE SAPABAP1 ACDOCA 2023-05-22 06:23:29.4420000 1,024,161 HARD FALSE 2,489 column store error: [2489] Out of memory during delta merge; table = SID::SAPABAP1:_SYS_SPLIT_ACDOCA~8 (t 111888), exception = exception 1000002: Allocation failed ; $size$=5759008; $name$=Rle; $type$=pool; $inuse_count$=37; $allocated_size$=3212901968; $failure_type$=GLOBAL_ALLOCATION_LIMIT; $failure_flag$=
vhmglsh1db01 30,440 MERGE SAPABAP1 ACDOCA 2023-05-22 06:23:20.4990000 1,146,441 HARD FALSE 2,489 column store error: [2489] Out of memory during delta merge; table = SID::SAPABAP1:_SYS_SPLIT_ACDOCA~4 (t 111888), exception = exception 1000002: Allocation failed ; $size$=976660480; $name$=Indirect; $type$=pool; $inuse_count$=2953; $allocated_size$=8773779104; $failure_type$=GLOBAL_ALLOCATION_LIMIT; $failure_flag$= (early exit)
vhmglsh1db01 30,440 MERGE SAPABAP1 ACDOCA 2023-05-22 06:23:20.4990000 1,209,660 HARD FALSE 2,489 column store error: [2489] Out of memory during delta merge; table = SID::SAPABAP1:_SYS_SPLIT_ACDOCA~2 (t 111888), exception = exception 1000002: Allocation failed ; $size$=1381054504; $name$=Rle; $type$=pool; $inuse_count$=6; $allocated_size$=6144; $failure_type$=GLOBAL_ALLOCATION_LIMIT; $failure_flag$=
vhmglsh1db01 30,440 SPARSE SAPABAP1 ACDOCA 2023-05-22 06:30:45.3180000 945,196 CRITICAL FALSE 2,009 "column store error: [2009] Memory allocation failed;prepareOptimizations,object=SID::SAPABAP1:_SYS_SPLIT_ACDOCA~5en,exception 70002009: Memory allocation failed;PrepareOptimizationsJob,object=SID::SAPABAP1:_SYS_SPLIT_ACDOCA~5en,exception 1000002: Allocation failed ; $size$=139831048; $name$=Single; $type$=pool; $inuse_count$=93894; $allocated_size$=1259046168; $failure_type$=GLOBAL_ALLOCATION_LIMIT; $failure_flag$= (early exit)
,out of memory during optimize compression"
vhmglsh1db01 30,440 SPARSE SAPABAP1 ACDOCA 2023-05-22 06:30:45.2960000 987,327 CRITICAL FALSE 2,009 column store error: [2009] Memory allocation failed;doOptimizeCompression,object=SID::SAPABAP1:_SYS_SPLIT_ACDOCA~3en,exception 1000002: Allocation failed ; $size$=38731164; $name$=SAPABAP1:_SYS_SPLIT_ACDOCA~3; $type$=pool; $inuse_count$=286; $allocated_size$=5620619568; $failure_type$=GLOBAL_ALLOCATION_LIMIT; $failure_flag$= (early exit)
vhmglsh1db01 30,440 SPARSE SAPABAP1 ACDOCA 2023-05-22 06:48:27.3170000 114,701 CRITICAL FALSE 2,009 "column store error: [2009] Memory allocation failed;prepareOptimizations,object=SID::SAPABAP1:_SYS_SPLIT_ACDOCA~3en,exception 70002009: Memory allocation failed;PrepareOptimizationsJob,object=SID::SAPABAP1:_SYS_SPLIT_ACDOCA~3en,exception 1000002: Allocation failed ; $size$=993871304; $name$=Single; $type$=pool; $inuse_count$=93834; $allocated_size$=113002904; $failure_type$=GLOBAL_ALLOCATION_LIMIT; $failure_flag$=
,out of memory during optimize compression"
- A quick check of the affected table (SELECT * FROM M_CS_TABLES WHERE TABLE_NAME = 'ACDOCA' ) , shows it's PMEM size is 152 GB alone under PERSISTENT_MEMORY_SIZE_IN_TOTAL.
152,716,037,048
- Total size of the overall columnstore tables in PMEM is 600 GB (SELECT SUM (PERSISTENT_MEMORY_SIZE_IN_TOTAL) FROM M_CS_TABLES), leaving < 400 GB for heap memory and delta merge operations.
601,297,655,942
Read more...
Environment
SAP HANA , Platform Edition 2.0
Product
SAP HANA, platform edition 2.0
Keywords
NVM_ALLOCATED_SIZE, ResourceContainerShrink.cpp, TraceContext.cpp, GLOBAL_ALLOCATION_LIMIT, Total used memory, , KBA , HAN-DB , SAP HANA Database , HAN-DB-ENG , SAP HANA DB Engines , Problem
About this page
This is a preview of a SAP Knowledge Base Article. Click more to access the full version on SAP for Me (Login required).Search for additional results
Visit SAP Support Portal's SAP Notes and KBA Search.
/support/notes/service/facebook.svg)
/support/notes/service/twitter.svg)
/support/notes/service/youtube.svg)
/support/notes/service/linkedin.svg)
/support/notes/service/instagram2.svg)