3618814 - Work processes are long running with VMC related operations

• Work processes have long runtime on VMC related operations, e.g.<VMC EVENTS COMPILE> and other application programs indicating communications with VMC.
• There are lots of RFC_VMC_COMMUNICATION_ERROR dumps in ST22. 
• In the VM overview of SM52, lots of VMs stuck at starting up status. 

By checking the call stacks of work processes via Kernel Snapshot Analyzer(SAP Note 2640476), several work processes are running with system call mmap() and some are spinning on getting EM2(for VMC memory management) mutex.

Similar logs can be found in work process traces that spin on getting EM2 related mutex via Em2MtxLock because being blocked by some work processes sticking at operating system call mmap().

• Work processes waiting for mutex "EM2_CTX_":

…

I  *** WARNING =>  MtxLock 7f4ec5116b30 EM2_CTX_ owner=ABAP_WP_59 deadlock ? [mtSID.c      715]

M  ------------------ C-STACK ----------------------

[…]

dw.sapSID_D00[S](Em2MtxLock+0x1d)[0x5562f5e2b79f,0x1e7b79f]

dw.sapSID_D00[S](Em2CtxGetFreeSlot+0x5c6)[0x5562f5e2ec10,0x1e7ec10]

dw.sapSID_D00[S](Em2CtxTemplateRestore+0x22a)[0x5562f5e2ef66,0x1e7ef66]

dw.sapSID_D00[S](ThJCloneVm(DP_VMC_TYPE)+0xb69)[0x5562f46193b0,0x6693b0]

dw.sapSID_D00[S](ThJCreateContext(DP_VMC_TYPE, DP_VMC_SUB_TYPE, THJ_CREATE_FOR, TH_VMC_DEBUGGING, char16_t*, unsigned char, DP_VMC_INPUT_INFO*)+0x7b7)[0x5562f461b4a6,0x66b4a6]

dw.sapSID_D00[S](ThJAttachVm2Context(DP_VMC_TYPE, DP_VMC_SUB_TYPE, DP_VMC_INPUT_INFO*, unsigned char, TH_VMC_DEBUGGING, char16_t*, unsigned char*)+0x7b2)[0x5562f461cb21,0x66cb21]

• Work processes sticking at mmap() call and holding mutex "EM2_CTX_":

…

M  *** ERROR => ThVmcICallServer: ThVmcAttachServer failed (-1005) [thxxvmc.c    3811]

M  ------------------ C-STACK ----------------------

[...]

libc.so.6[S](__GI___mmap64+0x45)[0x7f4ebfec6a43,0x111a43]

dw.sapSID_D00[S](Es2MapBlocks+0xa9)[0x5562f5e62825,0x1eb2825]

dw.sapSID_D00[S](Es2CtxAttach+0x925)[0x5562f5e5ee84,0x1eaee84]

dw.sapSID_D00[S](Em2HeapCtxAttach_ES2+0xa2)[0x5562f5e2a89b,0x1e7a89b]

dw.sapSID_D00[S](Em2CtxAttach+0x1f4)[0x5562f5e2e06b,0x1e7e06b]

dw.sapSID_D00[S](ThVmcAttachMem(unsigned char, unsigned char)+0x3af)[0x5562f46690f9,0x6b90f9]

[…]

M  DpWpMutexLocked: W59 inside critical section, csTrack=2

M    mtx info at level 2 = "ES2_CTX_"

M    mtx info at level 1 = "EM2_CTX_"

M  ThInsideCriticalSection: mutex locked 

SAP NetWeaver ABAP system 

VMC, Virtual Machine Container, SM52, Es2MapBlocks, mmap, Es2CtxAttach, Em2MtxLock, EM2(Extended Memory 2nd Generation) , KBA , BC-VMC , Virtual Machine Container , Problem