SAP Knowledge Base Article - Preview

2785814 - HANA high CPU utilization caused by large backup catalog

Symptom

  • High cpu utilization by HANA hdbindexserver process
  • Runtime dumps collected during high cpu are showing e.g:

    671347[thr=97002]: SqlExecutor, TID: 106, UTID: -1, CID: 202084, LCID: 202084, parent: ---, SQLUserName: "SYSTEM", AppUserName: "joe-adm", AppName: "HDBStudio", ConnCtx: 202084 {LDBID: 2, LCID: 202084}, StmtCtx: (1) 867947425905061 {Parent: 202084, MemoryLimit: 0, ObjHdl: 139944950352896, User: "SYSTEM", Schema: "SYSTEM, SesCtx: {ObjHdl: 139007399698944, CID: 202084, LCID: 202084, User: "SYSTEM", Schema: "SYSTEM, Ver: 7, CtxID: 329726}}, type: "SqlExecutor", method: "", detail: "", command: "
    {
    SQL: SELECT TOP 1 t1.BACKUP_ID, ( SELECT sum(t2.BACKUP_SIZE) FROM public.M_BACKUP_CATALOG_FILES t2 WHERE t2.BACKUP_ID = t1.BACKUP_ID GROUP BY t2.BACKUP_ID ) as BBSIZE, t1.UTC_START_TIME, t1.UTC_END_TIME, t1.ENTRY_TYPE_NAME FROM public.M_BACKUP_CATALOG AS t1 WHERE t1.STATE_NAME = 'successful' AND ( t1.ENTRY_TYPE_NAME = 'complete data backup' OR t1.ENTRY_TYPE_NAME = 'data snapshot' ) ORDER BY t1.UTC_START_TIME DESC

    QUERY PLAN:
    |OPERATOR_NAME|OPERATOR_DETAILS|OPERATOR_PROPERTIES|EXECUTION_ENGINE|DATABASE_NAME|SCHEMA_NAME|TABLE_NAME|TABLE_TYPE|TABLE_SIZE|OUTPUT_SIZE|SUBTREE_SIZE|OPERATOR_ID|PARENT_OPERATOR_ID|LEVEL|POSITION|HOST|PORT|BUFFER_ID|
    |ROW SEARCH|M_BACKUP_CATALOG.BACKUP_ID, SUM(SUM(M_BACKUP_CATALOG_FILES.BACKUP_SIZE)), M_BACKUP_CATALOG.UTC_START_TIME, M_BACKUP_CATALOG.UTC_END_TIME, SUBSTR(T0.ENTRY_TYPE_NAME, 0, 64)|
    ....


    RELATED OBJECTS:
    -TABLE:SYS.P_PRIVILEGES_(aaa-bbb-hdb1:32003),SYS.DUMMY(aaa-bbb-hdb1:32003)
    -VIEW:SYS.HAS_NEEDED_SYSTEM_PRIV_INCL_SYS_STAT,SYS.M_BACKUP_CATALOG,SYS.M_BACKUP_CATALOG_FILES
    -SYNONYM:M_BACKUP_CATALOG,M_BACKUP_CATALOG_FILES
    -USER:SYSTEM

    ...
    }..

    " at 0x00007f5cae0ca7f9 in syscall+0x19 (libc.so.6)

  • Large number of entries in backup catalog (HANA_BACKUP_Catalog_Size.txt shows CATALOG_SIZE_MB over 1000 (1Gb)) 
  • Very regular pattern showing cpu spikes e.g. every hour
  • OOM errors showing large allocation e.g : 

    Top "M_HEAP_MEMORY" allocators (component, name, size). Ordered descending by exclusive_size_in_use.
    1: Monitoring & Statistical Data: Pool/RowEngine/MonitorView/StatisticsMonitors/M_DEV_BACKUP_CATALOG_LOG 51.41gb (55202974208b)
    2: Monitoring & Statistical Data: Pool/RowEngine/MonitorView/StatisticsMonitors/M_DEV_BACKUP_CATALOG_LOG_FILES 29.66gb (31851651328b)




Read more...

Environment

SAP HANA Enterprise Edition 1.0

Product

SAP HANA, platform edition all versions

Keywords

9aaf8105bdad5308ab706eea28078d28,nameserver,oom , KBA , HAN-DB-BAC , SAP HANA Backup & Recovery , HAN-DB-PERF , SAP HANA Database Performance , Problem

About this page

This is a preview of a SAP Knowledge Base Article. Click more to access the full version on SAP for Me (Login required).

Search for additional results

Visit SAP Support Portal's SAP Notes and KBA Search.