Symptom
- Performance degradation (high thread workload / increased CPU consumption) and/or long running transactions are observed on the HANA DB.
Further analysis on the thread callstacks & thread samples reveal a lot of jobworker threads for transaction(s) are processing in a "Network Poll" state.
Callstacks and thread samples may look something similar to this :
24157447[thr=11486]: JobWrk0384 at
1: 0x00007f04b1ba655d in __poll_nocancel+0x20 (libc.so.6)
2: 0x00007f04b3e4d410 in Stream::CompletionSet::pollImpl(int)+0xd0 at CompletionSet.cpp:115 (libhdbbasis.so)
3: 0x00007f04b3e4ca0a in Stream::CompletionSet::poll(int)+0x116 at CompletionSet.cpp:450 (libhdbbasis.so)
4: 0x00007f04b3e43492 in .LTHUNK24.lto_priv.4+0x2f0 at NetworkChannel.cpp:2005 (libhdbbasis.so)
5: 0x00007f04b94b6b25 in comm::read(void*, void*, int, int)+0x81 at commlibImpl.cpp:1209 (libhdbbasement.so)
6: 0x00007f04b93b4914 in TrexNet::Channel::receive(void*, int, int&)+0x90 at Channel.cpp:482 (libhdbbasement.so)
7: 0x00007f04b93c8f4c in TrexNet::NetInput::peek()+0xe8 at BufferedIO.cpp:761 (libhdbbasement.so)
8: 0x00007f04b93c9760 in TrexNet::NetInput::checkType(unsigned char)+0x60 at BufferedIO.cpp:3795 (libhdbbasement.so)
9: 0x00007f04b93f8d1c in TrexNet::NetInput::operator>>(int&)+0x28 at BufferedIO.cpp:2946 (libhdbbasement.so)
10: 0x00007f04c0c75a61 in AdapterService::AdapterManager::readMessage(DPFramework::Message*, TrexNet::Request*, bool)+0x30 at AdapterManager.cpp:4522 (libhdbrskernel.so)
11: 0x00007f04c0c77fb1 in AdapterService::AdapterManager::sendMessage(DPFramework::Message*, ltt::basic_string<char, ltt::char_traits<char>, ltt::integral_constant<int, 64> >&, bool*, bool, bool)+0x3e0 at AdapterManager.cpp:4476 (libhdbrskernel.so)
12: 0x00007f04c0c7fda9 in AdapterService::AdapterManager::getAgentProtocolVersion(ltt::basic_string<char, ltt::char_traits<char>, ltt::integral_constant<int, 64> >&, DPFramework::Message::ProtocolVersion&, ltt::basic_string<char, ltt::char_traits<char>, ltt::integral_constant<int, 64> >&)+0x165 at AdapterManager.cpp:2402 (libhdbrskernel.so)
13: 0x00007f04c0c97244 in AdapterService::AdapterManager::getAgent(long, ltt::basic_string<char, ltt::char_traits<char>, ltt::integral_constant<int, 64> >&, ltt::basic_string<char, ltt::char_traits<char>, ltt::integral_constant<int, 64> >&, ltt::basic_string<char, ltt::char_traits<char>, ltt::integral_constant<int, 64> >&, ltt::basic_string<char, ltt::char_traits<char>, ltt::integral_constant<int, 64> >&, ltt::basic_string<char, ltt::char_traits<char>, ltt::integral_constant<int, 64> >&, DPFramework::Message::ProtocolVersion&, bool, bool)+0x490 at AdapterManager.cpp:489 (libhdbrskernel.so)
. . . . . . .
- 'THREAD_STATE ' column in the '_SYS_STATISTICS.HOST_SERVICE_THREAD_SAMPLES'
THREAD_METHOD THREAD_STATE THREAD_TYPE TIMESTAMP TRANSACTION_ID UPDATE_TRANSACTION_ID USER_NAME
PartitionUpdateJobNode Network Poll JobWorker 20221004135953891553000 2,165 4,228,762,215 SAPSR3
PartitionUpdateJobNode Network Poll JobWorker 20221004135953891553000 2,165 4,228,762,215 SAPSR3
PartitionUpdateJobNode Network Poll JobWorker 20221004135953891553000 1,433 4,228,762,410 SAPSR3
PartitionUpdateJobNode Network Poll JobWorker 20221004135953891553000 292 4,228,762,221 SAPSR3
PartitionUpdateJobNode Network Poll JobWorker 20221004135953891553000 1,008 4,228,762,216 SAPSR3
- 'HANA_SQL_StatementHash_DataCollector_' trace :
| |
|****************** |
|* THREAD SAMPLES * |
|****************** |
| |
|SAMPLES PERCENT HOST PORT THREAD_TYPE |
|======= ======= ======= ===== ============================== |
| 28114 27.53 host 33003 JobWorker |
| 24546 24.03 host 33003 Request |
| 24324 23.82 host 33003 JobWorker |
| 17070 16.71 host 33003 Request |
| 4479 4.38 host 33003 SqlExecutor |
| 3582 3.50 host 33003 SqlExecutor |
| |
|SAMPLES PERCENT THREAD_STATE LOCK_NAME |
|======= ======= =================== ====================================================================== |
| 51921 50.84 Network Poll ? |
| 38704 37.90 Running ? |
| 8051 7.88 Job Exec Waiting JobBarrier Wait for Jobs |
| 1957 1.91 ExclusiveLock Enter RWLock[query_plan_cache.cc:221]@0x00007f0284ed7324: ptime::Query::PlanCache::Manager::Manager |
| 1034 1.01 ExclusiveLock Enter RWLock[query_plan_cache.cc:221]@0x00007f3fa560b324: ptime::Query::PlanCache::Manager::Manager |
| 438 0.42 Mutex Wait unnamed Mutex |
| 3 0.00 ExclusiveLock Enter BTree GuardContainer |
| 2 0.00 Mutex Wait subtransHandleMutex |
| 2 0.00 Semaphore Wait SleepSemaphore |
| 1 0.00 Mutex Wait Transaction object latch |
| 1 0.00 Network Read ? |
| 1 0.00 Sleeping ? |
| |
|SAMPLES PERCENT THREAD_TYPE THREAD_METHOD |
|======= ======= =========== ================================================== |
| 51994 50.91 JobWorker PartitionUpdateJobNode |
| 41613 40.75 Request __queryRequestHandler |
| 8061 7.89 SqlExecutor ExecutePrepared |
| 413 0.40 JobWorker PrepareStatement |
| 31 0.03 JobWorker AttributeIndexJob |
| 3 0.00 Request ? |
| |
Read more...
Environment
SAP HANA platform edition 2.0
SAP HANA platform edition 1.0
Product
SAP HANA 1.0, platform edition ; SAP HANA, platform edition 2.0
Keywords
KBA , HAN-DB , SAP HANA Database , HAN-DB-PERF , SAP HANA Database Performance , Problem
About this page
This is a preview of a SAP Knowledge Base Article. Click more to access the full version on SAP for Me (Login required).Search for additional results
Visit SAP Support Portal's SAP Notes and KBA Search.