SAP Knowledge Base Article - Preview

3249362 - Work processes hung in 'Sequential Read of DB' when querying HANA DB Monitoring views

Symptom

  • The work processes of a Netweaver system show sessions stuck in a "Sequential Read of DB" state where the request is made to a HANA Monitoring view like, M_LANDSCAPE_HOST_CONFIGURATION, M_DATABASE etc

SM50 

Time                           01.09.2022, 02:55:50 
Total Number of Work Processes        16
Dialog                         10 / 7 (Total/Free) , Average Load = ( 1,110 / 0,235 / 0,082)
Update                                 1 /         1 (Total/Free)
Background                     3 / 0 (Total/Free) , Average Load = ( 3,000 / 2,196 / 1,888)
Spool                                  1 /         1 (Total/Free)
Update Task 2                          1 /         1 (Total/Free)
Configurable                          16
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
|  Number|Type| Process ID|Status |"On Hold"|Failures|LockedSem.|Requ.Sem.|CPU Time|                           Time|Priority|Executed Program                |Client|User ID|Current Action       |Action Info                   |
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
|      0 |DIA |17.104.898 |Waiting|         |        |          |         |00:00:09|                               |        |                                |      |       |                     |                              |
|      1 |DIA |15.401.138 |Running|         |        |          |         |00:00:07|                             39|High    |SAPLSDIFRUNTIME                 |120   |SAP_OSS|Sequential Read of DB|M_DATABASE
|      2 |DIA |16.122.028 |Waiting|         |        |          |         |00:00:07|                               |        |                                |      |       |                     |                              |
|      3 |DIA |20.775.060 |Waiting|         |        |          |         |00:00:06|                               |        |                                |      |       |                     |                              |
|      4 |DIA |11.599.928 |Waiting|         |        |          |         |00:00:13|                               |        |                                |      |       |                     |                              |
|      5 |DIA |11.862.178 |Waiting|         |        |          |         |00:00:11|                               |        |                                |      |       |                     |                              |
|      6 |DIA |17.694.776 |Running|         |        |          |         |00:00:55|                               |High    |CL_SERVER_INFO================CP|120   |SAP_OSS|                     |                              |
|      7 |DIA |20.250.826 |Waiting|         |        |          |         |00:00:22|                               |        |                                |      |       |                     |                              |
|      8 |DIA |16.187.438 |Running|         |        |          |         |00:00:37|                             19|High    |SAPLSDBADIAG                    |120   |SAP_OSS|Sequential Read of DB|M_LANDSCAPE_HOST_CONFIGURATION|
|      9 |DIA |19.791.968 |Waiting|         |        |          |         |00:00:51|                               |        |                                |      |       |                     |                              |
|     10 |UPD |12.189.932 |Waiting|         |        |          |         |00:00:02|                               |        |                                |      |       |                     |                              |
|     11 |BTC |15.007.968 |Running|         |        |          |         |00:00:33|                           3354|        |CL_HDB_DBCON==================CP|000   |DDIC   |Sequential Read of DB|M_LANDSCAPE_HOST_CONFIGURATION|
|     12 |BTC |15.663.258 |Running|         |        |          |         |00:00:31|                            116|        |RSDBSPJS                        |000   |DDIC   |Sequential Read of DB|M_DATABASE
|     13 |BTC |14.549.234 |Running|         |        |          |         |00:00:26|                           5003|        |CL_SQL_STATEMENT==============CP|000   |DDIC   |Sequential Read of DB|                              |
|     14 |SPO |18.350.276 |Waiting|         |        |          |         |00:00:07|                               |        |                                |      |       |                     |                              |
|     15 |UPD2|21.627.014 |Waiting|         |        |          |         |00:00:02|                               |        |                                |      |       |                     |                              |
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 




=> Stack Trace from the call in the work process dev_w<num> trace :


 __fd_poll() at 0x900000000153a54
 poll__FPvUll() at 0x900000019361a30
 receive__Q2_7Network18SimpleClientSocketFPvUlbRl() at 0x9000000193613a0
 receive__Q3_13Communication3Raw12ClientSocketFPvUlRl() at 0x900000019370314
 receiveBuffer__Q2_6SQLDBC19SocketCommunicationFPUcUlRQ2_7support9StopWatch() at 0
 receiveHeader__Q2_6SQLDBC19SocketCommunicationFRlRQ3_13Communication8Protocol33Pa
 receive__Q2_6SQLDBC19SocketCommunicationFPPvUlRUlUiRlRQ2_4lttc9allocator() at 0x9
 receive__Q2_6SQLDBC13ClientRuntimeFRCQ2_4lttc9smart_ptrXTQ3_6SQLDBC7Runtime19Comm
 receive__Q2_6SQLDBC25SingleThreadClientRuntimeFRCQ2_4lttc9smart_ptrXTQ3_6SQLDBC7R
 receive__Q2_6SQLDBC18PhysicalConnectionFPPvRUlT2RQ2_4lttc9allocatorRQ3_6SQLDBC7Ru
 doReceive__Q2_6SQLDBC10ConnectionFRQ2_6SQLDBC18PhysicalConnectionPPvRUlRQ3_6SQLDB
 sqlaexecute__Q2_6SQLDBC10ConnectionFRQ2_4lttc10shared_ptrXTQ2_6SQLDBC18PhysicalCo
 sqlaexecute__Q2_6SQLDBC10ConnectionFiRQ2_6SQLDBC13RequestPacketRQ2_6SQLDBC11Reply
 sendCommand__Q2_6SQLDBC9StatementFiRQ2_6SQLDBC13RequestPacketRCQ2_6SQLDBC13Encode
 sendSQL__Q2_6SQLDBC9StatementFiRCQ2_6SQLDBC13EncodedStringRQ2_6SQLDBC11ReplyPacke
 execute__Q2_6SQLDBC9StatementFRCQ2_6SQLDBC13EncodedStringbN22() at 0x900000018c0b
 execute__Q2_6SQLDBC9StatementFPCcLQ2_25SQLDBC_StringEncodingType8EncodingbT4() at
 execute__Q2_6SQLDBC16SQLDBC_StatementFPCcLQ2_25SQLDBC_StringEncodingType8Encoding
 open__16CSdbStaticCursorFPCUsT1() at 0x9000000178c75d8
 getActualConnection__Q2_7DBSLHDB15CSdbHDBDbslStmtFv() at 0x9000000179114c4
 updateHostInfomation__Q2_7DBSLHDB17CSdbHDBConnectionFv() at 0x900000017911328
 DbSlHDBControl__Fs12DBSL_CMDTYPEPv() at 0x90000001778172c
 DbSlControl() at 0x1022b42c8
 db_describe2__Fb@OL@729() at 0x101f85bc4
 IPRA.$db_describe2__Fb() at 0x101fb96e0
 IPRA.$@60@dbinfo__Fv() at 0x101fb8bcc
 dbac_function() at 0x101f95948
 ab_jcaly__Fv() at 0x105389fd4
 ab_extri__Fv() at 0x10152fcd8




 

  

  • Indexserver traces shows connection time outs during communication with the nameserver when trying to retrieve landscape host configuration information like 'NAMESERVER_CONFIG_ROLE' & 'NAMESERVER_ACTUAL_ROLE' :

indexserver_alert_hostname_
------------------------------------------

[143337]{-1}[-1/-1] 2022-09-01 02:54:47.289743 e TrexNet          EndPoint.cpp(00255) : ERROR: failed to open channel 127.0.0.1:30001! reason: (internal error)
details: Internal Error Details. Basis/IO/Stream/impl/NetworkChannel.cpp:2513: Connect timed out after 180000 us (-1 means infinite). channel={<NetworkChannel>={<NetworkChannelBase>={this=562909455090712, fd=117, refCnt=1, local=127.0.0.1/18642_tcp, remote=127.0.0.1/30001_tcp, state=ConnectWait, pending=[----]}}}; $Context$=[bf24e074f6023bbf,127.0.0.1:18642,127.0.0.1:30001,TRN,0]
[143337]{-1}[-1/-1] 2022-09-01 02:54:47.290091 e TNS              TNSInfo.cpp(00553) : sendRequest getLicenseValidUntilForProduct to Hostname:30001 failed with NetException. data=(S)productName=SAP-HANA-EPM|
[143337]{-1}[-1/-1] 2022-09-01 02:54:47.290105 e TNS              TNSInfo.cpp(00426) : nameserver Hostname:30001 not responding. retry in 5 sec...
[143334]\{300140\}[33/62702249] 2022-09-01 02:55:17.554790 e commlib          commlibImpl.cpp(00960) : ERROR: comm::connect to Host: 127.0.0.1, port: 30001, Error: exception  1: no.2110018  (Basis/IO/Stream/impl/NetworkChannel.cpp:2513) TID: 143334
    Connect timed out after 180000 us (-1 means infinite). channel={<NetworkChannel>={<NetworkChannelBase>={this=562917885443608, fd=47, refCnt=1, local=127.0.0.1/56958_tcp, remote=127.0.0.1/30001_tcp, state=ConnectWait, pending=[----]}}}; $Context$=[bf24e074f6023bb0,127.0.0.1:56958,127.0.0.1:30001,TRN,0]
exception throw location:
 1: 0x00007fff7a4f6d1c in .LTHUNK33.lto_priv.1+0x3ac at NetworkChannel.cpp:2513 (libhdbbasis.so)
 2: 0x00007fff7a4cc4fc in Stream::NetworkChannelManager::connect(NetworkAccess::NetworkAddress const*, NetworkAccess::NetworkAddress const&, Stream::NetworkChannelParameters const&, bool, ltt::smartptr_handle<Stream::ChannelCallback>*, int, Stream::NetworkChannelComponent)+0x40c at NetworkChannelManager.cpp:190 (libhdbbasis.so)
 3: 0x00007fff7a4ccab4 in Stream::NetworkChannelManager::connect(NetworkAccess::NetworkAddress const*, NetworkAccess::NetworkAddress const&, Stream::NetworkChannelParameters const&, ltt::smartptr_handle<Stream::ChannelCallback>*, int, Stream::NetworkChannelComponent)+0x34 at NetworkChannelManager.cpp:128 (libhdbbasis.so)
 4: 0x00007fff7f609a34 in comm::connect(void*, char const*, unsigned short, int, Crypto::Configuration*, Stream::NetworkChannelComponent)+0x2e4 at commlibImpl.cpp:948 (libhdbbasement.so)
 5: 0x00007fff7f4d2970 in IANATimezones.cpp.e80da39b+0x15b at Channel.cpp:355 (libhdbbasement.so)
 6: 0x00007fff7f4c5c7c in TrexNet::ServerRep::openNewChannel(char const*, char const*, ltt::smartptr_handle<Crypto::Configuration>&)+0xcc at EndPoint.cpp:238 (libhdbbasement.so)
 7: 0x00007fff7f4ac7f4 in TrexNet::Requestor::getChannel(char const*, char const*, unsigned short, char, ltt::smartptr_handle<Crypto::Configuration>&)+0x124 at Requestor.cpp:168 (libhdbbasement.so)
 8: 0x00007fff7f4ad680 in TrexNet::Request::Request(char const*, TRexUtils::HostAndPort const&, char, ltt::smartptr_handle<Crypto::Configuration>)+0x2c0 at Request.cpp:556 (libhdbbasement.so)
 9: 0x00007fff7f4ae240 in TrexNet::Request::Request(char const*, char const*, unsigned short, char)+0x100 at Request.cpp:508 (libhdbbasement.so)
10: 0x00007fff7effd96c in NameServer::TNSInfo::sendRequestTo(NameServer::Request const&, NameServer::Response&, ltt_adp::basic_string<char, ltt::char_traits<char>, ltt::integral_constant<bool, true> > const&, unsigned short, NameServer::TrexNetRequestHolder*, bool, char)+0x11c at TNSInfo.cpp:515 (libhdbbasement.so)
11: 0x00007fff7eff9024 in NameServer::TNSInfo::processRequest(NameServer::Request const&, NameServer::Response&)+0x214 at TNSInfo.cpp:397 (libhdbbasement.so)
12: 0x00007fff7f0824c4 in NameServer::TNSClient::processRequest(NameServer::Request const&, NameServer::Response&)+0x44 at TNSClient.cpp:508 (libhdbbasement.so)
13: 0x00007fff7f07f184 in NameServer::TNSClient::queryTree(NameServer::TNode const&, NameServer::TNode&, NameServer::QueryMode)+0x134 at TNSClient.cpp:573 (libhdbbasement.so)
14: 0x00007fff7f028fc0 in NameServer::TNSClient::getLandscapeConfiguration(ltt_adp::vector<NameServer::LandscapeConfiguration::Host, ltt::integral_constant<bool, true> >&, unsigned int, bool)+0x2e10 at TNSClient.cpp:7238 (libhdbbasement.so)
15: 0x00007fff7f024060 in NameServer::TNSClient::getLandscapeConfiguration(ltt_adp::vector<NameServer::LandscapeConfiguration::Host, ltt::integral_constant<bool, true> >&, bool)+0x60 at TNSClient.cpp:7179 (libhdbbasement.so)
16: 0x00007fff7f023f8c in NameServer::TNSClient::getLandscapeConfiguration(ltt_adp::vector<NameServer::LandscapeConfiguration::Host, ltt::integral_constant<bool, true> >&)+0x2c at TNSClient.cpp:7188 (libhdbbasement.so)
17: 0x00007fff87b75d2c in ptime::LandscapeHostConfigurationMonitor::create_objects(ptime::Monitor::ListEntry*, ptime::mo_Env*, ptime::fastvector<ptime::Monitor::PredicateInfo, 16ul> const*) [clone .constprop.0]+0xac at LandscapeHostConfigurationMonitor.cc:91 (libhdbrskernel.so)
18: 0x00007fff8acdf26c in ptime::LandscapeHostConfigurationMonitorHandle::create_objects(ptime::mo_Env*, ptime::Monitor::ListEntry*, ptime::fastvector<bool, 16ul> const&, ptime::Monitor::LimitInfo const*)+0x1c at LandscapeHostConfigurationMonitor.h:76 (libhdbrskernel.so)
19: 0x00007fff8a4e7e2c in ptime::Monitor_scan::do_open(ptime::OperatorEnv&, ptime::QEParams, int) const+0x4cc at qe_monitor_scan.cc:90 (libhdbrskernel.so)
20: 0x00007fff8a5a5d00 in ptime::Table::open(ptime::Env&, ptime::QEParams, int) const+0x230 at qe_table.cc:233 (libhdbrskernel.so)
21: 0x00007fff8a55b8a8 in ptime::Table::open_child(ptime::Env&, ptime::Table const*, ptime::QEParams&, int, bool) const+0x68 at qe_base.h:646 (libhdbrskernel.so)
22: 0x00007fff8a561064 in ptime::Limit::do_open(ptime::OperatorEnv&, ptime::QEParams, int) const+0x134 at qe_operator_env.h:175 (libhdbrskernel.so)
23: 0x00007fff8a5a5d00 in ptime::Table::open(ptime::Env&, ptime::QEParams, int) const+0x230 at qe_table.cc:233 (libhdbrskernel.so)
24: 0x00007fff8a55b8a8 in ptime::Table::open_child(ptime::Env&, ptime::Table const*, ptime::QEParams&, int, bool) const+0x68 at qe_base.h:646 (libhdbrskernel.so)
25: 0x00007fff8a560e58 in ptime::Project_copy::do_open(ptime::OperatorEnv&, ptime::QEParams, int) const+0x3a8 at qe_project.cc:158 (libhdbrskernel.so)
26: 0x00007fff8a5a5d00 in ptime::Table::open(ptime::Env&, ptime::QEParams, int) const+0x230 at qe_table.cc:233 (libhdbrskernel.so)
exception type information:
 - 0: public ltt::exception@0x7fff79dff188
[143334]\{300140\}[33/62702249] 2022-09-01 02:55:17.622185 e TrexNet          EndPoint.cpp(00255) : ERROR: failed to open channel 127.0.0.1:30001! reason: (internal error)
details: Internal Error Details. Basis/IO/Stream/impl/NetworkChannel.cpp:2513: Connect timed out after 180000 us (-1 means infinite). channel={<NetworkChannel>={<NetworkChannelBase>={this=562917885443608, fd=47, refCnt=1, local=127.0.0.1/56958_tcp, remote=127.0.0.1/30001_tcp, state=ConnectWait, pending=[----]}}}; $Context$=[bf24e074f6023bb0,127.0.0.1:56958,127.0.0.1:30001,TRN,0]
[143334]\{300140\}[33/62702249] 2022-09-01 02:55:17.622560 e TNS              TNSInfo.cpp(00553) : sendRequest querytree to Hostname:30001 failed with NetException. data=(I)mode=0|(N)query=/.int.failover/Hostname|
[143334]\{300140\}[33/62702249] 2022-09-01 02:55:17.622580 e TNS              TNSInfo.cpp(00426) : nameserver Hostname:30001 not responding. retry in 5 sec...
[10939]\{325578\}[29/-1] 2022-09-01 02:55:31.880525 e commlib          commlibImpl.cpp(00960) : ERROR: comm::connect to Host: 127.0.0.1, port: 30001, Error: exception  1: no.2110018  (Basis/IO/Stream/impl/NetworkChannel.cpp:2513) TID: 10939
    Connect timed out after 180000 us (-1 means infinite). channel={<NetworkChannel>={<NetworkChannelBase>={this=562919911832600, fd=34, refCnt=1, local=127.0.0.1/65376_tcp, remote=127.0.0.1/30001_tcp, state=ConnectWait, pending=[----]}}}; $Context$=[bf24e074f6023bb1,127.0.0.1:65376,127.0.0.1:30001,TRN,0]
exception throw location:




 

[30115]\{364521\}[55/-1] 2022-09-04 20:50:02.900412 e TrexNet          EndPoint.cpp(00255) : ERROR: failed to open channel 127.0.0.1:30001! reason: (internal error)
details: Internal Error Details. Basis/IO/Stream/impl/NetworkChannel.cpp:2513: Connect timed out after 180000 us (-1 means infinite). channel={<NetworkChannel>={<NetworkChannelBase>={this=562925314763288, fd=135, refCnt=1, local=127.0.0.1/40404_tcp, remote=127.0.0.1/30001_tcp, state=ConnectWait, pending=[----]}}}; $Context$=[bf24e074f602945c,127.0.0.1:40404,127.0.0.1:30001,TRN,0]
[30115]\{364521\}[55/-1] 2022-09-04 20:50:02.900795 e TNS              TNSInfo.cpp(00553) : sendRequest querytree to host:30001 failed with NetException. data=(I)mode=0|(N)query=/.int.failover/host:30001|
[30115]\{364521\}[55/-1] 2022-09-04 20:50:02.900815 e TNS              TNSInfo.cpp(00426) : nameserver host:30001 not responding. retry in 5 sec...
[143345]{-1}[-1/-1] 2022-09-04 20:50:13.202907 i SQLSessionCmd    sm_watchdog.cc(00277) : canceled session still not finished (cancellation might be still on-going): request_id=844437817553080, trans_id=55, update_trans_id=0, sql_string=SELECT TOP 1 D.DATABASE_NAME || '/' || H.VALUE, L.HOST, CURRENT_USER, CURRENT_SCHEMA, CURRENT_CONNECTION FROM M_DATABASE D, SYS.M_HOST_INFORMATION H, SYS.M_LANDSCAPE_HOST_CONFIGURATION L WHERE H.KEY = 'sapsystem' AND L.INDEXSERVER_ACTUAL_ROLE IN ('MASTER', 'COORDINATOR') , statement_hash=8cbf5a58cd04fc0b21dcc3ad5fc74b06, user_name=SAPSR3, conn_id=364521, thread_id=30115, app_user_name=sidadm, app_name=ABAP:SID, app_source=




 

[143373]\{200067\}[15/-1] 2022-09-02 04:39:37.646424 e TrexNet          EndPoint.cpp(00255) : ERROR: failed to open channel 127.0.0.1:30001! reason: (internal error)
details: Internal Error Details. Basis/IO/Stream/impl/NetworkChannel.cpp:2513: Connect timed out after 180000 us (-1 means infinite). channel={<NetworkChannel>={<NetworkChannelBase>={this=562927520491032, fd=120, refCnt=1, local=127.0.0.1/13514_tcp, remote=127.0.0.1/30001_tcp, state=ConnectWait, pending=[----]}}}; $Context$=[bf24e074f60256eb,127.0.0.1:13514,127.0.0.1:30001,TRN,0]
[143373]\{200067\}[15/-1] 2022-09-02 04:39:37.646838 e TNS              TNSInfo.cpp(00553) : sendRequest getTopologyAsString to host:30001 failed with NetException. data=(I)databaseId=3|
[143373]\{200067\}[15/-1] 2022-09-02 04:39:37.646856 e TNS              TNSInfo.cpp(00426) : nameserver host:30001 not responding. retry in 5 sec...
[115292]\{335962\}[53/-1] 2022-09-02 04:39:52.220943 e commlib          commlibImpl.cpp(00960) : ERROR: comm::connect to Host: 127.0.0.1, port: 30001, Error: exception  1: no.2110018  (Basis/IO/Stream/impl/NetworkChannel.cpp:2513) TID: 115292
    Connect timed out after 180000 us (-1 means infinite). channel={<NetworkChannel>={<NetworkChannelBase>={this=562910603979288, fd=37, refCnt=1, local=127.0.0.1/51526_tcp, remote=127.0.0.1/30001_tcp, state=ConnectWait, pending=[----]}}}; $Context$=[bf24e074f60256ec,127.0.0.1:51526,127.0.0.1:30001,TRN,0]



Read more...

Environment

SAP HANA, platform edition 2.0

SAP HANA, platform edition 1.0

SAP Netweaver

Product

SAP HANA 1.0, platform edition ; SAP HANA, platform edition 2.0 ; SAP NetWeaver all versions

Keywords

KBA , HAN-DB , SAP HANA Database , Problem

About this page

This is a preview of a SAP Knowledge Base Article. Click more to access the full version on SAP for Me (Login required).

Search for additional results

Visit SAP Support Portal's SAP Notes and KBA Search.