SAP Knowledge Base Article - Preview

3338233 - "[2982] A communication error occurred, with the HDB TcpIp Server" dump occurs for transactions executed on a HANA DB Scale Out Environment

Symptom

  • Dumps like below occur in the ABAP system, where the HANA Database platform is on a distributed Scale Out Environment with multiple nodes :
Category               Installation error
Runtime Errors         DBSQL_SQL_ERROR
Except.                CX_SY_OPEN_SQL_DB
Date and Time          03/16/2023 06:42:42    

----------------------------------------------------------------------------------------------------
|Short Text                                                                                        |
|    SQL error "SQL code: 2048" occurred while accessing table "/BIC/AGWMD00501".                  |
----------------------------------------------------------------------------------------------------

----------------------------------------------------------------------------------------------------
|What happened?                                                                                    |
|    Database error text: "SQL message: column store error: search table error:                    |
|     [2982] A communication error occurred, with the HDB TcpIp Server"                            |
----------------------------------------------------------------------------------------------------



.....

  

  • Master node host210 is the db server host ("Database server") from where the transaction was initiated and dumped in ST22 :
----------------------------------------------------------------------------------------------------
|System environment                                                                                |
|    SAP Release..... 750                                                                          |
|    SAP Basis level 0023                                                                          |
|                                                                                                  |
|    Application server... host241                                                            |
|    Network address...... <>                                                        |
|    Operating system... Linux                                                                     |
|    Release.............. 4.18.0-305.19.1.el8_                                                    |
|    Hardware type....... x86_64                                                                   |
|    Character length..... 16 Bits                                                                 |
|    Pointer length........ 64 Bits                                                                |
|    Work process number... 395                                                                    |
|    Shortdump setting. full                                                                       |
|                                                                                                  |
|    Database server... host210                                                               |
|    Database type..... HDB                                                                        |
|    Database name..... SID                                                                        |
|    Database user ID SAPHDB                                                                       |
|                                                                                                  |
|    Terminal..........                                                                            |
|                                                                                                  |
|    Character set C                                                                               |
|                                                                                                  |
|    SAP kernel....... 753                                                                         |
|    Created on....... Mar  3 2022 15:19:26                                                        |
|    Created at....... Linux GNU SLES-11 x86_64 cc4.8.5 use-pr220303                               |
|    Database version SQLDBC 2.12.025.1651858745                                                   |
|    Patch level....... 1000                                                                       |
|    Patch text.......                                                                             |
|                                                                                                  |
|    Database............. HANA 1.0, HANA 2.0                                                      |
|    SAP database version. 753                                                                     |
|    Operating system... Linux                                                                     |
|                                                                                                  |
|    Memory consumption                                                                            |
|    Roll.... 0                                                                                    |
|    EM...... 0                                                                                    |
|    Heap.... 74739968                                                                             |
|    Page.... 24576                                                                                |
|    MM used. 70417784                                                                             |
|    MM free. 805600                                                                               |
----------------------------------------------------------------------------------------------------

 

  • Transaction information id information is displayed here :
----------------------------------------------------------------------------------------------------
|User and Transaction                                                                              |
|    Client................. 400                                                                   |
|    User.................. USER                                                              |
|    Language key.......... E                                                                      |
|    Transaction.........                                                                          |
|    Transaction ID...... 1093697C2A1B18B0E00646320B58713D                                         |
|                                                                                                  |
|    EPP whole context ID...... 005056AB558D1EDC93AD4573FA2FBE8E                                   |
|    EPP connection ID........ 00000000000000000000000000000000                                    |
|    EPP call counter......... 0                                                                   |
|                                                                                                  |
|    Program.............. GP00O2TO0R8PTIKWKPJ49AB57MD                                             |
|    Screen.............. SAPMSSY0                                1000                             |
|    Screen line.......... 6                                                                       |
|    Active debugger..... "none"                                                                   |
----------------------------------------------------------------------------------------------------

----------------------------------------------------------------------------------------------------
|Information on where terminated                                                                   |
|    The termination occurred in ABAP program or include                                           |
|     "GP00O2TO0R8PTIKWKPJ49AB57MD", in "F5000_GET_AVAIL_CAPACITY". The                            |
|    main program was "RSBATCH_EXECUTE_PROZESS".                                                   |
|                                                                                                  |
|    In the source code, the termination point is in line 966 of  program or                       |
|    include "Z_CAPACITY_FORMS_5".                                                                 |
|    Program "GP00O2TO0R8PTIKWKPJ49AB57MD" was started as a background job.                        |
|                                                                                                  |
|    Job name....... BIDTPR_20230516104526000017_1                                                 |
|    Job initiator.. ADMDTLOAD                                                                     |
|    Job number...... 06460501                                                                     |
|    The termination is due to exception "CX_SY_OPEN_SQL_DB" occurring in procedure                |
|     "F5000_GET_AVAIL_CAPACITY" "(FORM)".                                                         |
|    This exception was not handled locally or declared in the RAISING                             |
|    clause in the procedure's signature however.                                                  |
|    The procedure is in program "GP00O2TO0R8PTIKWKPJ49AB57MD". The source code                    |
|     begins in line 847 of                                                                        |
|    (include) program "Z_CAPACITY_FORMS_5".                                                       |
----------------------------------------------------------------------------------------------------


 

  • We can see the transaction id information (EppRootContextId, EppTransactionId) from one of the worker nodes "host11" as found in the ABAP dump under 'User and Transaction'.
    Worker node 'indexserver_host211.31003.301.trc' shows the hanging network channel for the distributed transaction between worker nodes 11 & 12 on the addresses '.67' and '.47' from the same subnet : 'local=xx.xxx.xxx.67/60850_tcp', 'remote=xx.xxx.xxx.47/31003_tcp', state=ConnectWait, ...'

[74835]\{563533\}[526/-1] 2023-05-16 06:48:25.377229 i TraceContext     TraceContext.cpp(01347) : UserName=SAPHDB, ApplicationUserName=USER, ApplicationName=ABAP:SID, ApplicationSource=GP00O2TO0R8PTIKWKPJ49AB57MD:1877, Client=400, StatementHash=0b16e1e278db03340bffdbb68c11fce4, EppRootContextId=005056AB558D1EDC93AD4573FA2FBE8E, EppTransactionId=1093697C2A1B18B0E00646320B58713D, EppConnectionId=00000000000000000000000000000000, EppConnectionCounter=0, EppComponentName=KAP/host243_KAP_00, EppAction=BIDTPR_20230516104526000017_1, StatementExecutionID=844447853550869
[74835]\{563533\}[526/-1] 2023-05-16 06:48:25.377229 w commlib          commlibImpl.cpp(00925) : Connection to 'xx.xxx.xxx.47:31003' failed: exception  1: no.2110018  (Basis/IO/Stream/impl/NetworkChannel.cpp:2742) TID: 74835
    Connect timed out after 30000 ms (-1 means infinite). channel={<NetworkChannel>={<NetworkChannelBase>={this=139614566776344, fd=1504, refCnt=1, local=xx.xxx.xxx.67/60850_tcp, remote=xx.xxx.xxx.47/31003_tcp, state=ConnectWait, pending=[----]}}}; $Context$=[93c656e7843c9987,xx.xxx.xxx.67:60850,xx.xxx.xxx.47:31003,TRN,0]
exception throw location:
   0: 0x00007fd8af9121d6 in Stream::NetworkChannel::connectBlocking(int)::{lambda(std::chrono::duration<long, std::ratio<1l, 1000l> >)#1}::operator()(std::chrono::duration<long, std::ratio<1l, 1000l> >) const+0x2a2 at Basis/IO/Stream/impl/NetworkChannel.cpp:2742 (libhdbbasis.so)
   1: 0x00007fd8af9125be in .LTHUNK36.lto_priv.0+0x1fa at Basis/IO/Stream/impl/NetworkChannel.cpp:2805 (libhdbbasis.so)
   2: 0x00007fd8af9233b3 in Stream::NetworkChannelManager::connect(NetworkAccess::NetworkAddress const*, NetworkAccess::NetworkAddress const&, Stream::NetworkChannelParameters const&, bool, ltt::smartptr_handle<Stream::ChannelCallback>*, Stream::NetworkChannelComponent, int, ltt::allocator&)+0x5f0 at Basis/IO/Stream/impl/NetworkChannelManager.cpp:203 (libhdbbasis.so)
   3: 0x00007fd8b05e5ed8 in comm::connect(void*, char const*, unsigned short, int, Crypto::Configuration*, Stream::NetworkChannelComponent)+0x1a4 at commlib/Impl/commlibImpl.cpp:913 (libhdbnetrpc.so)
   4: 0x00007fd8b067a754 in TrexNet::Channel::open(char const*, TrexNet::RequestParameters const&)+0x320 at TrexNet/Channel.cpp:304 (libhdbnetrpc.so)
   5: 0x00007fd8b0696b61 in TrexNet::ServerRep::openNewChannel(char const*, TrexNet::RequestParameters const&)+0x1a0 at TrexNet/EndPoint.cpp:246 (libhdbnetrpc.so)
   6: 0x00007fd8b06b29a8 in TrexNet::Requestor::getChannel(char const*, unsigned short, TrexNet::RequestParameters const&)+0x294 at TrexNet/Requestor.cpp:161 (libhdbnetrpc.so)
   7: 0x00007fd8b06b0b47 in TrexNet::Request::Request<TrexNet::Request::Request(ltt::basic_string_view<char, ltt::char_traits<char> >, Basis::HostAndPort const&, TrexNet::RequestParameters)::{lambda()#1}, void>(ltt::basic_string_view<char, ltt::char_traits<char> >, TrexNet::Request::Request(ltt::basic_string_view<char, ltt::char_traits<char> >, Basis::HostAndPort const&, TrexNet::RequestParameters)::{lambda()#1}&&, TrexNet::RequestParameters) [clone .lto_priv.0]+0x433 at ltt/string.hpp:576 (libhdbnetrpc.so)
   8: 0x00007fd8b06a15ea in TrexNet::Request::Request(ltt::basic_string_view<char, ltt::char_traits<char> >, Basis::HostAndPort const&, TrexNet::RequestParameters)+0xd6 at TrexNet/Request.cpp:525 (libhdbnetrpc.so)
   9: 0x00007fd8b06a2bcc in TrexNet::Request::Request(ltt::basic_string_view<char, ltt::char_traits<char> >, char const*, TrexNet::RequestParameters)+0x218 at TrexNet/Request.cpp:508 (libhdbnetrpc.so)
  10: 0x00007fd8cef7c049 in TRexAPI::(anonymous namespace)::RemoteSearchPartImpl::doSearch()+0x165 at ims_search_api/Search/RemotePartSearchImpl.cpp:115 (libhdbcsapi.so)
  11: 0x00007fd8cef7c65d in TRexAPI::(anonymous namespace)::RemoteSearchPartJob::run()+0x9 at ims_search_api/Search/RemotePartSearchImpl.cpp:172 (libhdbcsapi.so)
  12: 0x00007fd8b3a95542 in TRexUtils::Parallel::JobBase::runEx()+0x50 at TRexUtils/ParallelDispatcher.cpp:194 (libhdbbasement.so)
  13: 0x00007fd8b3a8f870 in TRexUtils::Parallel::JobBase::run(Execution::Context&, Execution::JobObject&)+0x40 at Basis/Timer.hpp:88 (libhdbbasement.so)
  14: 0x00007fd8af7d35a0 in Execution::JobObjectImpl::run(Execution::JobWorker*)+0x15b0 at Basis/Execution/impl/JobExecutionLog.hpp:155 (libhdbbasis.so)
  15: 0x00007fd8af7e0b03 in Execution::JobWorker::runJob(ltt::smartptr_handle<Execution::JobObjectForHandle>&)+0x860 at Basis/Execution/impl/JobExecutorThreads.cpp:366 (libhdbbasis.so)
  16: 0x00007fd8af7e26ab in Execution::JobWorker::run(Execution::ThreadRC&)+0x887 at Basis/Execution/impl/JobExecutorThreads.cpp:1354 (libhdbbasis.so)
  17: 0x00007fd8af833da6 in Execution::Thread::staticMainImp(Execution::Thread*)+0x542 at Basis/Execution/impl/Thread.cpp:574 (libhdbbasis.so)
  18: 0x00007fd8af8346e5 in Execution::pthreadFunctionWrapper(Execution::Thread*)+0x1c1 at Basis/Execution/impl/ThreadInterposition.cpp:693 (libhdbbasis.so)
  19: 0x0000000000000000 in <no symbol>+0x0 (<unknown>)
exception type information:
 - 0: public ltt::exception@0x00007fd8aecd08e8
[74835]\{563533\}[526/-1] 2023-05-16 06:48:25.403823 e commlib          commlibError.cpp(00257) : Internal Error Details cannot be saved to TLS: comm::thread_attach was not called on the current thread.
8049322193[thr=74835][parent thr=151232]: JobWrk4772012 at 
   0: 0x00007fd8b06f785f in comm::setLastError(ltt::exception const&, bool) [clone .constprop.0]+0x38b at commlib/Impl/commlibError.cpp:257 (libhdbnetrpc.so)
   1: 0x00007fd8b074aea4 in comm::connect(void*, char const*, unsigned short, int, Crypto::Configuration*, Stream::NetworkChannelComponent) [clone .cold]+0x56 at commlib/Impl/commlibImpl.cpp:926 (libhdbnetrpc.so)
   2: 0x00007fd8b067a754 in TrexNet::Channel::open(char const*, TrexNet::RequestParameters const&)+0x320 at TrexNet/Channel.cpp:304 (libhdbnetrpc.so)
   3: 0x00007fd8b0696b61 in TrexNet::ServerRep::openNewChannel(char const*, TrexNet::RequestParameters const&)+0x1a0 at TrexNet/EndPoint.cpp:246 (libhdbnetrpc.so)
   4: 0x00007fd8b06b29a8 in TrexNet::Requestor::getChannel(char const*, unsigned short, TrexNet::RequestParameters const&)+0x294 at TrexNet/Requestor.cpp:161 (libhdbnetrpc.so)
   5: 0x00007fd8b06b0b47 in TrexNet::Request::Request<TrexNet::Request::Request(ltt::basic_string_view<char, ltt::char_traits<char> >, Basis::HostAndPort const&, TrexNet::RequestParameters)::{lambda()#1}, void>(ltt::basic_string_view<char, ltt::char_traits<char> >, TrexNet::Request::Request(ltt::basic_string_view<char, ltt::char_traits<char> >, Basis::HostAndPort const&, TrexNet::RequestParameters)::{lambda()#1}&&, TrexNet::RequestParameters) [clone .lto_priv.0]+0x433 at ltt/string.hpp:576 (libhdbnetrpc.so)
   6: 0x00007fd8b06a15ea in TrexNet::Request::Request(ltt::basic_string_view<char, ltt::char_traits<char> >, Basis::HostAndPort const&, TrexNet::RequestParameters)+0xd6 at TrexNet/Request.cpp:525 (libhdbnetrpc.so)
   7: 0x00007fd8b06a2bcc in TrexNet::Request::Request(ltt::basic_string_view<char, ltt::char_traits<char> >, char const*, TrexNet::RequestParameters)+0x218 at TrexNet/Request.cpp:508 (libhdbnetrpc.so)
   8: 0x00007fd8cef7c049 in TRexAPI::(anonymous namespace)::RemoteSearchPartImpl::doSearch()+0x165 at ims_search_api/Search/RemotePartSearchImpl.cpp:115 (libhdbcsapi.so)
   9: 0x00007fd8cef7c65d in TRexAPI::(anonymous namespace)::RemoteSearchPartJob::run()+0x9 at ims_search_api/Search/RemotePartSearchImpl.cpp:172 (libhdbcsapi.so)
  10: 0x00007fd8b3a95542 in TRexUtils::Parallel::JobBase::runEx()+0x50 at TRexUtils/ParallelDispatcher.cpp:194 (libhdbbasement.so)
  11: 0x00007fd8b3a8f870 in TRexUtils::Parallel::JobBase::run(Execution::Context&, Execution::JobObject&)+0x40 at Basis/Timer.hpp:88 (libhdbbasement.so)
  12: 0x00007fd8af7d35a0 in Execution::JobObjectImpl::run(Execution::JobWorker*)+0x15b0 at Basis/Execution/impl/JobExecutionLog.hpp:155 (libhdbbasis.so)
  13: 0x00007fd8af7e0b03 in Execution::JobWorker::runJob(ltt::smartptr_handle<Execution::JobObjectForHandle>&)+0x860 at Basis/Execution/impl/JobExecutorThreads.cpp:366 (libhdbbasis.so)
  14: 0x00007fd8af7e26ab in Execution::JobWorker::run(Execution::ThreadRC&)+0x887 at Basis/Execution/impl/JobExecutorThreads.cpp:1354 (libhdbbasis.so)
  15: 0x00007fd8af833da6 in Execution::Thread::staticMainImp(Execution::Thread*)+0x542 at Basis/Execution/impl/Thread.cpp:574 (libhdbbasis.so)
  16: 0x00007fd8af8346e5 in Execution::pthreadFunctionWrapper(Execution::Thread*)+0x1c1 at Basis/Execution/impl/ThreadInterposition.cpp:693 (libhdbbasis.so)
  17: 0x0000000000000000 in <no symbol>+0x0 (<unknown>)
[74835]\{563533\}[526/-1] 2023-05-16 06:48:25.410591 e commlib          commlibError.cpp(00259) : Internal Error Details. exception  1: no.2110018  (Basis/IO/Stream/impl/NetworkChannel.cpp:2742) TID: 74835
    Connect timed out after 30000 ms (-1 means infinite). channel={<NetworkChannel>={<NetworkChannelBase>={this=139614566776344, fd=1504, refCnt=1, local=xx.xxx.xxx.67/60850_tcp, remote=xx.xxx.xxx.47/31003_tcp, state=ConnectWait, pending=[----]}}}; $Context$=[93c656e7843c9987,xx.xxx.xxx.67:60850,xx.xxx.xxx.47:31003,TRN,0]
exception throw location:
   0: 0x00007fd8af9121d6 in Stream::NetworkChannel::connectBlocking(int)::{lambda(std::chrono::duration<long, std::ratio<1l, 1000l> >)#1}::operator()(std::chrono::duration<long, std::ratio<1l, 1000l> >) const+0x2a2 at Basis/IO/Stream/impl/NetworkChannel.cpp:2742 (libhdbbasis.so)
   1: 0x00007fd8af9125be in .LTHUNK36.lto_priv.0+0x1fa at Basis/IO/Stream/impl/NetworkChannel.cpp:2805 (libhdbbasis.so)
   2: 0x00007fd8af9233b3 in Stream::NetworkChannelManager::connect(NetworkAccess::NetworkAddress const*, NetworkAccess::NetworkAddress const&, Stream::NetworkChannelParameters const&, bool, ltt::smartptr_handle<Stream::ChannelCallback>*, Stream::NetworkChannelComponent, int, ltt::allocator&)+0x5f0 at Basis/IO/Stream/impl/NetworkChannelManager.cpp:203 (libhdbbasis.so)
   3: 0x00007fd8b05e5ed8 in comm::connect(void*, char const*, unsigned short, int, Crypto::Configuration*, Stream::NetworkChannelComponent)+0x1a4 at commlib/Impl/commlibImpl.cpp:913 (libhdbnetrpc.so)
   4: 0x00007fd8b067a754 in TrexNet::Channel::open(char const*, TrexNet::RequestParameters const&)+0x320 at TrexNet/Channel.cpp:304 (libhdbnetrpc.so)
   5: 0x00007fd8b0696b61 in TrexNet::ServerRep::openNewChannel(char const*, TrexNet::RequestParameters const&)+0x1a0 at TrexNet/EndPoint.cpp:246 (libhdbnetrpc.so)
   6: 0x00007fd8b06b29a8 in TrexNet::Requestor::getChannel(char const*, unsigned short, TrexNet::RequestParameters const&)+0x294 at TrexNet/Requestor.cpp:161 (libhdbnetrpc.so)
   7: 0x00007fd8b06b0b47 in TrexNet::Request::Request<TrexNet::Request::Request(ltt::basic_string_view<char, ltt::char_traits<char> >, Basis::HostAndPort const&, TrexNet::RequestParameters)::{lambda()#1}, void>(ltt::basic_string_view<char, ltt::char_traits<char> >, TrexNet::Request::Request(ltt::basic_string_view<char, ltt::char_traits<char> >, Basis::HostAndPort const&, TrexNet::RequestParameters)::{lambda()#1}&&, TrexNet::RequestParameters) [clone .lto_priv.0]+0x433 at ltt/string.hpp:576 (libhdbnetrpc.so)
   8: 0x00007fd8b06a15ea in TrexNet::Request::Request(ltt::basic_string_view<char, ltt::char_traits<char> >, Basis::HostAndPort const&, TrexNet::RequestParameters)+0xd6 at TrexNet/Request.cpp:525 (libhdbnetrpc.so)
   9: 0x00007fd8b06a2bcc in TrexNet::Request::Request(ltt::basic_string_view<char, ltt::char_traits<char> >, char const*, TrexNet::RequestParameters)+0x218 at TrexNet/Request.cpp:508 (libhdbnetrpc.so)
  10: 0x00007fd8cef7c049 in TRexAPI::(anonymous namespace)::RemoteSearchPartImpl::doSearch()+0x165 at ims_search_api/Search/RemotePartSearchImpl.cpp:115 (libhdbcsapi.so)
  11: 0x00007fd8cef7c65d in TRexAPI::(anonymous namespace)::RemoteSearchPartJob::run()+0x9 at ims_search_api/Search/RemotePartSearchImpl.cpp:172 (libhdbcsapi.so)
  12: 0x00007fd8b3a95542 in TRexUtils::Parallel::JobBase::runEx()+0x50 at TRexUtils/ParallelDispatcher.cpp:194 (libhdbbasement.so)
  13: 0x00007fd8b3a8f870 in TRexUtils::Parallel::JobBase::run(Execution::Context&, Execution::JobObject&)+0x40 at Basis/Timer.hpp:88 (libhdbbasement.so)
  14: 0x00007fd8af7d35a0 in Execution::JobObjectImpl::run(Execution::JobWorker*)+0x15b0 at Basis/Execution/impl/JobExecutionLog.hpp:155 (libhdbbasis.so)
  15: 0x00007fd8af7e0b03 in Execution::JobWorker::runJob(ltt::smartptr_handle<Execution::JobObjectForHandle>&)+0x860 at Basis/Execution/impl/JobExecutorThreads.cpp:366 (libhdbbasis.so)
  16: 0x00007fd8af7e26ab in Execution::JobWorker::run(Execution::ThreadRC&)+0x887 at Basis/Execution/impl/JobExecutorThreads.cpp:1354 (libhdbbasis.so)
  17: 0x00007fd8af833da6 in Execution::Thread::staticMainImp(Execution::Thread*)+0x542 at Basis/Execution/impl/Thread.cpp:574 (libhdbbasis.so)
  18: 0x00007fd8af8346e5 in Execution::pthreadFunctionWrapper(Execution::Thread*)+0x1c1 at Basis/Execution/impl/ThreadInterposition.cpp:693 (libhdbbasis.so)
  19: 0x0000000000000000 in <no symbol>+0x0 (<unknown>)
exception type information:
 - 0: public ltt::exception@0x00007fd8aecd08e8
[74835]\{563533\}[526/-1] 2023-05-16 06:48:25.420536 w TrexNet          EndPoint.cpp(00268) : failed to open channel to 'xx.xxx.xxx.47:31003&0': (internal error)
details: 
[74835]\{563533\}[526/-1] 2023-05-16 06:48:25.420801 e Oltp_Search      RemotePartSearchImpl.cpp(00139) : Remote part search on host host212:31003 failed due to communication error: exception  1: no.2111001  (TrexNet/EndPoint.cpp:270) TID: 74835
    generic; $TYPE$=OpenChannel; $MESSAGE$=an error occurred while opening the channel; $INFO$=internal error; $PARAM$=xx.xxx.xxx.47:31003&0
exception throw location:

 

  • The other involved worker node 'indexserver_host212.31003.929.trc' shows a read timeout to node 11 :
    
[38573]\{625004\}[706/-1] 2023-05-16 06:47:52.887792 i TraceContext     TraceContext.cpp(01347) : UserName=SAPHDB, ApplicationUserName=USER, ApplicationName=ABAP:KAP, ApplicationSource=GP00O2TO0R8PTIKWKPJ49AB57MD:304, Client=400, StatementHash=0b16e1e278db03340bffdbb68c11fce4, EppRootContextId=005056AB558D1EDC93AD4573FA2FBE8E, EppTransactionId=6FDC7DBDB51B1860E0064632C519B6AA, EppConnectionId=00000000000000000000000000000000, EppConnectionCounter=0, EppComponentName=KAP/host243_KAP_00, EppAction=BIDTPR_20230516104526000017_1, StatementExecutionID=844453046650012
[38573]\{625004\}[706/-1] 2023-05-16 06:47:52.887792 w TrexNet          BufferedIO.cpp(02416) : erroneous channel (since 109mues) 10194701 with request-id 8409680334250871036 {0x00007edba1943f20: id=10194701, channel={<NetworkChannel>={<NetworkChannelBase>={this=139463707317272, fd=1987, refCnt=1, local=xx.xxx.xxx.47/53448_tcp, remote=xx.xxx.xxx.67/31003_tcp, state=Connected, pending=[----]}}}} from 53448 to xx.xxx.xxx.67:31003 [rx-stats: sb=42,ub=42,sc=1,uc=1][tx-stats: sb=66978,ub=66978,sc=3,uc=3] with method planexecution with id 8409680334250871036 on host host212 on port 31003 from process hdbindexserver with service port 31003, pid 162132 and tid 38573: exception  1: no.2111019  (TrexNet/BufferedIO.cpp:801) TID: 38573
    generic; $TYPE$=ReadTimeout; $MESSAGE$=a timeout occurred while reading from the channel; $INFO$=timeout occured; $PARAM$=active channel (since 10022956mues) 10194701 with request-id 8409680334250871036 {0x00007edba1943f20: id=10194701, channel={<NetworkChannel>={<NetworkChannelBase>={this=139463707317272, fd=1987, refCnt=1, local=xx.xxx.xxx.47/53448_tcp, remote=xx.xxx.xxx.67/31003_tcp, state=Connected, pending=[----]}}}} from 53448 to xx.xxx.xxx.67:31003 [rx-stats: sb=42,ub=42,sc=1,uc=1][tx-stats: sb=66978,ub=66978,sc=3,uc=3] with method planexecution with id 8409680334250871036 on host host212 on port 31003 from process hdbindexserver with service port 31003, pid 162132 and tid 38573
exception throw location:

 

  • Further investigation into the thread samples for the transactions, shows the jobworker threads stuck during execution between nodes 11 and 12 in a 'Network Poll' state when checking the local thread samples :
    (SELECT * FROM _SYS_STATISTICS.HOST_SERVICE_THREAD_SAMPLES where timestamp between 'dd.mm.2023 00:00:00.000000' and 'dd.mm.2023 00:00:00.000000')

CALLERCLIENT_IPHOSTPASSPORT_ACTIONPASSPORT_CONNECTION_IDPASSPORT_ROOTCONTEXT_IDPASSPORT_TRANSACTION_IDROOT_STATEMENT_HASHTHREAD_DETAILTHREAD_IDTHREAD_METHODTHREAD_STATETHREAD_TYPETIMESTAMPTRANSACTION_ID
               
               
185923@host211:31003<IP>host211BIDTPR_20230516104524000589_100000000000000000000000000000000005056AB558D1EDC93AD4573FA2FBE8E1093697C2A1B1810E00646355670FFB70b16e1e278db03340bffdbb68c11fce4peer: conn=host212:51682 service=3100335,960planexecutionNetwork PollRequest202305160649176923720002,617
185923@host211:31003<IP>host211BIDTPR_20230516104524000589_100000000000000000000000000000000005056AB558D1EDC93AD4573FA2FBE8E1093697C2A1B1810E00646355670FFB70b16e1e278db03340bffdbb68c11fce4peer: conn=host212:51682 service=3100335,960planexecutionNetwork PollRequest202305160650078290810002,617
43064<IP>host212BIDTPR_20230516104524000589_100000000000000000000000000000000005056AB558D1EDC93AD4573FA2FBE8E1093697C2A1B1810E00646355670FFB70b16e1e278db03340bffdbb68c11fce4peer: conn=host211:51682 service=31003185,923PlanExecutor commNetwork PollJobWorker202305160650085114440002,617
43064<IP>host212BIDTPR_20230516104524000589_100000000000000000000000000000000005056AB558D1EDC93AD4573FA2FBE8E1093697C2A1B1810E00646355670FFB70b16e1e278db03340bffdbb68c11fce4peer: conn=host211:51682 service=31003185,923PlanExecutor commNetwork PollJobWorker202305160649183644990002,617


Read more...

Environment

SAP HANA, Platform Edition 2.0

SAP HANA, Platform Edition 1.0

Product

SAP HANA 1.0, platform edition ; SAP HANA, platform edition 2.0

Keywords

KBA , HAN-DB , SAP HANA Database , HAN-DB-CLI , SAP HANA Clients (JDBC, ODBC) , Problem

About this page

This is a preview of a SAP Knowledge Base Article. Click more to access the full version on SAP for Me (Login required).

Search for additional results

Visit SAP Support Portal's SAP Notes and KBA Search.