2677366 - Tenant with Dynamic Tiering hang - SAP HANA Dynamic Tiering

Connections to SAP Dynamic Tiering could hang. Other operations would become slower and slower.

This is likely to occur when client applications use a large number of threads.

For example, one of the situation could be:

• A tenant with HANA, Dynamic Tiering, and Replication. Connections to the tenant from HANA Studio could take a long time.
• Once connected, the "Overview" tab could take a long time to be refreshed.
• Doing a "Transport" in "Life Cycle Management" could take a long time.

indexserver.trc, many TLS errors can be seen:

e FedTrace odbcaccess.cpp(04070) : ODBC error: connected: 0 state: 08S01 code: -829 [SAP][ODBC Driver]TLS handshake failure
e FedTrace odbcaccess.cpp(04070) : ODBC error: connected: 0 state: 08001 code: -832 [SAP][ODBC Driver]Connection error: Timeout occurred while waiting for connection response

From the runtime dumps, 2 sets of threads can be found.

1. One set of threads show connection locked

__lll_lock_wait
2 _L_lock_918
3 __GI___pthread_mutex_lock
4 OS_MutexGet
5 c_strm_sess_layer::add_proxy
6 c_strm_sconn::start_receive
7 StrmConnect
8 i_cs_EngineConnect
9 CmdSeqEngineConnect
10 i_cs_StringConnectNoRedirect
11 i_cs_StringConnect
12 CmdSeqStringConnect
13 DBStringConnect
14 Attempt_connect
15 DoSQLDriverConnect
16 SQLDriverConnectW
17 SQLDriverConnectW
18 federation::OdmService::doOdbc
19 federation::OdmInternalService::do_odbc
20 federation::OdbcAccess::do_odbc
21 federation::SQLDriverConnect
22 federation::OdbcAccess::Connect
23 federation::FederationRdes::rdes_sub_connect
24 federation::FederationRdes::rdes_connect
25 federation::FederationContext::rdes_for_props
26 federation::FederationContext::OpenCursor
27 federation::ExtendedStorageDRHandler::GetReplicationStatistics(bool, ltt::basic_string<char, ltt::char_traits<char>, ltt::integer_constant<64> > (&) [3], DataAccess::ReplicationStatistics (&) [3], DataAccess::CurrentPersistenceStatus
28 PersistenceController::DisasterRecoveryESCallbackImpl::GetReplicationStatistics(bool, ltt::basic_string<char, ltt::char_traits<char>, ltt::integer_constant<64> > (&) [3], DataAccess::ReplicationStatistics (&) [3], DataAccess::CurrentPersistenceStatus
29 TrexService::PersistenceHandler::handle_getReplicationStatus
30 TrexService::PersistenceHandler::handle
31 TrexService::WorkerThread::run
32 TrexThreads::PoolThread::run
33 TrexThreads::PoolThread::run
34 Execution::Thread::staticMainImp
35 Execution::Thread::staticMain

2. Another set of threads  show connection attempts dropped

1 __write_nocancel
2 c_strm_sock_sel::remove_socket_with_event
3 c_strm_tcp_pconn::do_write
4 c_strm_tcp_pconn::blocking_put
5 c_strm_tcp_pconn::_blocking_put
6 c_strm_tcp_pconn::disconnect
7 c_strm_tconn::disconnect
8 c_strm_sconn::drop_connection
9 StrmDisconnect
10 CmdSeqDropConnection
11 i_cs_StringConnectNoRedirect
12 i_cs_StringConnect
13 CmdSeqStringConnect
14 DBStringConnect
15 Attempt_connect
16 DoSQLDriverConnect
17 SQLDriverConnectW
18 SQLDriverConnectW
19 federation::OdmService::doOdbc
20 federation::OdmInternalService::do_odbc
21 federation::OdbcAccess::do_odbc
22 federation::SQLDriverConnect
23 federation::OdbcAccess::Connect
24 federation::FederationRdes::rdes_sub_connect
25 federation::FederationRdes::rdes_connect
26 federation::FederationContext::rdes_for_props
27 federation::FederationContext::OpenCursor
28 federation::ExtendedStorageDRHandler::GetReplicationStatistics(bool, ltt::basic_string<char, ltt::char_traits<char>, ltt::integer_constant<64> > (&) [3], DataAccess::ReplicationStatistics (&) [3], DataAccess::CurrentPersistenceStatus
29 PersistenceController::DisasterRecoveryESCallbackImpl::GetReplicationStatistics(bool, ltt::basic_string<char, ltt::char_traits<char>, ltt::integer_constant<64> > (&) [3], DataAccess::ReplicationStatistics (&) [3], DataAccess::CurrentPersistenceStatus
30 TrexService::PersistenceHandler::handle_getReplicationStatus
31 TrexService::PersistenceHandler::handle
32 TrexService::WorkerThread::run
33 TrexThreads::PoolThread::run
34 TrexThreads::PoolThread::run
35 Execution::Thread::staticMainImp
36 Execution::Thread::staticMain

• SAP Dynamic Tiering 2.0
• SAP HANA Platform Edition
• SAP IQ 16.0
• SAP IQ 16.1

CR815583 ; CR#815583 ; 815583 ; TLS, "TLS handshake"; ODBC ; timeout; __write_nocancel ; c_strm_sconn::drop_connection ; CmdSeqDropConnection ;
CmdSeqStringConnect ; DBStringConnect ; Attempt_connect ; DoSQLDriverConnect ; __lll_lock_wait ; i_cs_EngineConnect ; CmdSeqEngineConnect ; , KBA , HAN-DYT , SAP HANA Dynamic Tiering , BC-SYB-IQ , Sybase IQ , BC-SYB-SQA , SQL Anywhere (on premise, on demand) , Problem