Symptom
- A HANA system is failing to startup.
- Output from sapcontrol /usr/sap/hostctrl/exe/sapcontrol -nr <instance number> -function GetProcessList , shows all services in a YELLOW status
GetProcessList
OK
name, description, dispstatus, textstatus, starttime, elapsedtime, pid
hdbdaemon, HDB Daemon, YELLOW, Initializing, 2023 03 23 09:58:53, 2:24:43, 64750
hdbcompileserver, HDB Compileserver, YELLOW, Scheduled, , , -1
hdbdiserver, HDB Deployment Infrastructure Server, YELLOW, Scheduled, , , -1
hdbdiserver, HDB Deployment Infrastructure Server-SID, YELLOW, Scheduled, , , -1
hdbdiserver, HDB Deployment Infrastructure Server-SID, YELLOW, Scheduled, , , -1
hdbdocstore, HDB DocStore-SID, YELLOW, Scheduled, , , -1
hdbdpserver, HDB DPserver-SID, YELLOW, Scheduled, , , -1
hdbindexserver, HDB Indexserver-SID, YELLOW, Scheduled, , , -1
hdbindexserver, HDB Indexserver-SID, YELLOW, Scheduled, , , -1
hdbnameserver, HDB Nameserver, YELLOW, Initializing, 2023 03 23 12:22:33, 0:01:03, 148035
hdbpreprocessor, HDB Preprocessor, YELLOW, Scheduled, , , -1
hdbwebdispatcher, HDB Web Dispatcher, YELLOW, Scheduled, , , -1
hdbxscontroller, HDB XS Controller, YELLOW, Scheduled, , , -1
hdbxsengine, HDB XSEngine-SID, YELLOW, Scheduled, , , -1
hdbxsengine, HDB XSEngine-SID, YELLOW, Scheduled, , , -1
hdbxsexecagent, HDB XS Execution Agent, YELLOW, Scheduled, , , -1
hdbxsuaaserver, HDB XS UAA Server, YELLOW, Scheduled, , , -
- The daemon service trace shows the following error pattern repeating over and over again since most recent startup attempts :
[72413]{-1}[-1/-1] 2023-03-23 01:19:09.927498 i Daemon Daemon.cpp(00155) : Shared memory updated, no daemon state change
[72413]{-1}[-1/-1] 2023-03-23 01:19:10.178854 e Stream NetworkChannel.cpp(03412) : [d86b9fde900c0019,127.0.0.1:30000,-,DAE,0] Exception occurred in accept: exception 1: no.300012 (Crypto/Shared/SSL/CommonCrypto/Context.cpp:346) TID: 72413
Cannot create SSL engine: $ErrorText$; $Context$=[d86b9fde900c0373,127.0.0.1:30000,127.0.0.1:58882,DAE,0]; $channel$={<NetworkChannelSSLFilter>={<NetworkChannelBase>={this=94908956732840, fd=12, refCnt=1, local=127.0.0.1/30000_tcp, remote=127.0.0.1/58882_tcp, state=Connected, pending=[----]}}}
exception throw location:
0: 0x00007f1abcd004c8 in Crypto::SSL::CommonCrypto::Context::createEngine(Crypto::SSL::Engine::Type, ltt::allocator&)+0x394 at Crypto/Shared/SSL/CommonCrypto/Context.cpp:346 (libcrypto_base.so)
1: 0x00007f1abccef52c in Crypto::SSL::Filter::Acceptor::accept(void const*, unsigned long, Crypto::SSL::FilterWithStepByStepHandshake::ServerHandshakePhase)+0xad8 at Crypto/Shared/SSL/Filter.cpp:186 (libcrypto_base.so)
2: 0x00007f1ac0cc7649 in Stream::NetworkChannelSSLFilter::doSSLHandShake(void const*, unsigned long)+0x1c5 at Basis/IO/Stream/impl/NetworkChannelSSLFilter.cpp:344 (libhdbbasis.so)
3: 0x00007f1ac0cc80d9 in Stream::NetworkChannelSSLFilter::initServerChannel()+0x55 at Basis/IO/Stream/impl/NetworkChannelSSLFilter.cpp:225 (libhdbbasis.so)
4: 0x00007f1ac0ccf0ea in Stream::NetworkListener::connectionEvent(Stream::CompletionSet::EventType, int, ltt::refcounted_handle<Stream::NetworkChannel>&, unsigned long)+0x336 at Basis/IO/Stream/impl/NetworkChannel.cpp:3463 (libhdbbasis.so)
5: 0x00007f1ac0ccf7bd in Stream::NetworkListener::acceptEvent(Stream::CompletionSet::EventType, int)+0x69 at Basis/IO/Stream/impl/NetworkChannel.cpp:3408 (libhdbbasis.so)
6: 0x00007f1ac1983c1f in TrexDaemon::NetworkListenerChannel::handleEvent(Stream::CompletionSet::EventType, int)+0x2b at TrexDaemon/impl/NetworkListener.cpp:994 (libhdbservice.so)
7: 0x00007f1ac19a31c3 in TrexDaemon::waitForChild(ltt::vector<ltt::pair<ltt::smart_ptr<TrexDaemon::RunningInstance, ltt::integral_constant<bool, false>, ltt::integral_constant<bool, false> >, ltt::smart_ptr<TrexDaemon::Program, ltt::integral_constant<bool, false>, ltt::integral_constant<bool, false> > > > const&, ltt::basic_string<char, ltt::char_traits<char>, ltt::integral_constant<int, 64> >&, long)+0x660 at TrexDaemon/impl/SignalsUNIX.cpp:831 (libhdbservice.so)
8: 0x00007f1ac196a60f in TrexDaemon::Daemon::loop()+0x44b at TrexDaemon/impl/DaemonLoop.cpp:237 (libhdbservice.so)
9: 0x00007f1ac197aefa in daemonMainImpl(int, char**)+0x1ef6 at TrexDaemon/impl/MainImpl.cpp:927 (libhdbservice.so)
10: 0x00007f1ac0f21d81 in System::mainWrapper(int, char**, char**)+0x80 at Basis/System/impl/IsInMain.cpp:399 (libhdbbasis.so)
11: 0x00007f1abec1a493 in .annobin_libc_start.c+0xf0 (libc.so.6)
12: 0x00005651b41468f3 in _sub_I_65535_0.0+0xa0 at ../sysdeps/x86_64/start.S:112 (hdbdaemon)
exception type information:
- 0: public ltt::exception@0x00007f1ac009d8e8
[72413]{-1}[-1/-1] 2023-03-23 01:19:10.179009 e Daemon NetworkListener.cpp(01008) : Daemon on 127.0.0.1:30000, timeout 21188 cannot accept new connection
[72413]{-1}[-1/-1] 2023-03-23 01:20:13.970691 i Daemon DaemonHandle.cpp(00323) : Instance count 1. Searching for terminated processes
[72413]{-1}[-1/-1] 2023-03-23 01:20:13.970777 i Daemon DaemonHandle.cpp(00334) : Process 'hdbnameserver', pid 398487 exited normally with status 0
[72413]{-1}[-1/-1] 2023-03-23 01:20:13.970856 i Daemon Events.cpp(00031) : "StartInstanceEvent" is due in 10000 ms. Program 'hdbnameserver', pid 0
[72413]{-1}[-1/-1] 2023-03-23 01:20:13.970876 i Daemon Daemon.cpp(01142) : Daemon state "starting", runlevels [current:reached:target]=[1:1:5]
[72413]{-1}[-1/-1] 2023-03-23 01:20:13.971009 i Daemon Daemon.cpp(00155) : Shared memory updated, no daemon state change
[72413]{-1}[-1/-1] 2023-03-23 01:20:13.974057 w Daemon Daemon.cpp(00757) : Found terminated and planned instance 0 of program 'nameserver'
[72413]{-1}[-1/-1] 2023-03-23 01:20:23.980225 i Daemon Program.cpp(00038) : Starting instance right now
[72413]{-1}[-1/-1] 2023-03-23 01:20:23.980460 i Daemon DaemonChild.cpp(00481) : Starting program 'hdbnameserver' with args ''
[72413]{-1}[-1/-1] 2023-03-23 01:20:23.986753 i Environment Environment.cpp(00236) : Changing environment put PYTHONDONTWRITEBYTECODE=1
[72413]{-1}[-1/-1] 2023-03-23 01:20:23.990021 i Daemon DaemonChild.cpp(00708) : Child program 'hdbnameserver', pid 399539 started in its own process group
[72413]{-1}[-1/-1] 2023-03-23 01:20:23.990128 i Daemon RunningInstance.cpp(00259) : Start 'hdbnameserver' as process 399539, process group 399539, instance 0 of <HDB Nameserver>
[72413]{-1}[-1/-1] 2023-03-23 01:20:23.990152 i Daemon Daemon.cpp(01142) : Daemon state "starting", runlevels [current:reached:target]=[1:1:5]
[72413]{-1}[-1/-1] 2023-03-23 01:20:23.990283 i Daemon Daemon.cpp(00155) : Shared memory updated, no daemon state change
[72413]{-1}[-1/-1] 2023-03-23 01:20:28.960965 e Stream NetworkChannel.cpp(03412) : [d86b9fde900c0019,127.0.0.1:30000,-,DAE,0] Exception occurred in accept: exception 1: no.300012 (Crypto/Shared/SSL/CommonCrypto/Context.cpp:346) TID: 72413
Cannot create SSL engine: $ErrorText$; $Context$=[d86b9fde900c0374,127.0.0.1:30000,127.0.0.1:45924,DAE,0]; $channel$={<NetworkChannelSSLFilter>={<NetworkChannelBase>={this=94908956732840, fd=12, refCnt=1, local=127.0.0.1/30000_tcp, remote=127.0.0.1/45924_tcp, state=Connected, pending=[----]}}}
exception throw location:
0: 0x00007f1abcd004c8 in Crypto::SSL::CommonCrypto::Context::createEngine(Crypto::SSL::Engine::Type, ltt::allocator&)+0x394 at Crypto/Shared/SSL/CommonCrypto/Context.cpp:346 (libcrypto_base.so)
1: 0x00007f1abccef52c in Crypto::SSL::Filter::Acceptor::accept(void const*, unsigned long, Crypto::SSL::FilterWithStepByStepHandshake::ServerHandshakePhase)+0xad8 at Crypto/Shared/SSL/Filter.cpp:186 (libcrypto_base.so)
2: 0x00007f1ac0cc7649 in Stream::NetworkChannelSSLFilter::doSSLHandShake(void const*, unsigned long)+0x1c5 at Basis/IO/Stream/impl/NetworkChannelSSLFilter.cpp:344 (libhdbbasis.so)
3: 0x00007f1ac0cc80d9 in Stream::NetworkChannelSSLFilter::initServerChannel()+0x55 at Basis/IO/Stream/impl/NetworkChannelSSLFilter.cpp:225 (libhdbbasis.so)
4: 0x00007f1ac0ccf0ea in Stream::NetworkListener::connectionEvent(Stream::CompletionSet::EventType, int, ltt::refcounted_handle<Stream::NetworkChannel>&, unsigned long)+0x336 at Basis/IO/Stream/impl/NetworkChannel.cpp:3463 (libhdbbasis.so)
5: 0x00007f1ac0ccf7bd in Stream::NetworkListener::acceptEvent(Stream::CompletionSet::EventType, int)+0x69 at Basis/IO/Stream/impl/NetworkChannel.cpp:3408 (libhdbbasis.so)
6: 0x00007f1ac1983c1f in TrexDaemon::NetworkListenerChannel::handleEvent(Stream::CompletionSet::EventType, int)+0x2b at TrexDaemon/impl/NetworkListener.cpp:994 (libhdbservice.so)
7: 0x00007f1ac19a31c3 in TrexDaemon::waitForChild(ltt::vector<ltt::pair<ltt::smart_ptr<TrexDaemon::RunningInstance, ltt::integral_constant<bool, false>, ltt::integral_constant<bool, false> >, ltt::smart_ptr<TrexDaemon::Program, ltt::integral_constant<bool, false>, ltt::integral_constant<bool, false> > > > const&, ltt::basic_string<char, ltt::char_traits<char>, ltt::integral_constant<int, 64> >&, long)+0x660 at TrexDaemon/impl/SignalsUNIX.cpp:831 (libhdbservice.so)
8: 0x00007f1ac196a60f in TrexDaemon::Daemon::loop()+0x44b at TrexDaemon/impl/DaemonLoop.cpp:237 (libhdbservice.so)
9: 0x00007f1ac197aefa in daemonMainImpl(int, char**)+0x1ef6 at TrexDaemon/impl/MainImpl.cpp:927 (libhdbservice.so)
10: 0x00007f1ac0f21d81 in System::mainWrapper(int, char**, char**)+0x80 at Basis/System/impl/IsInMain.cpp:399 (libhdbbasis.so)
11: 0x00007f1abec1a493 in .annobin_libc_start.c+0xf0 (libc.so.6)
12: 0x00005651b41468f3 in _sub_I_65535_0.0+0xa0 at ../sysdeps/x86_64/start.S:112 (hdbdaemon)
exception type information:
- 0: public ltt::exception@0x00007f1ac009d8e8
[72413]{-1}[-1/-1] 2023-03-23 01:20:28.961265 e Daemon NetworkListener.cpp(01008) : Daemon on 127.0.0.1:30000, timeout 7156 cannot accept new connection
[72413]{-1}[-1/-1] 2023-03-23 01:20:30.799021 i Daemon SignalsUNIX.cpp(00601) : signo 10=SIGUSR1 from user. errno 0 code 0. Requested 'RECONFIGURE'. Sender pid 399539, real user 'sidadm'=248592, executable 'hdbnameserver'
[72413]{-1}[-1/-1] 2023-03-23 01:20:30.799377 i Daemon DaemonHandle.cpp(00144) : Reconfiguring all services
Read more...
Environment
SAP HANA, Platform Edition 2.0
Product
SAP HANA, platform edition 2.0
Keywords
Daemon.cpp, Shared memory updated, no daemon state change, NetworkChannel.cpp, Stream, SSL, CommonCrypto, createEngine , KBA , HAN-DB , SAP HANA Database , HAN-DB-SEC , SAP HANA Security & User Management , Problem
About this page
This is a preview of a SAP Knowledge Base Article. Click more to access the full version on SAP for Me (Login required).Search for additional results
Visit SAP Support Portal's SAP Notes and KBA Search.