3426504 - SAP HANA Secondary Site Not Started After Upgrade

SAP HANA secondary site indexserver doesn't start after upgrade, HANA parameters under the Tenant DB global.ini [system_replication] sections were removed before the upgrade.

The following errors are written in indexserver trace:

<timestamp> e assign           TREXIndexServer.cpp(01926) : assign failed with ltt exception. stopping service... :
exception  1: no.7010003  (TREXNameServer/SecurityUtils/RootKeyStoreDistribution.cpp:70) TID: 241584
    service is aborting: Fetching root keys failed - probably inconsistent SSFS! Please refer to SAP note http://service.sap.com/sap/support/notes/2194396
exception throw location:
   0: 0x00007fff9b527ef8 in NameServer::AbortException::AbortException(char const*, int, ltt_adp::basic_string<char, ltt::char_traits<char>, ltt::integral_constant<bool, true> > const&)+0x58 at TREXNameServer/AbortException.cpp:10 (libhdbns.so)
   1: 0x00007fff9b4fd9bc in NameServer::fetchRootKeyStoreForDbid(unsigned int, int, NameServer::TREXNameServerUtilsWrapper&, ltt::allocator&)+0x93c at TREXNameServer/SecurityUtils/RootKeyStoreDistribution.cpp:70 (libhdbns.so)
   2: 0x00000001153abd10 in CacheCSComponent.cpp.699d2ec5+0x1525 at TREXIndexServer2/TREXIndexServer.cpp:1120 (hdbindexserver)
   3: 0x00000001153b1264 in TRexAPI::AssignThread::run(void*)+0x34 at TREXIndexServer2/TREXIndexServer.cpp:524 (hdbindexserver)
   4: 0x00007fff6be0fa40 in TrexThreads::PoolThread::run()+0x12a0 at TrexThreads/PoolThread.cpp:437 (libhdbbasement.so)
   5: 0x00007fff6bdfebe8 in TrexThreads::PoolThread::run(Execution::ThreadRC&)+0x28 at TrexThreads/PoolThread.cpp:121 (libhdbbasement.so)
   6: 0x00007fff6776fd3c in Execution::Thread::staticMainImp(Execution::Thread*)+0x64c at Basis/Execution/impl/Thread.cpp:612 (libhdbbasis.so)
   7: 0x00007fff67779fd8 in Execution::pthreadFunctionWrapper(Execution::Thread*)+0x248 at Basis/Execution/impl/ThreadInterposition.cpp:684 (libhdbbasis.so)
   8: 0x00007fff66ee9748 in start_thread+0xf8 (libpthread.so.0)
exception type information:
 - 0: public NameServer::AbortException@0x00007fff9b75bb50 SI
   - 0: public ltt::exception@0x00007fff66dff340

SAP HANA 2.0 with system replication

Secondary site upgrade, Fetching root keys failed, service is aborting, hdblcm, system_replication , KBA , HAN-LM-UPG-DB , Upgrade of HANA Database , HAN-DB-HA , SAP HANA High Availability (System Replication, DR, etc.) , Problem