3771197 - S4 Hana production system we are facing intermittent performance issues

- Intermittent slow performance of queries on tables with high number of partitions. 

- Call stacks of the threads running the query: 

 65806903245[thr=166982]: JobWrk24603589 at
   0: 0x00007fff542edc00 in UnifiedTable::RowStateScanContext::initScanContext(int, bool)+0x40 at UnifiedTable/Paging/spi/BlockHandle.hpp:226 (libhdbunifiedtable.so)
   1: 0x00007fff542dbcb0 in UnifiedTable::MVCCObject::loadIterator(UnifiedTable::MVCCReader::iterator&, unsigned long*, unsigned long) const+0xd0 at UnifiedTable/MVCC/MVCCSPTSHelper.hpp:626 (libhdbunifiedtable.so)
   2: 0x00007fff543160f4 in UnifiedTable::MVCCReader::iterator::load(unsigned long)+0x34 at UnifiedTable/MVCC/impl/MVCCReader.cpp:38 (libhdbunifiedtable.so)
   3: 0x00007fff542e0d1c in UnifiedTable::MVCCObject::getEstimatedVisibleRows(int, unsigned long) const+0x3fc at UnifiedTable/MVCC/MVCCReader.hpp:303 (libhdbunifiedtable.so)
   4: 0x00007fff54318d08 in UnifiedTable::MVCCReader::getEstimatedVisibleRows(int, unsigned long) const+0x18 at UnifiedTable/MVCC/impl/MVCCReader.cpp:729 (libhdbunifiedtable.so)
   5: 0x00007fff6bf79f9c in TrexStore::UdivListContainerMVCC::getEstimatedVisibleRows()+0x1ac at TrexStore/UdivListContainerMVCC.cpp:2942 (libhdbcsstore.so)
   6: 0x00007fff6bfa9660 in TrexStore::UdivListManagerImpl::getEstimatedRowsTotal(TrexBase::IndexName const&)+0x70 at TrexStore/UdivListManager/UdivListManagerImpl.cpp:1451 (libhdbcsstore.so)
   7: 0x00007fff5b7dd234 in TRexConfig::RuntimeData::getEstimatedVisibleRowsTotal() const+0x54 at TRexConfig/RuntimeData.cpp:989 (libhdbcsmd.so)
   8: 0x00007fff7afc0d5c in DataStatistics::CS_StatisticsImpl::getStatisticsForLocalColumns(ltt::smart_ptr<DataStatistics::AttributeApiWrapperBase, ltt::integral_constant<bool, false>, ltt::integral_constant<bool, false> >, TrexBase::IndexName const&, ltt_adp::vector<ltt_adp::basic_string<char, ltt::char_traits<char>, ltt::integral_constant<bool, true> >, ltt::integral_constant<bool, true> > const&, bool, ltt_adp::vector<DataStatistics::PartitionStatistics, ltt::integral_constant<bool, true> >&, unsigned long&, TRexCommonObjects::TRexApiError&) const+0x1d0c at DataStatistics/CSDataStatistics/CS_StatisticsImpl.cpp:2189 (libhdbdatastatistics.so)
   9: 0x00007fff7afc4cd0 in DataStatistics::CS_StatisticsImpl::getStatisticsForALocalPartition(ltt::smart_ptr<DataStatistics::AttributeApiWrapperBase, ltt::integral_constant<bool, false>, ltt::integral_constant<bool, false> >, TrexBase::IndexName const&, ltt_adp::vector<ltt_adp::basic_string<char, ltt::char_traits<char>, ltt::integral_constant<bool, true> >, ltt::integral_constant<bool, true> > const&, unsigned long&, ltt_adp::vector<DataStatistics::PartitionStatistics, ltt::integral_constant<bool, true> >&, bool, TRexCommonObjects::TRexApiError&) const+0x80 at DataStatistics/CSDataStatistics/CS_StatisticsImpl.cpp:1885 (libhdbdatastatistics.so)
  10: 0x00007fff7afdd8d8 in _ZZNK14DataStatistics17CS_StatisticsImpl31getStatisticsForLocalPartitionsERKN7ltt_adp6vectorIN8TrexBase9IndexNameEN3ltt17integral_constantIbLb1EEEEERKNS2_INS1_12basic_stringIcNS5_11char_traitsIcEES7_EES7_EERNS2_INS2_INS_19PartitionStatisticsES7_EES7_EEbRmRN17TRexCommonObjects12TRexApiErrorEENKUlRKT_RKT0_E_clIjjEEvSS_SV_.lto_priv.0+0x2c8 at DataStatistics/CSDataStatistics/CS_StatisticsImpl.cpp:2002 (libhdbdatastatistics.so)
  11: 0x00007fff7afdddc8 in Execution::impl::ForImplJob<Execution::SimpleBlockPartitioner<unsigned int>, DataStatistics::CS_StatisticsImpl::getStatisticsForLocalPartitions(ltt_adp::vector<TrexBase::IndexName, ltt::integral_constant<bool, true> > const&, ltt_adp::vector<ltt_adp::basic_string<char, ltt::char_traits<char>, ltt::integral_constant<bool, true> >, ltt::integral_constant<bool, true> > const&, ltt_adp::vector<ltt_adp::vector<DataStatistics::PartitionStatistics, ltt::integral_constant<bool, true> >, ltt::integral_constant<bool, true> >&, bool, unsigned long&, TRexCommonObjects::TRexApiError&) const::{lambda(auto:1 const&, auto:2 const&)#1}>::run(Execution::Context&, Execution::JobObject&) [clone .lto_priv.0]+0xa8 at Basis/Timer.hpp:136 (libhdbdatastatistics.so)
  12: 0x00007fff4d6fae10 in Execution::JobObjectImpl::run(Execution::JobWorker*)+0x1380 at Basis/Execution/impl/JobExecutorRunJob.cpp:772 (libhdbbasis.so)
  13: 0x00007fff4d6fd9c0 in Execution::JobWorker::runJob(ltt::smartptr_handle<Execution::JobObjectForHandle>&)+0xe60 at Basis/Execution/impl/JobExecutorTypes.hpp:487 (libhdbbasis.so)
  14: 0x00007fff4d6ff99c in Execution::JobWorker::run(Execution::ThreadRC&)+0xb7c at Basis/Execution/impl/JobExecutorThreads.cpp:1385 (libhdbbasis.so)
  15: 0x00007fff4d754e1c in Execution::Thread::staticMainImp(Execution::Thread*)+0x5ec at Basis/Execution/impl/Thread.cpp:605 (libhdbbasis.so)
  16: 0x00007fff4d75e0b8 in Execution::pthreadFunctionWrapper(Execution::Thread*)+0x248 at Basis/Execution/impl/ThreadInterposition.cpp:691 (libhdbbasis.so)
  17: 0x00007fff4ceb9748 in start_thread+0xf8 (libpthread.so.0)

- Check the m_sql_plan_cache entry for the statement hash, see very high avg_preparation_time. 

HANA revision 2.0

KBA , HAN-DB-PERF , SAP HANA Database Performance , Problem