2238504 - Informational stack trace from out of virtual memory condition - SAP IQ

• IQ SQL transactions start to fail with varying out of virtual memory conditions. For example...

I. 08/27 13:41:41. 0000055370     ********************************************************
I. 08/27 13:41:41. 0000055370     **
I. 08/27 13:41:41. 0000055370     **       Informational stack trace from out of virtual memory condition
I. 08/27 13:41:41. 0000055370     **       at slib/s_index.cxx:3931  on thread 445 (TID 436)
I. 08/27 13:41:41. 0000055370     **       Failed allocation request (bytes): 16793600
I. 08/27 13:41:41. 0000055370     **       Failed allocation request type:  s_indexFindAccumulator::_memRegionA
I. 08/27 13:41:41. 0000055370     **
I. 08/27 13:41:41. 0000055370     **  Error from IQ connection:  SA connHandle: 2398  SA connID: 56  IQ connID: 0000055370  User: dc
I. 08/27 13:41:41. 0000055370     **  Time of error:  2015-08-27 13:41:41
I. 08/27 13:41:41. 0000055370     **  IQ Version:  SAP IQ/16.0.0/150514/P/sp08.32
I. 08/27 13:41:41. 0000055370     **  OS info:  IQ built on: Sun_x64/OS 5.10,  Executed on: SunOS/atrcxb2908/5.10/Generic_150401-20/i86pc

I. 08/27 13:42:19. 0000055370     ***************** This is the STACKTRACE ***************
I. 08/27 13:42:19. 0000055370 ===== Thread Number  637  (IQ connID: 0000055370) =====
I. 08/27 13:42:19. 0000055370 pc: 0x0 (0x274, 0xfffffd7fd48fc220)
I. 08/27 13:42:19. 0000055370 pc: 0xfffffd7fe436be04 void StackTraceForThisThread(const char*,int,const char*,const char*,const char*)+0x3c4(0xfffffd7fd48fc9a0, 0xfffffd7fe6bb3bd0)
I. 08/27 13:42:19. 0000055370 pc: 0xfffffd7fe42d427d void*operator new(unsigned long)+0xd9d(0xfffffd7fd48fc9d0, 0x2768b1b548)
I. 08/27 13:42:19. 0000055370 pc: 0xfffffd7fe4ba1801 void s_indexFindAccumulator::RegisterIndex(s_index*,int)+0xb1(0x4, 0x3)
I. 08/27 13:42:19. 0000055370 pc: 0xfffffd7fe4d007db int hs_ohc::Find(const void*,int,hos_op,const void*,int,hos_op,s_bm*,hos_boolop,const s_bm*,s_indexFindContext*,hos_op,unsigned long long)+0x2fb(0x4, 0x2896dcdd88)
I. 08/27 13:42:19. 0000055370 pc: 0xfffffd7fe52c035b void vpp_PWUContext::DoOneWorkunit()+0x127b(0x6, 0x2a025d4468)
I. 08/27 13:42:19. 0000055370 pc: 0xfffffd7fe52b388c void vpp::Execute(s_bm**,const s_bm*,const s_bm*,hos_boolop)+0x61c(0xfffffd7fd48fcf18, 0xfffffd7fd48fcf18)
I. 08/27 13:42:19. 0000055370 pc: 0xfffffd7fe52c376b int vpp_Combiner::ExecWork(int&,unsigned)+0x15b(0x0, 0x28fe1aa300)
I. 08/27 13:42:19. 0000055370 pc: 0xfffffd7fe433624a void workAllocator::DoWork(unsigned)+0x21a(0x0, 0xfffffd7fe6532250)
I. 08/27 13:42:19. 0000055370 pc: 0xfffffd7fe4368bc6 int hos_thread::Main()+0x342(0x0, 0x0)
I. 08/27 13:42:19. 0000055370 pc: 0xfffffd7fe42b6770 int hos_lwtask::Start(hos_lwtask*)+0x224(0xfffffd7fd48fef60, 0xfffffd7fd487d000)
I. 08/27 13:42:19. 0000055370 pc: 0xfffffd7ffdf05a92 void IQWorkerStarter(void*)+0x46(0xfffffd7fd48fefc0, 0xd67250)
I. 08/27 13:42:19. 0000055370 pc: 0xfffffd7ffe205681 void*UnixTask::pre_body(void*)+0x119(0x0, 0x0)
I. 08/27 13:42:19. 0000055370 pc: 0xfffffd7ffd70da9b _fini+0x173fd2bb(0x0, 0x0)
I. 08/27 13:42:19. 0000055370     ******************* End of STACKTRACE ******************
I. 08/27 13:42:19. 0000055370     **
I. 08/27 13:42:19. 0000055370     **       End of informational stack trace from out of virtual memory condition
I. 08/27 13:42:19. 0000055370     **
I. 08/27 13:42:19. 0000055370     ********************************************************
I. 08/27 13:42:19. 0000055370
I. 08/27 13:42:19. 0000055370
I. 08/27 13:42:19. 0000055370     ********************************************************
I. 08/27 13:42:19. 0000055370 Exception Thrown from oslib/hos_mem.cxx:578, Err# 1, tid 628 origtid 628
I. 08/27 13:42:19. 0000055370     **
I. 08/27 13:42:19. 0000055370    O/S Err#: 0, ErrID: 517 (hos_memexception); SQLCode: -1006042, SQLState: 'QBA42', Severity: 14
I. 08/27 13:42:19. 0000055370     **       Informational stack trace from out of virtual memory condition
I. 08/27 13:42:19. 0000055370 [20280]: All available virtual memory has been used; allocation cancelled: [Extra info: 16793600]

• The OS free available RAM is exhausted and disk swapping activity is high or overloaded , possibly reaching its limit.
• Other non IQ processes running on the OS may also start to fail with memory related issues , possibly leading to a high level OS system abort requiring a reboot of the machine
• The issue can also lead to the IQ process failing with  a st_server SEGV Abort. For example...

I. 08/27 13:49:36. 0000054286 **************************************************
I. 08/27 13:49:36. 0000054286 ***   SAP IQ Abort:
I. 08/27 13:49:36. 0000054286 ***      From:  stcxtlib/st_server.cxx:1709
I. 08/27 13:49:36. 0000054286 ***      PID: 28260
I. 08/27 13:49:36. 0000054286 ***      Message: caught signal 10, program abort
I. 08/27 13:49:36. 0000054286 ***      Thread: 635  (TID: 626)
I. 08/27 13:49:36. 0000054286 **************************************************
I. 08/27 13:49:36. 0000054286
I. 08/27 13:49:36. 0000054286     **  Error from IQ connection:  SA connHandle: 2347  SA connID: 48  IQ connID: 0000054286  User: dc
I. 08/27 13:49:36. 0000054286     **  Time of error:  2015-08-27 13:49:36
I. 08/27 13:49:36. 0000054286     **  IQ Version:  SAP IQ/16.0.0/150514/P/sp08.32
I. 08/27 13:49:36. 0000054286     **  OS info:  IQ built on: Sun_x64/OS 5.10,  Executed on: SunOS/atrcxb2908/5.10/Generic_150401-20/i86pc
I. 08/27 13:49:36. 0000054286     **  Command status when error occured:  CURSOR ACTIVE
I. 08/27 13:49:36. 0000054286     **  Command text:
  <  failing SQL statement >
I. 08/27 13:49:36. 0000054286
    ***************** This is the STACKTRACE ***************

I. 08/27 13:49:36. 0000054286
===== Thread Number  635  (IQ connID: 0000054286) =====
I. 08/27 13:49:36. 0000054286 pc: 0x0 (0x100000000, 0x9ed408e98)
I. 08/27 13:49:36. 0000054286 pc: 0xfffffd7fe4369743 void DumpAllThreads(const char*,unsigned,int)+0x29f(0xfffffd7fd4a1af20, 0xfffffd7fe69eb830)
I. 08/27 13:49:36. 0000054286 pc: 0xfffffd7fe404f57a void hos_ABORT(const char*,unsigned,const char*,char*,char*)+0x1a2(0x7320746867756163, 0x3031206c616e6769)
I. 08/27 13:49:36. 0000054286 pc: 0xfffffd7fe5026d55 SigHndlr+0x41(0xfffffd7fd4a1b390, 0xfffffd7ffd7026a2)
I. 08/27 13:49:36. 0000054286 pc: 0xfffffd7ffd70ddd6 _fini+0x173fd5f6(0xfffffd7fd4a1ba98, 0xafd771c40)
I. 08/27 13:49:36. 0000055125 Txn 97829061 0 96887746
I. 08/27 13:49:36. 0000054286 pc: 0xfffffd7ffd7026a2 _fini+0x173f1ec2(0x27837b8738, 0x2a007ef728)
I. 08/27 13:49:36. 0000054286 pc: 0xfffffd7ffd7028ce _fini+0x173f20ee(0xa, 0xfffffd7fd4a1ba98)
I. 08/27 13:49:36. 0000054286 pc: 0xffffffffffffffff _fini+0x12830ff(0xfffffd7fd4a1bc80, 0x122effa8)
I. 08/27 13:49:36. 0000054286 pc: 0xfffffd7fe3caf46c int dfo_RowID::FetchOpus()+0x7c(0x0, 0x1)
I. 08/27 13:49:36. 0000054286 pc: 0xfffffd7fe3e2f078 int dfo_IndexInsert::FetchOpus()+0x80(0x278383c6d0, 0x0)
I. 08/27 13:49:36. 0000054286 pc: 0xfffffd7fe3c57c9d const s_packedRecDesc*CombinerFetchCallback(void*)+0x79(0x3, 0x2850dbc918)
I. 08/27 13:49:36. 0000054286 pc: 0xfffffd7fe4da253e void hc_dfrun::FillRun(unsigned&,int,int)+0x3a(0x0, 0xfffffd7fe668bb00)
I. 08/27 13:49:36. 0000054286 pc: 0xfffffd7fe4f6f0d9 int hc_dmfrun::InitNextWorkUnit(unsigned,int,unsigned)+0x7ed(0x0, 0x297e6e2f20)
I. 08/27 13:49:36. 0000056100 Cmt 97829062
I. 08/27 13:49:36. 0000054286 pc: 0xfffffd7fe433624a void workAllocator::DoWork(unsigned)+0x21a(0x0, 0xfffffd7fe6532250)

SAP IQ 16

KBA , BC-SYB-IQ , Sybase IQ , Problem