2508228 - IQ server got crashed at "slib/s_bufman.cxx:3436" and "s_bufpartition::Find"

1. IQ server got crashed at "slib/s_bufman.cxx:3436" with stacktrace while running a load table command.

0000025481 **************************************************
0000025481 *** SAP IQ Abort:
0000025481 *** From: slib/s_bufman.cxx:3436
0000025481 *** Rethrown from: oslib/hos_throw.cxx:313
0000025481 *** PID: 65069
0000025481 *** Exception Location: 0x0x7f1de11f73d0
0000025481 *** Error ID: 2055, Exception Type: s_bufman_exception
0000025481 *** Thread: 139766307731200 (TID: 1114, Original TID: 1114)
0000025481 *** Error #: 2, O/S Error #: 0
0000025481 *** Argument:
-- (slib/s_bufman.cxx 3436)
0000025481 *** Argument: main Bufman
0000025481 *** Argument: unknown buffer
0000025481 *** Argument: unknown page
0000025481 **************************************************

2. Abort stacktrace.

===== Thread Number 139766307731200 (IQ connID: 0000025481) =====
hos_ABORT(............
Throw(............
s_bufpartition::Find(............
s_blockmap::FindAndMakeDirtiable(............
hs_btCache::FGetFromDisk(............
hs_btCache::FFind(............
s_bt::BTFFindEQPREV(............
s_bt::Find(............
s_BTCursorAndBuf::FindPage(............
s_mergeFINALWriter::DoInsert()+............
hs_dpInsert::Insert(s_ColumnVector............
dfiio_FP::FetchOpus(............
dfo_IndexInsert::FetchOpus(............
df_CombinerFetchContext::Fetch(............
hc_dfrun::FillRun(............
hc_dmfrun::InitNextWorkUnit(............
hc_dmfrun::ExecWork(............

3. Before the IQ server got aborted, there were lots of error messages below logged regarding to 'main Bufman' as below.

1) 'main Bufman: Incorrect page header read'
=========================================================================================================
0000018875 {139766603532032} main Bufman: Incorrect page header read; from slib/s_trycatch.cxx:2928
0000018875 {139766603532032} expected: btype=13,pbn=4452698495,txn=23606690,sp=1,nlb=16,npb=3,iscmp=1,uid=290435,blk=1098,mn=9996,upmn=9996,flgs=0x80000018
0000018875 {139766603532032} head: btype=17,pbn=4452698495,txn=23489227,sp=1,nlb=16,npb=3,iscmp=0,ctr=131,cta=131,ver=1,pb=1,pus=46964
0000018875 {139766603532032} buffer: 0x11001000030083837fc9660901000000cb6a66010000000001000000010174b7
0000018875 {139766603532032} Expected Pad1: 131 (0x83), Pad2: 1098 (0x044a)
0000018875 {139766603532032} Got Pad1: 1 (0x01), Pad2: 46964 (0xb774)
0000018875 There was an s_buf::Read() error during s_bufman::Find().
0000018875 Exception Thrown from slib/s_blockmap.cxx:6578, Err# 0, tid 833 origtid 833
0000018875 O/S Err#: 0, ErrID: 2100 (s_corruptdiskblkhdrexception); SQLCode: -1009039, SQLState: 'QSA39', Severity: 14
0000018875 [20060]: main Bufman: An error was detected on a database page. You may have a damaged index. For additional information, please check your IQ message file or run sp_iqcheckdb.
-- (slib/s_blockmap.cxx 6578)

2) An IO error was encountered while reading a database page in XXX DBSpace.
=========================================================================================================
0000019709 There was an s_buf::Read() error during s_bufman::Find().
0000019709 The following exception is from GetBlockmapEntryInfo
0000019709 Blockmap Member Variables for 0x7f1fd5a66218
0000019709 From slib/s_dpidx.cxx:590
0000019709 _nextPartitionForHelperFlush = 0
............................................
............................................
0000019709 {139766496159488} Expected Pad1: 171 (0xab), Pad2: 941 (0x03ad)
0000019709 {139766496159488} Got Pad1: 0 (0x00), Pad2: 0 (0x0000)
0000019709 There was an s_buf::Read() error during s_bufman::Find().
0000019709 Exception Thrown from slib/s_blockmap.cxx:5782, Err# 0, tid 935 origtid 935
0000019709 O/S Err#: 0, ErrID: 2094 (s_readexception); SQLCode: -1009044, SQLState: 'QSA44', Severity: 23
0000019709 [20065]: An IO error was encountered while reading a database page in XXX DBSpace. Please check your IQ message file for additional information.
-- (slib/s_blockmap.cxx 5782)
............................................
............................................
0000019709 ** Non-Fatal IQ Internal Error Detected
0000019709 ** at /remote/rip/iqbuild/views/links/r16.0.0_sp11_git_linux_64/gitsrc/cm/iqa/include/s_unitList.h:467 on thread 139766726694656 (TID 716)
..................
..................
0000019709 ** Command text:
select "TAB001"."COL001"(:p0,:p1,:p2)
0000019709
***************** This is the STACKTRACE ***************

===== Thread Number 139766726694656 (IQ connID: 0000019709) =====
0000019709 pc: 0x7f1e14878d42 pcstkwalk(.................................
0000019709 pc: 0x7f1e148793b1 ucstkgentrace(.................................
0000019709 pc: 0x7f1e1487af14 StackTraceForThisThread(.................................
0000019709 pc: 0x7f1e1528e6d7 v2s_bmcontext::bmlfblkget(.................................
0000019709 pc: 0x7f1e152d2e12 v2s_bmcontext::bmint(.................................
0000019709 pc: 0x7f1e14953f33 s_bm::And(.................................
0000019709 pc: 0x7f1e14f5ff30 s_index::EvaluateNulls(.................................
0000019709 pc: 0x7f1e14ab93c5 hs_dp::DeleteInternal(.................................
0000019709 pc: 0x7f1e14ace3f1 hs_dp::Delete(.................................
0000019709 pc: 0x7f1e154d1b1b hdb_delete_indices::Unit_of_Work(.................................
0000019709 pc: 0x7f1e154d26e1 hdb_delete_indices::hdb_delete_index_iter::ExecWork(.................................
=========================================================================================================

4. Tried to run DBCC against the database level and a suspect table, but it aborted with stacktraces as well.

1) call "sp_iqcheckdb"('check dbspace XXX')
0000000020 pc: 0x7f699a466d42 pcstkwalk(.................................
0000000020 pc: 0x7f699a4673b1 ucstkgentrace(.................................
0000000020 pc: 0x7f699a468f14 StackTraceForThisThread(.................................
0000000020 pc: 0x7f699ae7c6d7 v2s_bmcontext::bmlfblkget(.................................
0000000020 pc: 0x7f699aea49c1 v2s_bmcontext::bmver(.................................
0000000020 pc: 0x7f699aeed191 v2s_bmcontext::bmverify(.................................
0000000020 pc: 0x7f699aeece91 s_bm::GatherStatistics(.................................
0000000020 pc: 0x7f699a6b8fe7 hs_dp::GatherStatistics(.................................
0000000020 pc: 0x7f699b03c67c db_IndexDef::GatherStatistics(.................................

2) call "sp_iqcheckdb"('check table OWNER001.TAB002')
0000000062 pc: 0x7f8da744ed42 pcstkwalk(.................................
0000000062 pc: 0x7f8da744f3b1 ucstkgentrace(.................................
0000000062 pc: 0x7f8da7450f14 StackTraceForThisThread(.................................
0000000062 pc: 0x7f8da751b811 sbm_exception::sbm_exception(.................................
0000000062 pc: 0x7f8da7ed500d s_bm::GatherStatistics(.................................
0000000062 pc: 0x7f8da76a0fe7 hs_dp::GatherStatistics(.................................
0000000062 pc: 0x7f8da802467c db_IndexDef::GatherStatistics(.................................

3) call "sp_iqcheckdb"('check table OWNER001.TAB002')
0000000083 pc: 0x7f5770799e6f SigHndlr+0x4f
0000000083 pc: 0x378d40f7e0
0000000083 pc: 0x7f576fcb299e s_TempBufPool::Pop()+0xde
0000000083 pc: 0x7f576fdcb599 s_buf::Read(.................................
0000000083 pc: 0x7f576fde5efa s_bufpartition::Find(.................................
0000000083 pc: 0x7f576fd1164e s_ROBlockmapCursor::Find(.................................
0000000083 pc: 0x7f576fcb742d s_bArrayCursor::Find(.................................
0000000083 pc: 0x7f576fcecfa2 s_bArrayLocater::LoadBlock(.................................
0000000083 pc: 0x7f576fcf5364 s_bArrayLocater::DbccCheckRidMap(.................................

IQ Version: SAP IQ 16.0 SP11.09
OS info: Enterprise Linux64 - x86_64 - 2.6.18-194.el5

SAP IQ, IQ, crash, slib/s_bufman, s_bufpartition::Find, s_blockmap::FindAndMakeDirtiable, hs_btCache::FGetFromDisk, DML_Options8, CR 809726, CR#809726, CR809726 , KBA , BC-SYB-IQ , Sybase IQ , Problem