Symptom
After upgrading from IQ16.1 SP02.11 to IQ16.1 SP04.10, a Writer node hit Dynamic collision and Emergency shutdown. After the error on the Writer, CN encountered corruptions with "main Bufman: Incorrect page header read" appear and went to crash.
Before the upgrade, customer checked the database and there were no errors. After the upgrade, the database worked fine for a day, then the node crashed with an error: MPX dynamic collision. As a result, found corruptions on objects. Customer had to restore DB from a backup.
Following is sequences of events by chronological order;
1- Writer hits a dynamic collision and an Emergency Shutdown was required:
...
I. 07/17 11:17:49. 0000125922 -- Too many retries on dropping user connections. Server is shutting down.
I. 07/17 11:17:49. 0000125922 ***Emergency Shutdown***
I. 07/17 11:17:49. 0000125922 I. 07/17 11:17:49. Exception Thrown from dblib/db_iqutility.cxx:5351, Err# 46, tid 34 origtid 34
I. 07/17 11:17:49. 0000125922 O/S Err#: 0, ErrID: 1025 (db_catalogException); SQLCode: -1009413, SQLState: 'QCB11', Severity: 23
I. 07/17 11:17:49. 0000125922 [21051]: Secondary server cannot maintain catalog sync.
-- (dblib/db_iqutility.cxx 5351)
2- CN hits corruption
...
I. 07/17 11:50:58. 0000101953 [QMap] s_dbmvbio::DeAllocate(0x7fe1285df618): Freelist blocks are already deallocated. beforeCount =856193904 afterCount =856193904
...
I. 07/17 11:51:02. 0000102190 \{140609131992832\} main Bufman: Incorrect page header read; from slib/s_trycatch.cxx:3175 into buffer 0x7fe0b1641280
I. 07/17 11:51:02. 0000102190 \{140609131992832\} expected: btype=14,pbn=758364152,txn=3316404817,sp=14,nlb=16,npb=16,...
3- Writer hits corruptions
...
I. 07/17 11:54:51. 0000000945 Txn 3316421769 0 3316410374
I. 07/17 11:54:51. 0000000444 \{139924300134144\} main Bufman: Incorrect page header read; from slib/s_trycatch.cxx:3175 into buffer 0x7f49e0236b00
I. 07/17 11:54:51. 0000000444 \{139924300134144\} expected: btype=14,pbn=377144580,txn=3316410479,sp=7,nlb=16,npb=16,...
4- CN hits corruption again and crash (down)
...
I. 07/17 12:22:09. 0000104520 s_blockmap: in use on destruction
I. 07/17 12:22:09. 0000104520 Blockmap Member Variables for 0x7f42a0fc88b8
I. 07/17 12:22:09. 0000104520 From slib/s_ohcidx.cxx:7178
...
I. 07/17 12:22:09. 0000104520 **************************************************
I. 07/17 12:22:09. 0000104520 *** SAP IQ Abort:
I. 07/17 12:22:09. 0000104520 *** From: slib/s_blockmap.cxx:1399
...
I. 07/17 12:22:09. 0000104520 **************************************************
5- As CN was down, writer lost communication with CN
...
I. 07/17 12:24:09. 0000005036 GThrow sqlcode: -1004011 from dblib/db_catalog.cxx:845 Args: "Database server not found"
I. 07/17 12:24:09. 0000005036 I. 07/17 12:24:09. Exception Thrown from dblib/db_catalog.cxx:845, Err# 5, tid 5 origtid 5
I. 07/17 12:24:09. 0000005036 O/S Err#: 0, ErrID: 12291 (inc_CommandHandlerException); SQLCode: -1004012, SQLState: 'QIA12', Severity: 14
I. 07/17 12:24:09. 0000005036 [21223]: INC communication between coordinator and secondary failed due to : Database server not found
-- (dblib/db_catalog.cxx 845)
6- Writer needs to shutdown itself to avoid inconsistency with the CN
...
I. 07/17 12:32:37. 0000004959 I. 07/17 12:32:37. Exception Thrown from stcxtlib/st_txnMgrn.cxx:603, Err# 5, tid 2 origtid 2
I. 07/17 12:32:37. 0000004959 O/S Err#: 0, ErrID: 12291 (inc_CommandHandlerException); SQLCode: -1006274, SQLState: 'QBC74', Severity: 14
I. 07/17 12:32:37. 0000004959 [22036]: Transaction cannot continue due to unrecoverable INC failure. Roll back current transaction.
-- (stcxtlib/st_txnMgrn.cxx 603)
I. 07/17 12:32:37. 0000004959 GLOBAL ROLLBACK FAILURE
...
I. 07/17 12:32:37. 0000005432 IQ Checkpoint position: 0
I. 07/17 12:32:37. 0000005432 Chk ShutDQPCxt
I. 07/17 12:32:37. 0000005432 Chk
I. 07/17 12:32:37. 0000005432 ChkDone [NumTxnCP: 0]
I. 07/17 12:32:37. 0000005432 IQ Checkpoint block: 1042082
I. 07/17 12:32:37. 0000005432 PostChk
I. 07/17 12:32:37. 0000000000 CloseDatabase
Read more...
Environment
SAP IQ16.1 SP04.X
Product
Keywords
"maintain catalog sync", s_trycatch, s_blockmap, "INC communication", "INC failure" , , KBA , BC-SYB-IQ , Sybase IQ , Problem
About this page
This is a preview of a SAP Knowledge Base Article. Click more to access the full version on SAP for Me (Login required).Search for additional results
Visit SAP Support Portal's SAP Notes and KBA Search.