3385883 - DB6: SQL1768N Unable to start HADR. Reason code = "99"

Your system is running on database with HADR configuration.

You have finished system migration to new server using the OS image copy.

Afterwards, primary database can not start up due to following error:

SQL1768N  Unable to start HADR. Reason code = "99" 


Database log (db2diag.log) shows: 

2023-10-01-09.00.58.333570+330 I351643323E485        LEVEL: Info
PID     : 20065                TID : 140325498644224 PROC : db2sysc 0
INSTANCE: db2abc               NODE : 000            DB   : abc
APPHDL  : 0-7                  APPID: *LOCAL.db2abc.230930033047
AUTHID  : DB2abc               HOSTNAME: ABCDB
EDUID   : 26                   EDUNAME: db2agent (abc) 0
FUNCTION: DB2 UDB, High Availability Disaster Recovery, hdrEduStartup, probe:21151
MESSAGE : HADR Startup has begun.

2023-10-01-09.00.58.355384+330 I351649499E783        LEVEL: Error
PID     : 20065                TID : 140305433093888 PROC : db2sysc 0
INSTANCE: db2abc               NODE : 000            DB   : abc     
HOSTNAME: ABCDB
EDUID   : 153                  EDUNAME: db2hadrp.0.1 (abc) 0
FUNCTION: DB2 UDB, recovery manager, sqlplfrFindLastFullRecord, probe:9000
MESSAGE : ZRC=0x8710001D=-2028994531=SQLP_LERR "Fatal Logic Error"
          DIA8526C A fatal error occurred in data protection services.
DATA #1 : String, 43 bytes
No log records found but they should exist.
DATA #2 : String, 36 bytes
searchStartLso/LastRecLsoLFH/baseLso
DATA #3 : unsigned integer, 8 bytes
18541990452197
DATA #4 : unsigned integer, 8 bytes
18541990455911
DATA #5 : unsigned integer, 8 bytes
18541438680001

2023-10-01-09.00.58.363566+330 I351650283E714        LEVEL: Error
PID     : 20065                TID : 140305433093888 PROC : db2sysc 0
INSTANCE: db2abc               NODE : 000            DB   : abc     
HOSTNAME: ABCDB
EDUID   : 153                  EDUNAME: db2hadrp.0.1 (abc) 0
FUNCTION: DB2 UDB, High Availability Disaster Recovery, hdrFindAndSetLastFullRecForArcuOrCR, probe:25010
MESSAGE : ZRC=0x8710001D=-2028994531=SQLP_LERR "Fatal Logic Error"
          DIA8526C A fatal error occurred in data protection services.
DATA #1 : <preformaabcd>
Failed to find the last full log record for offline inconsistent
assisted member.  Assisted remote catchup cannot be performed.
Scan start position: 18541990452197

<snipped>

2023-10-01-09.00.58.379742+330 I351656085E573        LEVEL: Error
PID     : 20065                TID : 140325498644224 PROC : db2sysc 0
INSTANCE: db2abc               NODE : 000            DB   : abc
APPHDL  : 0-7                  APPID: *LOCAL.db2abc.230930033047
AUTHID  : DB2abc               HOSTNAME: ABCDB
EDUID   : 26                   EDUNAME: db2agent (abc) 0
FUNCTION: DB2 UDB, High Availability Disaster Recovery, hdrSpawnPrimaryEdus, probe:23300
MESSAGE : ZRC=0x8780013F=-2021654209=HDR_ZRC_STARTUP_INTERNAL_ERROR
          "Internal error during HADR startup"

2023-10-01-09.00.58.380157+330 I351657226E569        LEVEL: Error
PID     : 20065                TID : 140325498644224 PROC : db2sysc 0
INSTANCE: db2abc               NODE : 000            DB   : abc
APPHDL  : 0-7                  APPID: *LOCAL.db2abc.230930033047
AUTHID  : DB2abc               HOSTNAME: ABCDB
EDUID   : 26                   EDUNAME: db2agent (abc) 0
FUNCTION: DB2 UDB, High Availability Disaster Recovery, hdrEduShutdownOnSpawningError, probe:25300
DATA #1 : <preformaabcd>
HADR startup has been interrupted.  Shutting down the spawned HADR EDU now.

2023-10-01-09.00.58.380810+330 I351659759E561        LEVEL: Error
PID     : 20065                TID : 140325498644224 PROC : db2sysc 0
INSTANCE: db2abc               NODE : 000            DB   : abc
APPHDL  : 0-7                  APPID: *LOCAL.db2abc.230930033047
AUTHID  : DB2abc               HOSTNAME: ABCDB
EDUID   : 26                   EDUNAME: db2agent (abc) 0
FUNCTION: DB2 UDB, High Availability Disaster Recovery, hdrEduStartup, probe:21250
MESSAGE : SQL1768N  Unable to start HADR. Reason code = "99".  
DATA #1 : ZRC, PD_TYPE_ZRC, 4 bytes
0x8780013F

Reason code 99 is mapped to "Internal error during HADR startup".

SAP on IBM Db2 for Linux, UNIX, and Windows

HADR, SQL1768N, 99, DIA8526C, Internal error, startup , KBA , BC-DB-DB6 , DB2 Universal Database for Unix / NT , Problem