3284630 - Attempts to restart IQ server, iqdemo, utility_srv or even run mkiqdemo.sh script all hang - SAP IQ

• Unable to restart the SAP IQ server after a previous clean shutdown.
  • Server process becomes stuck and does not respond.
  • Trying to run a IQ RESTORE from a virtual backup, the utility db server also hangs on startup.
  • After successful restore to a full OMBS disk image backup, the IQ server is still unable to boot.
  • The initial restart attempt of the IQ server appeared to hang at the point after these messages appear in the iqmsg file:
    st_database::CompleteMpxRecovery() - change Recovery State from RECOVERED to RECOVERING
    Incremented UID on s_bufman (0x7fc2f4202528)temp Bufman by 1048576 to get 70041721
  • However subsequent restart attempts failed to even get this far.
  • A full reboot of the host OS machine does not resolve the problem.
  • No corresponding errors found in the OS message logs.
• The startup does not get as far as writing any messages to either the .srv (server log) nor the .iqmsg file, but initial startup messages are seen in the std err console log.
• Attempts to even run the mkiqdemo.sh script to create an iqdemo database hangs.
  
  • The same startup hang is seen even if start and iqdemo database is tried.
  • Attempts to take pstacks of the utility_srv startup attempt may show two processes, one a dbspawn the other a iqsrv16
  • Attempts to try take a pstack of these processes fails all the time for the iqsrv16 process but on occasions works for dbspawn (by fail it means the pstack <PID> returns nothing)
• Sometimes when trying to KILL the hung iqsrv16 process it does not fully "go away"
  • ps shows a <defunct> state. 
  • Only a reboot of the host seems to clear it.

• SAP IQ 16.0 any version
• SAP IQ 16.1 any version

sybase, block, stuck, unresponsive, failed, upgrade, backup, abort , KBA , BC-SYB-IQ , Sybase IQ , How To