3395885 - Poor system performance on SAP IQ production on AIX 7.3. - SAP IQ

The performance issue seems to have started after OS upgrade from 7.1 to 7.3 (7300-01-01-2246) in IQ environment.

Typical symptoms were 

- CPU usage is high and mostly in system/kernel mode.
  high %sys usage is evident when lparstat is run in 5 second intervals.

  Output example.

   %user  %sys  %wait  %idle physc %entc  lbusy   app  vcsw phint  %nsp  %utcyc
   ----- ----- ------ ------ ----- ----- ------   --- ----- ----- -----  ------
   17.2  68.6    3.5   10.7  5.14 122.3   64.8 168.57 18163     1   111   9.70 
   17.8  67.3    3.6   11.4  5.14 122.3   62.7 169.55 19222     0   111   9.70 
   16.4  50.7    3.7   29.2  3.53  84.0   38.7 171.52 13499     1   111   9.70 
   18.8  30.8    2.6   47.7  3.07  73.0   24.4 176.78 16192     1   111   9.69 
   23.0  61.5    2.1   13.4  4.46 106.2   54.9 178.42 16590     2   111   9.70 
   19.8  62.4    2.6   15.2  5.20 123.8   56.4 170.90 22423     2   111   9.70 

- Increasing VP ( virtual processors ) does not help and performance worsened.

- Slow hourly incremental backup runtimes. Normal runtime was under 5 minutes. Impacted runtime could exceed 30-40 minutes.

- Monitoring script execution times increased.

- Query usually runs in under 10 seconds, it can run over 40 minutes.

- BW NLS queries running for an extended period of time rather than minutes & caused CPU spikes
  Example NLS query
  ------
  select "T1"."FISCPER",   "T1"."/BIC/ZPCRGACCT",   "T1"."/BIC/ZPCRGLACT",   "T1"."/BIC/ZPCRRFDOC",   "T1"."DOC_ITEM",   "T1"."COMP_CODE",   "T1"."/BIC/ZFDM_DIV",   "T1"."/BIC/ZFDM_MKT",   "T1"."/BIC/ZFDM_FUNC"
  from "/BIC/OUZSLZ4_O4" as "T1" where exists
  (select 1 from "/BIC/OMTLT_2023091413181854685" as "T2"
    where "T1"."COMP_CODE" = "T2"."COMP_CODE"     and "T1"."FISCPER" = "T2"."FISCPER"     and "T1"."/BIC/ZPCRRFDOC" = "T2"."REF_DOC_NO")
  group by "T1"."FISCPER",   "T1"."/BIC/ZPCRGACCT",   "T1"."/BIC/ZPCRGLACT",   "T1"."/BIC/ZPCRRFDOC",   "T1"."DOC_ITEM",   "T1"."COMP_CODE",   "T1"."/BIC/ZFDM_DIV",   "T1"."/BIC/ZFDM_MKT",   "T1"."/BIC/ZFDM_FUNC"
/* R3:CL_SQL_STATEMENT==============CP:500 T:/BIC/OUZSLZ4_O4 */
  -----

• SAP IQ 16.1 SP05.04
• AIX 7.3 oslevel 7300-01-01-2246

AIX 7.3 , performance , KBA , BC-SYB-IQ , Sybase IQ , Problem