2786441 - R3trans connect fails with SQL error 'SQL5005C'

SAP system startup fails.  You observe 'R3trans' (in trans.log) is failing with sqlcode SQL5005C.

----------------------------------------------------------------------------------------------------------------------------------------
4 ETW000 [ dev trc,00000] DB2Trc: dbs/db2/dsn_alias is set (XXX) -> we use the DB2 Connect fail over feature
4 ETW000 [ dev trc,00000] Set _cliFeatures.sqlGetDiagFieldBytes = TRUE
4 ETW000 [ dev trc,00000] DB2Trc: 000000 CLI_ALLOC_ENV 1
4 ETW000 [ dev trc,00000] Could not open db2applcompat.ini file '/sapmnt/XXX/global/db2applcompat.ini'.
4 ETW000 [ dev trc,00000] DB2Trc: Application Compatibility ignored.
4 ETW000 [ dev trc,00000] Using user(xxxadbc) and password(<pwd>) from password service.
4 ETW000 [ dev trc,00000] DB2Trc: trace level of dbdb2cli set to 1
4 ETW000 [ dev trc,00000] COLLECTION ID used is "SAP1101U"
4 ETW000 [dbdb2cli.c ,00000] *** ERROR => DB2 Call 'SQLConnectW' Error: conHdl = 00 SQLCODE = -5005 : [IBM][CLI Driver] SQL5005C The operation failed because the da
tabase manager failed to access either the database manager configuration file or the database configuration file.
4 ETW000 [dbdb2s.c ,00000] *** ERROR => CONNECT failed with SQL error '-5005'
4 ETW000 [ dblink ,00000] ***LOG BY2=>sql error -5005 performing CON
4 ETW000 [ dblink ,00000] ***LOG BY0=>[IBM][CLI Driver] SQL5005C The operation failed because the database manager failed to access either the database manager c
onfiguration file or the database configuration file.
4 ETW000 [dbdb2cli.c ,00000] *** ERROR => DB2 Call 'SQLGetInfoW' Error: conHdl = 00 SQLCODE = -99999 : invalid connection handle 0
2EETW169 no connect possible: "DBMS = DB2/MVS --- "
----------------------------------------------------------------------------------------------------------------------------------------

Further analysis using truss (or linux strace) shows that R3trans execution is failing due to an unavailable resource.  Problem is pointing to the 'db2dsdriver.cfg' file.

----------------------------------------------------------------------------------------------------------------------------------------
open("/usr/sap/XXX/D50/exe/db2_clidriver/conv/ucs20819.cnv", O_RDONLY) = -1 ENOENT (No such file or directory)
open("/usr/sap/XXX/D50/exe/db2_clidriver/conv/IBM00819.ucs", O_RDONLY) = -1 ENOENT (No such file or directory)
access("/sapmnt/XXX/global/db2dsdriver.cfg", F_OK) = 0
open("/sapmnt/XXX/global/db2dsdriver.cfg", O_RDONLY) = 4
fcntl(4, F_SETLK, {type=F_RDLCK, whence=SEEK_SET, start=0, len=0}) = -1 EAGAIN (Resource temporarily unavailable)
fcntl(4, F_GETLK, {type=F_UNLCK, whence=SEEK_SET, start=0, len=0, pid=0}) = 0
close(4) = 0
open("/usr/sap/XXX/D50/exe/db2_clidriver/msg/en_US.iso88591/db2sql.mo", O_RDONLY) = 4
fstat(4, {st_mode=S_IFREG|0777, st_size=443901, ...}) = 0
mmap(NULL, 446464, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f4d6a1ae000
read(4, "\0\0", 2) = 2
read(4, "\330\261\0\0", 4) = 4
read(4, "\0\0\0\0\0\0\0\0T\0\0\0\1\0\0\0/\0\0\0T\0\0\0\2\0\0\0r\0\0\0"..., 443895) = 443895
close(4) = 0
write(3, "4 ETW000 [dbdb2cli.c ,00000] "..., 283) = 283
write(3, "4 ETW000 [dbdb2s.c ,00000] "..., 83) = 83
write(3, "4 ETW000 [ dblink ,00000] "..., 76) = 76
write(3, "4 ETW000 [ dblink ,00000] "..., 222) = 222
write(3, "4 ETW000 [dbdb2cli.c ,00000] "..., 134) = 134
write(3, "2EETW169 no connect possible: \"D"..., 52) = 52
write(1, "2EETW169 no connect possible: \"D"..., 522EETW169 no connect possible: "DBMS = DB2/MVS --- "
) = 52
write(1, "R3trans finished (0012).\n", 25R3trans finished (0012).
----------------------------------------------------------------------------------------------------------------------------------------

• IBM DB2 z/OS
• Network File System v3
• Netweaver AS

-5005 SQL5005C mainframe db2cli.ini connect.ini DB2Connect , KBA , BC-DB-DB2 , DB2 for z/OS , Problem