3064946 - All transactions initiated by the same thread fail with "SQL search error - The connection is closed"

1. While reviewing the application logs, many "SQL search error - The connection is closed" errors are observed.
   
   
2. Using the Visualize tool in Kibana and creating buckets for logs.thread.keyword and kubernetes.pod_name.keyword, you find that the error is only observed on a single thread per pod. It may happen that more than one thread is affected per pod, but it will be evident from the chart that these threads are in the minority.
   
   
   
3. [Optional] Taking one of the results above, you can filter for the specific pod and thread when searching for "SQL search error - The connection is closed" to determine the time of its first occurrence.
   
   
   Similarly, you may do the same when searching for "SQL search error - Connection reset" and find that the time of its first occurrence was only a few seconds earlier.
   
   
4. [Optional] If the hac web application is configured on the affected aspect, debug-level logging on the de.hybris.platform.jdbcwrapper.ConnectionImpl class can be enabled during runtime. This is not a required step in identifying the problem, but it is used here to illustrate more clearly the inconsistency between the physical and logical (pooled) connections.
   
   
   
   

   For this to have the desired effect, it would need to be enabled on the exact pod where the errors were identified.

   3060752 - How to Access a Service Through a Specific Server in Commerce Cloud 

   Once the above is enabled, the logs will start showing additional information along with the error when it resurfaces. For every occurrence of the "connection is closed" error, the value of closed will be false.

   "hybrisHTTP4", "level": "DEBUG", "loggerName": "de.hybris.platform.jdbcwrapper.ConnectionImpl", "message": "[connection:393058,txBound:true,closed:false]:error prearing statememt", "thrown": {     "commonElementCount": 0,     "localizedMessage": "The connection is closed.",     "message": "The connection is closed.",     "name": "com.microsoft.sqlserver.jdbc.SQLServerException",     "extendedStackTrace": [         {             "class": "com.microsoft.sqlserver.jdbc.SQLServerException",             "method": "makeFromDriverError",             "file": "SQLServerException.java",             "line": 234,             "exact": false,             "location": "mssql-jdbc-8.2.1.jre11.jar",             "version": "?"         },         {             "class": "com.microsoft.sqlserver.jdbc.SQLServerConnection",             "method": "checkClosed",             "file": "SQLServerConnection.java",             "line": 1130,             "exact": false,             "location": "mssql-jdbc-8.2.1.jre11.jar",             "version": "?"         },         ...     } ]

SAP Commerce Cloud

hybris,db,jdbc,pool,sql,leak , KBA , CEC-SCC-CDM-BO-SRC , Search , Problem