3356092 - "found malformed host name in SNI extension"

In a scenario where there is an external component (e.g. a gateway, proxy, firewall, etc.) in front of the SAP Web Dispatcher or the ICM, it is not possible to connect to the backend system.

When checking the SAP Web Dispatcher or the ICM trace file, the following entries can be seen:

Image/data in this KBA is from SAP internal systems, sample data, or demo systems. Any resemblance to real data is purely coincidental.

• [Thr 6708] Wed Jul 12 16:07:46:835 2023
  [Thr 6708]   SSL_get_state()==0x1111 "TLS read client hello B"
  [Thr 6708] *** ERROR during secussl_read() from SSL_read()==SSL_ERROR_SSL
  [Thr 6708]   srv SSL session PSE "D:\usr\sap\<SID>\<instance>\sec\SAPSSLS.pse"
  [Thr 6708]   session ciphersuites=545:PFS:HIGH::EC_P256:EC_HIGH
  [Thr 6708]   Server SSL_CTX 000002B7A9FFBEF0 pvflags=513 (TLSv1.2,BC)
  [Thr 6708]   TLSextSNI server_name="<faulty hostname>"
  [Thr 6708] secussl_read: SSL_read() failed  (1535/0x000005ff)
  [Thr 6708]    => "The operation is not supported"
  [Thr 6708] >> ---------- Begin of Secu-SSL Errorstack ---------- >>
  [Thr 6708] 0x000005ff | SAPCRYPTOLIB | SSL_read
  [Thr 6708] SAPCRYPTO API error
  [Thr 6708] The operation is not supported
  [Thr 6708] 0xa0600015 | SSL_ | ssl23_read
  [Thr 6708] The operation is not supported
  [Thr 6708] 0xa0600015 | SSL_ | ssl3_accept
  [Thr 6708] The operation is not supported
  [Thr 6708] 0xa0600015 | SSL_ | ssl3_send_alert
  [Thr 6708] The operation is not supported
  [Thr 6708] Unknown SSL/TLS alert description
  [Thr 6708] 0xa0600294 | SSL_ | ssl3_get_client_hello
  [Thr 6708] found malformed host name in SNI extension
  [Thr 6708] 0xa0600294 | SSL_ | ssl3_decode_client_hello
  [Thr 6708] found malformed host name in SNI extension
  [Thr 6708] 0xa0600294 | SSL_ | ssl3_evaluate_client_hello
  [Thr 6708] found malformed host name in SNI extension
  [Thr 6708] 0xa0600294 | SSL_ | ssl_evaluate_clienthello_tlsext
  [Thr 6708] found malformed host name in SNI extension
  [Thr 6708] 0xa0600294 | SSL_ | parse_and_handle_extension
  [Thr 6708] found malformed host name in SNI extension
  [Thr 6708] 0xa0600294 | SSL_ | ssl_parse_ext_server_name
  [Thr 6708] found malformed host name in SNI extension
  [Thr 6708] 0xa0600294 | SSL_ | tls_check_host_name
  [Thr 6708] found malformed host name in SNI extension
  [Thr 6708] TLS extension with invalid hostname format: 
  [Thr 6708] << ---------- End of Secu-SSL Errorstack ----------
  [Thr 6708]   SSL NI-hdl 84: local=<SAP Web Dispatcher or ICM IP address>:<HTTPS port>  peer=<external component IP address>:<port number>
  [Thr 6708] <<- ERROR: SapSSLSessionStartNB(sssl_hdl=2b7abeb6390)==SSSLERR_SSL_READ
  [Thr 6708] *** ERROR => IcmConnInitServerSSL: SapSSLSessionStartNB returned (-58): SSSLERR_SSL_READ [icxxconn.c   2446]

ABAP Platform all versions.

SAP NetWeaver ABAP all versions.

SAP NetWeaver Java all versions.

SAP Web Dispatcher all versions.

SSL_ERROR_SSL, SSSLERR_SSL_READ, TLSextSNI server_name, The operation is not supported, Unknown SSL/TLS alert description, found malformed host name in SNI extension, TLS extension with invalid hostname format , KBA , BC-CST-IC , Internet Communication Manager , BC-CST-WDP , Web Dispatcher , Problem