3409476 - Error message "Error while calling a servlet on the JAVA Application Server" in step 'Define HTTP Connectivity'

"Test Connectivity" in step 'Define HTTP Connectivity' reports the following error messages when configuring with JAVA HTTPS links:

Error while calling a servlet on the Java Application Server
Message No. SISE_BASIS009
Diagnosis
The call to the servlet https://<SolmanJavaHost>:<SolmanJavaPort>/smd/upgrade/JavaSslPortCheck returned no answer
This may be because the URL is not reachable or if there is no trust relationship between the ABAP Technical System and JAVA Technical System.
Procedure
Make sure that the URL is reachable and SSL is correctly configured on ABAP and Java technical systems. 

Accessing the mentioned link https://<SolmanJavaHost>:<SolmanJavaPort>/smd/upgrade/JavaSslPortCheck from the web browser gets errors like "500 internal server error". In the meanwhile, the dev_icm trace reports the detailed errors:

[Thr 140191715403520]    => "received a fatal TLS bad certificate alert message from the peer"
...
[Thr 140191717517056] <<- ERROR: SapSSLSessionStartNB(sssl_hdl=XXXXXXXXXXX)==SSSLERR_ALERT_BAD_CERTIFICATE
[Thr 140191717517056] *** ERROR => IcmConnInitServerSSL: SapSSLSessionStartNB returned (-123): SSSLERR_ALERT_BAD_CERTIFICATE [icxxconn.c   2021]

SAP Solution Manager 7.2

KBA , SV-SMG-INS-CFG , Setup and Configuration of the Solution Manager system , Problem