2536642 - Introscope Workstation hanging during authentication and other EM / MOM issues

Symptom

When accessing the Introscope Workstation, during the authentication/ logon phase, it freezes indefinitely.

By following this CA article, it is possible to set the workstation logs to debug in order to gather more data when reproducing the issue. The logs in debug will then present the following error message reported many times:

1/17/17 02:16:56 AM CST [DEBUG] [Workstation.PostOfficeHub] Notifying transport Loopback PO: Workstation_<xxxx>.client_main that route Post Office: Agent_<xxxxx>.client_main, 2 hops via Socket Transport connected with Socket[addr=/<xx.xxx.xxx.xxx>,port=<xxxx>,localport=<xxxx>] is down

The following errors might be also be found in the Enterprise Manager logs:

... loadbalancing (?): [Manager.LoadBalancer] LoadBalancerBean: getCollectorReconnectList(...) failed to determine MOM socket port for socketType=default and # of entries in fMOMSocketInfos=0; using default momPort=

and

9/12/17 12:21:56.132 PM UTC [ERROR] [Dispatcher 1] [Manager.IncomingMessageDeliveryTask] Cannot deserialize message while reading from node at: Socket Transport connected with Socket[addr=<host>/<IP>,port=<PORT>,localport=<PORT>]
com.wily.isengard.postofficehub.link.v1.CacheCorruptionException: Internal cache is corrupt. Cannot determine class type for Object 0. A prior class deserialization error may have corrupted the cache. And fStreamState.fCachedObjects.containsKey(id) returned:false
at com.wily.isengard.postofficehub.link.v1.IsengardObjectInputStream.readClass(IsengardObjectInputStream.java:529)

...

The MOM thread dump will show all Outgoing Delivery threads stuck on Nio writing as in the following error:

...
"Outgoing Delivery 10" prio=10 tid=0x00007fdef800a000 nid=0x19cf runnable [0x00007fdf3776d000]
java.lang.Thread.State: RUNNABLE
at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:228)
at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:81)
at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87)
- locked <0x000000042b2297d0> (a sun.nio.ch.Util$2)
- locked <0x000000042b2297c0> (a java.util.Collections$UnmodifiableSet)
- locked <0x000000042b2297e0> (a sun.nio.ch.EPollSelectorImpl)
at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98)
at com.wily.isengard.postofficehub.link.v1.server.OutgoingMessageDelivererNio.waitForWriteNotification(OutgoingMessageDelivererNio.java:211)
at com.wily.isengard.postofficehub.link.v1.server.OutgoingMessageDelivererNio.run(OutgoingMessageDelivererNio.java:155)
- locked <0x00000003cc5a28a0> (a com.wily.isengard.postofficehub.link.v1.server.OutgoingMessageDelivererNio)
at com.wily.util.concurrent.SetExecutor.doWork(SetExecutor.java:224)
at com.wily.util.concurrent.SetExecutor.access$0(SetExecutor.java:178)
at com.wily.util.concurrent.SetExecutor$WorkerRequest.run(SetExecutor.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:722)
...

In the managed system configuration, the following error might be seem in the Introscope Host Adapter activity log:

On instance : <Instance>
Introscope Host Adapter configuration finished without errors, but on Enterprise Manager (<host>:<port>) no Introscope Host Adapter was detected from host <host>. Please check the log file jvm_smdagent.out whether the Wilyhost Agent could connect to the Enterprise Manager. If the Enterprise Manager configuration has been changed recently please restart the SMD Agent on OS level.

The following symptoms might also be observed:

Agent connection issues
Poor Performance
Problems to logon via workstation(intermittent behaviour)
Communication between WebView and EM is slow
Low CPU consumption
Slow response time for the Enterprise Manager UI after restart

Environment

SAP Solution Manager 7.1
SAP Solution Manager 7.2

Product

SAP Solution Manager 7.1 ; SAP Solution Manager 7.2

Keywords

ava.nio.channels.CancelledKeyException , Notifying transport Loopback PO: Workstation_<xxxx>.client_main that route Post Office: Agent_<xxxxx>.client_main, 2 hops via Socket Transport connected with Socket[addr=/<xx.xxx.xxx.xxx>,port=<xxxx>,localport=<xxxx>] is down , Java New I/O (Java NIO, or NIO) capabilities , Java NIO channels , Java I/O , Introscope Java NIO metrics , transport.enable.nio , webview , workstation EM_HOME/config/IntroscopeEnterpriseManager.properties , KBA , XX-PART-WILY , Introscope by CA Technologies , Problem

About this page

This is a preview of a SAP Knowledge Base Article. Click more to access the full version on SAP for Me (Login required).

Search for additional results

Visit SAP Support Portal's SAP Notes and KBA Search.