Symptom
Suddenly SMP MBO nodes restarted automatically. After the restart both MBO nodes work fine but the secondary one is showed in the Cockpit as STOPPED. The SMP log files and the event viewer are full of error messages like:
JZ0BE: BatchUpdateException: Error occurred while executing batch statement: SQL Anywhere Error -638: Right truncation of string data
com.sybase.djc.transaction.TransactionRolledbackSystemException: com.sybase.djc.transaction.TransactionContext.*AnonymousTransaction*
WARN Cluster Thread-11 [com.sybase.sup.server.cluster.ClusterUtils] [] [] Cluster primary changed. oldServer= <serverName1> newServer=<serverName2>
com.sybase.sup.server.dcn.DCNException was thrown by method com.sybase.sup.server.dcn.DCNSecurityFactory not loaded or implemented in the target webapp.\ncom.sybase.sup.server.dcn.DCNException: Package properties not found for package <packageName>. in domain default
E. <141293> [-10374] Aborting request that started during shutdown
E. <139646> [-10279] Connection was dropped due to lack of network activity
E. <Backend-0001> OEE1051: The Outbound Enabler was unable to access the session with ridx=11 sidx=23 snum=1314 sfp=<sfp> on a RS_CLI_SESSION_BEGIN packet due to cannot connect to backend server
E. <OEMaster> OEE1033: The Outbound Enabler is unable to connect to the backend server at <Hostname>:<Port> to ping it. Pinging again...
Index #[0] Message: [Communication error] NativeError: [-85] Source: [SAOLEDB.16] SQLState: [08S01]|||||10160
(x-sup-sessid=<sessID>) DeviceMutex failed to acquire lock. Waited 200 seconds to acquire lock for '13396'. Result 258|deviceid=<deviceID>|||8196
(x-sup-sessid=<sessID>) Event Type: ERROR__Object Name: monet:SettingsExchange.dll:SettingsExchange__Method Name: SetServerPropGroup__Error Code: 5643__Error Detail: The server is already processing a request for the tuple: user, device, application. Retry the operation again later.__Source: moRequestBroker.cpp 375.__|<user>|6643AF0BE8C041F0BAC2AF651934B235__RETAILEX|||5808
Load balanced services state set to stop. Stopping finished.|||||8744
Responder got near-fatal error: Index #[0] Message: [Database server not found] NativeError: [-100] Source: [SAOLEDB.16] SQLState: [08001]|||||11016
GenericExeController detected service process for service 'MlsrvWrapper' has terminated abnormally. Restarting it...|||||264820170120T193512.672|1|Node <NodeName> is becoming controller|||||11844
Service not running on local node skipping pause. Status = StopPending|||||11844
Marking node 5 offline|||||2244
Nodelist changed. Adjusting last rebalance time to <time>|||||2244
20170120T195040.314|1|ERROR. Controller status got reset by someone. |||||9836
Stopping LBManager|||||9608
Stopping MonitorServers thread for <NodeName>|||||9608
Stopping heartbeat for <NodeName>|||||9608
Starting LBManager|||||12576
Starting heartbeat for <NodeName>|||||12576
Starting server monitor thread for <NodeName>|||||12576
First check to become controller passed. |||||13508
Second check to become controller passed. |||||13508
Node <NodeName> is becoming controller|||||13508
Service was just started. Waiting for it to go to running state.|||||13508
Error pausing local service |||||13508
Error pausing local service. - System.ServiceProcess.TimeoutException: Time out has expired and the operation has not been completed.|||||13508
at System.ServiceProcess.ServiceController.WaitForStatus(ServiceControllerStatus desiredStatus, TimeSpan timeout)|||||13508
at com.iAnywhere.MobileOffice.LBManager.LBManager.PauseLocalService()|||||13508
Nodelist changed. Adjusting last rebalance time to <time>|||||5268
Error pausing local service. - System.InvalidOperationException: Cannot open Service Control Manager on computer '.'. This operation might require other privileges. ---> System.ComponentModel.Win32Exception: A system shutdown is in progress|||||4900
sup.mbs.moca.3 com.sybase.iiop.net.CorbaNetworkException: Connect Failed: 127.0.0.1:2001 ---> System.Net.Sockets.SocketException: No connection could be made because the target machine actively refused it 127.0.0.1:2001
sup.mbs.moca.5 com.sybase.iiop.net.CorbaNetworkException: Stream.Receive ---> System.IO.IOException: Unable to read data from the transport connection: An existing connection was forcibly closed by the remote host. ---> System.Net.Sockets.SocketException: An existing connection was forcibly closed by the remote host|||||4292
System.ComponentModel.Win32Exception: Access is denied|||||4072
at System.Diagnostics.Process.Kill()|||||4072
at com.sybase.sup.server.SUPServerWrapper.KillMlsrvIfExists() in u:\server\jmsbridge\src\cs\com\sybase\sup\server\SUPServerWrapper.cs:line 385|||||4072
Cluster node '<clusterNodeName>' was removed from the active failover cluster membership. The Cluster service on this node may have stopped.
Miniport NIC 'Microsoft Hyper-V Network Adapter' connected
Miniport NIC 'Microsoft Hyper-V Network Adapter' disconnected
Read more...
Environment
SAP Mobil Platform (SMP) 2.3/ 3.0 MBO Runtime - Cluster Installation
Product
Keywords
smp, restart, automatically, hyper-v, cluster, node, MBO, Runtime , KBA , MOB-ONP-MBO , SAP Mobile On Premise MBO Synchronization , Problem
About this page
This is a preview of a SAP Knowledge Base Article. Click more to access the full version on SAP for Me (Login required).Search for additional results
Visit SAP Support Portal's SAP Notes and KBA Search.