Oracle Exalogic Elastic Cloud X2-2 Qtr Rack: The Connection Between Gateway Switch And External Switch Is Lost

Asset ID:	1-72-1378600.1
Update Date:	2012-07-16
Keywords:

Solution Type Problem Resolution Sure

Solution 1378600.1 : Oracle Exalogic Elastic Cloud X2-2 Qtr Rack: The Connection Between Gateway Switch And External Switch Is Lost

Applies to:

Oracle Exalogic Elastic Cloud Software - Version 1.0.0.0.0 to 2.0.1.0.0
Oracle Exalogic Elastic Cloud X2-2 Qtr Rack - Version Not Applicable to Not Applicable [Release N/A]
Oracle Solaris on x86-64 (64-bit)

Symptoms

This Document applies only to customer who have firmware 1.1.2-2 in combination with a data centre cooling anomalies

Datacentre Air-conditioner failure can lead to the increase in the temperature in the room where Exalogic is installed.
After the air-conditioner power was restored the two Sun Gateway switches may keep giving errors and the connection between one of them and the external switch reported as lost.

These are the errors in the logs:

ID = 6 : 10/11/2011 : 08:02:44 : OEM sensor : CHASSIS_STATUS : State Asserted

ID = 7 : 10/11/2011 : 08:03:04 : OEM sensor : CHASSIS_STATUS : State Deasserted

ID = 13 : 10/13/2011 : 09:59:22 : OEM sensor : CHASSIS_STATUS : State Asserted

After a reboot, these errors may still pop up.

Cause

The mentioned alerts 'OEM sensor : CHASSIS_STATUS' on the management system even after the switches were rebooted, while the switch does not log any more messages.

The system is running with previous switch firmware 1.1.2-2 and seeing some issues here that are fixed in newer firmware:

[root@exagw01 ~]# /usr/local/bin/version
SUN DCS gw version: 1.1.2-2

[root@exagw02 ~]# /usr/local/bin/version
SUN DCS gw version: 1.1.2-2

New firmware version(s) available:

- <Patch 12353972> VERSION 1.3.2 OF SUN DCS QDR SWITCH (Patch) 1.3.2

6982832 Event forwarding subscriptions not cleaned up properly

7001646 Log filesystem gets full and causes system outages
/var/log filesystem can get 100% full with various log files and primary functions may become unavailable due to this.
-> could be t he reason why connection has been lost

7017412 130 - No Master SM seen in fabric during failover
-> Oct 13 09:56:50 qnlexagw02 whereismaster[2505]: No Master SubnetManager seen in the system

6998891: Fix for 6849329 does not fully follow recommendations from Analog Devices
-> The fix for 6849329 ADM1026 Erroneous readings on Temperature and Voltage Channels.

Solution

To resolve this issue, we must try the following

First power down/on the switch, for ex: exagw01 (currently SM standby)
After the switch (exagw01) is up & running again do the same with the other switch, for ex: exagw02
Then plan the upgrade to firmware 1.3.2 or better 1.3.3_1 as part of the Exalogic Infrastructure patch.

References

@ <BUG:7001646> - EM R2 : ADAS0170: HOST UNREACHABLE
@ <BUG:7017412> - QBFPP:WHEN RULE DELETED IN TREE BINDING EDITOR ACCESSOR NOT DELETED
@ <BUG:6998891> - 508CQSV: MAGIC: DYNAMIC LENS MAGNIFY GRID INCORRECTLY ON PREVIEW PAGE
@ <BUG:6982832> - CPUAPR2008: MERGE LABEL REQUEST ON TOP OF 10.1.2.2 FOR BUGS 6864151 6016022

Attachments

This solution has no attachment