VTL - HA Nodes Appear to Have Failed Over to Each Other

Asset ID:	1-72-1014113.1
Update Date:	2011-07-28
Keywords:

Solution Type Problem Resolution Sure

Solution 1014113.1 : VTL - HA Nodes Appear to Have Failed Over to Each Other

Applies to:

Sun StorageTek VTL Plus Storage Appliance - Version: 1.0 - Build 1323 to 2.0 - Build 1656 - Release: 1.0 to 2.0
Sun StorageTek VTL Storage Appliance - Version: 4.0 - Build 1221 to 4.0 - Build 1221 [Release: 4.0 to 4.0]
All Platforms
.
***Checked for relevance on 19-07-2011*** (dd-mm-yyyy)

Symptoms

In a VTL High Availability (HA) configuration, both nodes can failover to each
other, causing a "mutual failover" condition.

This is the expected reaction for VTL in an HA config under certain circumstances for
precautionary measures. For example, in a power outage situation this is expected.
Both servers are red in VTL Console indicating it has failed over. Each node can be
servicing the others resources or neither taking control of the other.
Or only one server may be servicing the other's services and the other node may not be due
to the failover not completing completely (this can happen when the one node fails over
completely first, and the second then can't complete the failover).

Changes

Network changes can cause this condition.

Cause

Double failovers can be caused by a number of external conditions including network problems.

Solution

In any of the above scenarios, the following procedure can be used to get back to a "normal" failover state.

To get back to normal failover mode, do the following:

1) Verify both servers are "ready" to failback:

Enter command from server prompt:
# sms -v

Verify FailOverStatus = 2(Ready)

2) Stop failover on Secondary (this will force a takeback of the Primary server). From the Secondary server command prompt:

# vtl stop fm

Note: this may take several minutes to complete the transfer of resources back to Primary server. Once GUI reports Primary is no longer failed over (refer to status line on bottom left corner of GUI or Failover information folder in GUI)

Using sms -v again to verify primary server is fine
It should now show as FailoverStatus=1(UP)

Run ifconfig to make sure heartbeat and virtual IPs are up

3) Stop failover on Primary (this will force a takeback of the secondary server), from the primary server command prompt:

# vtl stop fm

Using sms -v again to verify primary server is fine
It should now show as FailoverStatus=1(UP)

Run ifconfig to make sure heartbeat and virtual IPs are up

4) Start failover modules on both servers:

# vtl start fm

5) Check status of both VTL servers, from server command prompt:

# vtl status (all processes should report )

6) Verify "Normal" failover status in GUI:

From VTL Console (GUI), highlight server and select Failover tab. Status should = "Normal".

Attachments

This solution has no attachment