Sun Storage 7000 Unified Storage System: How to verify that the multipathing software ( MPxIO / VxDMP) is optimal

Asset ID:	1-71-1433014.1
Update Date:	2012-03-12
Keywords:

Solution Type Technical Instruction Sure

Solution 1433014.1 : Sun Storage 7000 Unified Storage System: How to verify that the multipathing software ( MPxIO / VxDMP) is optimal

Applies to:

Sun Storage 7310 Unified Storage System - Version: Not Applicable to Not Applicable - Release: N/A to N/A
Sun Storage 7410 Unified Storage System - Version: Not Applicable to Not Applicable   [Release: N/A to N/A]
Sun ZFS Storage 7320 - Version: Not Applicable to Not Applicable   [Release: N/A to N/A]
Sun ZFS Storage 7420 - Version: Not Applicable to Not Applicable   [Release: N/A to N/A]
Sun Storage 7110 Unified Storage System - Version: Not Applicable to Not Applicable   [Release: N/A to N/A]
7000 Appliance OS (Fishworks)

Goal

How to verify that the multipathing software (MPxIO / VxDMP) used to connect to the 7000 appliance is optimal

Solution

The functionality of FC target devices was added to the Sun 7000 series with the introduction of appliance software 2010.Q1.

The Sun ZFS Storage 7000 series uses Asymmetric Logical Unit Access (ALUA) to provide FC target multipathing support.
Properly configuring an FC initiator in an ALUA environment requires an ALUA-aware driver, and may require initiator-specific tuning. The following initiators have been tested and are known to work in a cluster:

OpenSolaris 2010.03    QLogic, Emulex
Solaris 10 10/09       QLogic, Emulex
Windows 2008           QLogic, Emulex
Red Hat Enterprise Linux 5.4 QLogic, Emulex
Oracle Enterprise Linux 5.4 QLogic, Emulex
VMware ESX 4.0         QLogic, Emulex

Sun Storage Traffic Manager(STMS or MPxIO)

Even though FC target on a clustered appliance was qualified with OpenSolaris 2010.03 and Solaris 10 10/09 ( S10U8 ), users must also apply MPxIO Patch:143120-03 (sparc), Patch:143121-03 (x86) or later to get fixes for
<Bug:6865431> scsi_vhci loses path state awareness under failover/load
<Bug:6915604> path state is not updated after failover success

The above patches are obsoleted by Patch:142909-17 (S10 U9 )

Additionally, install Patch:144575-04 (x86) or Patch:144574-04 (sparc) delivered in S10 U9 Feature KU or S10 U10 Patch:144500-19
to get fix for
<Bug:6959914> - Anago: FC LUN became inaccessible to client during takeover

Further,
S10u11 KU feature Patch:147147-08 (sparc) and Patch:147148-08 (x86) which can be obtained by logging a call with support, provides a fix for

<Bug:7107552> - A flag VLUN_KEEP_HELD conflicts with another two flags
<Bug:6966496> - Paths reported as standby are enumerated as online (active) paths

As far as the appliance is concerned, the latest firmware 2011.1 fixes <Bug:6951173> & <Bug:7043998> which are associated with long failback times on the appliance. But if this is a 7x10 ( SAS1) system, do not upgrade to 2011.1 due to <Bug:7132238> which is yet to be fixed (at the time of writing this document)).

MPxIO is enabled on Solaris x86 platforms but disabled on SPARC by default. The "mpathadm show LU" command shows the path state changing from active to standby or standby to active. Alternately, you can also use luxadm display to show path state.

luxadm display /dev/rdsk/c4t600144F0F8D6944400004E6090280004d0s2
DEVICE PROPERTIES for disk: /dev/rdsk/c4t600144F0F8D6944400004E6090280004d0s2
  Vendor:               SUN
  Product ID:           ZFS Storage 7320
  Revision:             1.0
  Serial Num:
  Unformatted capacity: 3072.000 MBytes
  Read Cache:           Enabled
    Minimum prefetch:   0x0
    Maximum prefetch:   0x0
  Device Type:          Disk device
  Path(s):

  /dev/rdsk/c4t600144F0F8D6944400004E6090280004d0s2
  /devices/scsi_vhci/disk@g600144f0f8d6944400004e6090280004:c,raw
   Controller           /dev/cfg/c2
    Device Address              21000024ff2e4ec7,1
    Host controller port WWN    21000024ff2a967c
    Class                       secondary
    State                       STANDBY
   Controller           /dev/cfg/c2
    Device Address              2101001b323baa4c,1
    Host controller port WWN    21000024ff2a967c
    Class                       primary
    State                       ONLINE
   Controller           /dev/cfg/c3
    Device Address              21000024ff2e4ec6,1
    Host controller port WWN    21000024ff2a967d
    Class                       secondary
    State                       STANDBY
   Controller           /dev/cfg/c3
    Device Address              2100001b321baa4c,1
    Host controller port WWN    21000024ff2a967d
    Class                       primary
    State                       ONLINE

The stmsboot utility enables and disables MPxIO, for example:

   1.To enable MPxIO:
   # stmsboot -D fp -e
   
   2.To disable MPxIO:
   # stmsboot -D fp -d
   
   3.To verify the state:
   # mpathadm show LU

Caution:
if MPxIO was enabled and Symantec's VxDMP is also setup, then
you must set:

vxdmpadm settune dmp_fast_recovery=off
vxdmpadm settune dmp_health_time=0
vxdmpadm settune dmp_path_age=0
vxdmpadm settune dmp_lun_retry_timeout= <atleast as much as the MPxIO's timeout > , say 200

If the above is not taken care of, VxDMP will fail a path before MPxIO does on the Solaris host.

When enabling MPxIO with Symantec's Storage Foundation , please refer to Symantec's guidelines in the following technote
Symantec Tech 1507

MPxIO is not supported with fencing:

- IO fencing and SCSI-3 Persistent Reservation are not supported on MPxIO devices. As a result, Storage Foundation for Oracle RAC is not supported on MPxIO devices.

Note: An FC failure outside of the Sun ZFS Storage Appliance will not trigger an automatic path failover.

If the above does not help and you need additional help with the MPxIO multipathing software, please contact Oracle Support

Veritas Dynamic Multi-Pathing(DMP)

Please refer to Symantec's guidelines in the following Symantec technote:
Symantec Tech 1507

On Solaris configurations affected by Sun Bug ID 6680575, Storage Foundation High Availability on MPxIO devices with Sun Active/Passive storage hardware is not supported.

Disable MPxIO on the Solaris host

For Solaris 9, this is done by setting mpxio-disable="yes" in /kernel/drv/scsi_vhci.conf file and rebooting.

For Solaris 10,use the stmsboot -d
Please refer to the following documentation on enabling and disabling MPxIO for Solaris:
Solaris Fibre Channel and Storage Multipathing Administration Guide

Please consult Symantec's 5.1 HCL for the latest qualified UNIX OS platforms / protocols :
Symantec Tech 74012

Please consult Symantec's HW tech notes for any parameter setting needed:
Symantec Tech 47728

Install the required Storage Foundation VRTSvxvm patches.

Required Storage Foundation 5.1 RP2 Patches
Ensure you are running the latest released Storage Foundation 5.1 patches. Reference
Symantec Storage Foundation 5.1 RP2 for the Storage Foundation patches.

For clustered 7000's we require you to install an additional hot fix:
VRTSvxvm 5.1RP2 or greater for Solaris or RedHat Linux

Install the latest Storage Foundation ASL/APM package supporting the 7000 series

Required Storage Foundation 5.1 ASL/APM package
With Storage Foundation 5.1, all array ASLs / APMs are bundled into one package. The 5.1 ASL/APM tech note is located at:
Storage Foundation 5.1 ASL

For Clustered 7000s, set DMP parameters to support takeover / failback commands.

In order to support clustered Sun ZFS Storage Appliance takeover / failback commands, the following DMP tunables need to be set:

vxdmpadm settune dmp_health_time=0
vxdmpadm settune dmp_path_age=0
vxdmpadm settune dmp_lun_retry_timeout=200

If for some reason you are seeing 'Disabled Dmpnode' messages for the 7000 FC luns in the dmpevents.log
file, you may need to increase this lun retry timeout value.
The 0 settings of the dmp_health_time and dmp_path_age tunables are required.
This allows DMP to reenable the path within the short period of time that it takes the 7000 to complete takeover/failback.

Note:
Storage Foundation for Oracle RAC is not supported on MPxIO devices.

Check the multipathed device paths

# vxdisk list sun_73100_0
Device: sun_73100_0
devicetag: sun_73100_0
type: auto
hostid: xxxx
disk: name=sun_73100_0 id=1270229856.12.isv-2000a
group: name=7310-h2-dg id=1270236420.52.isv-2000a
info: format=cdsdisk,privoffset=256,pubslice=2,privslice=2
flags: online ready private autoconfig autoimport imported
pubpaths: block=/dev/vx/dmp/sun_73100_0s2 char=/dev/vx/rdmp/sun_73100_0s2
guid: {6d62c284-3e7e-11df-8e71-00144f0f13da}
udid: SUN%5FSun%20Storage%207310%5FSUN%5F7x10%5FSeries%5F600144F0FA053DEA00004BB682060002
site: -
version: 3.1
iosize: min=512 (bytes) max=2048 (blocks)
public: slice=2 offset=65792 len=209482176 disk_offset=0
private: slice=2 offset=256 len=65536 disk_offset=0
update: time=1275076440 seqno=0.264
ssb: actual_seqno=0.1
headers: 0 240
configs: count=1 len=48144
logs: count=1 len=7296
Defined regions:
config priv 000048-000239[000192]: copy=01 offset=000000 enabled
config priv 000256-048207[047952]: copy=01 offset=000192 enabled
log priv 048208-055503[007296]: copy=01 offset=000000 enabled
lockrgn priv 055504-055647[000144]: part=00 offset=000000
Multipathing information:
numpaths: 4
c2t2100001B32145466d2s2 state=enabled type=primary
c2t2100001B32148966d2s2 state=enabled type=secondary
c1t2101001B32345466d2s2 state=enabled type=primary
c1t2101001B32348966d2s2 state=enabled type=secondary

References

<BUG:6865431> - SCSI_VHCI LOSES PATH STATE AWARENESS UNDER FAILOVER/LOAD
<BUG:6915604> - PATH STATE IS NOT UPDATED AFTER FAILOVER SUCCESS
<BUG:6959914> - DB11R2E1: DBCA DELETE RAC DATABASE DOESN'T RELEASE SPACE IN ASM
<BUG:7107552> - A FLAG VLUN_KEEP_HELD CONFLICTS WITH ANOTHER TWO FLAGS
<BUG:6966496> - PATHS REPORTED AS STANDBY ARE ENUMERATED AS ONLINE (ACTIVE) PATHS
<BUG:6951173> - 7310 FC TARGET LUN TAKING OVER 10 MINUTES TO COME BACK AFTER TAKEOVER CMD WAS ISSUED
<BUG:7043998> - FAILBACKS WITH FW 2010.08.17.3.0,1-1.25 CAN TAKE 45 MINUTES OR MORE
<BUG:7132238> - PHYLOCK LIKE PROBLEMS AFTER UPGRADE TO 2011.1.1.0 ON C0 PANDORA'S
Symantec Tech Note 1507: http://www.symantec.com/business/support/index?page=content&id=TECH51507
Enable or Disable MPxIO: http://docs.oracle.com/cd/E19957-01/819-0139/ch_4_config_multi_SW.html#44079
Symatec Tech Note 74012: http://www.symantec.com/business/support/index?page=content&id=TECH74012
Symantec Tech Note 47728: http://www.symantec.com/business/support/index?page=content&id=TECH47728
Symatec Storage Foundation Patch Matrix: https://sort.symantec.com/patch/matrix

Attachments

This solution has no attachment