Replacing Veritas Volume Manager (VXVM) managed FCAL disks in V280R, V480, V490, V880, V890, E3500 Servers

Asset ID:	1-71-1003122.1
Update Date:	2011-12-09
Keywords:

Solution Type Technical Instruction Sure

Solution 1003122.1 : Replacing Veritas Volume Manager (VXVM) managed FCAL disks in V280R, V480, V490, V880, V890, E3500 Servers

Applies to:

Sun Fire V490 Server
Sun Fire V480 Server
Sun Fire V890 Server
Sun Fire V880 Server
Sun Fire 280R Server
All Platforms

Goal

This document provides a working example of how to replace a Servers, failed, internal, FCAL disk, which is also under Veritas Volume Manager (VXVM) control. The most typical of scenarios involves a failed boot drive which is mirrored with VXVM. Caveats and alternatives are suggested herein, but have been omitted for clarity.

Solution

Lets start with a working, mirrored pair of boot disks in a V490. In
this example, the root and swap partitions are mirrored. Notice the
optimal outputs from utilities such as vxprint -ht, vxdisk list and
format. Data parsed for clarity...

#vxprint -ht
   dm rootdisk     c4t0d0s2     auto     66190    71057844 -
   dm rootmirror   c4t1d0s2     auto     66190    71057844 -

   v  rootvol      -            ENABLED  ACTIVE   25166079 ROUND     -        root
   pl rootvol-01   rootvol      ENABLED  ACTIVE   25166079 CONCAT    -        RW
   sd rootdisk-02  rootvol-01   rootdisk 29360906 25166079 0         c4t0d0   ENA
   pl rootvol-02   rootvol      ENABLED  ACTIVE   25166079 CONCAT    -        RW
   sd rootmirror-01 rootvol-02  rootmirror 0      25166079 0         c4t1d0   ENA

   v  swapvol      -            ENABLED  ACTIVE   4194828  ROUND     -        swap
   pl swapvol-01   swapvol      ENABLED  ACTIVE   4194828  CONCAT    -        RW
   sd rootdisk-01  swapvol-01   rootdisk 25166078 4194828  0         c4t0d0   ENA
   pl swapvol-02   swapvol      ENABLED  ACTIVE   4194828  CONCAT    -        RW
   sd rootmirror-02 swapvol-02  rootmirror 25166079 4194828 0        c4t1d0   ENA

# vxdisk list
   DEVICE       TYPE            DISK         GROUP        STATUS
   c4t0d0s2     auto:sliced     rootdisk     rootdg       online
   c4t1d0s2     auto:sliced     rootmirror   rootdg       online

# format
      28. c4t0d0 <SEAGATE-ST336605FSUN36G-0438 cyl 24620 alt 2 hd 27 sec 107>
          /pci@9,600000/SUNW,qlc@2/fp@0,0/ssd@w21000004cf7fe655,0
      29. c4t1d0 <SUN36G cyl 24620 alt 2 hd 27 sec 107>
          /pci@9,600000/SUNW,qlc@2/fp@0,0/ssd@w21000004cf8f57c1,0

Disk c4t0d0 fails. Notice how the outputs have changed. Mirrors are
DISABLED, Disks are failed and the inquiry data from format is
Unknown. Other logs such as /var/adm/messages should be reviewed
as well for evidence of failure.

# vxprint -ht
   dm rootdisk     -            -        -        -        NODEVICE
   dm rootmirror   c4t1d0s2     auto     66190    71057844 -

   v  rootvol      -            ENABLED  ACTIVE   25166079 ROUND     -        root
   pl rootvol-01   rootvol      DISABLED NODEVICE 25166079 CONCAT    -        RW
   sd rootdisk-02  rootvol-01   rootdisk 29360906 25166079 0         -        RLOC
   pl rootvol-02   rootvol      ENABLED  ACTIVE   25166079 CONCAT    -        RW
   sd rootmirror-01 rootvol-02  rootmirror 0      25166079 0         c4t1d0   ENA

   v  swapvol      -            ENABLED  ACTIVE   4194828  ROUND     -        swap
   pl swapvol-01   swapvol      DISABLED NODEVICE 4194828  CONCAT    -        RW
   sd rootdisk-01  swapvol-01   rootdisk 25166078 4194828  0         -        NDEV
   pl swapvol-02   swapvol      ENABLED  ACTIVE   4194828  CONCAT    -        RW
   sd rootmirror-02 swapvol-02  rootmirror 25166079 4194828 0        c4t1d0   ENA

# vxdisk list
   DEVICE       TYPE            DISK         GROUP        STATUS
   c4t1d0s2     auto:sliced     rootmirror   rootdg       online
   -            -         rootdisk     rootdg       failed was:c4t0d0s2

# format
      28. c4t0d0 <drive type unknown>
          /pci@9,600000/SUNW,qlc@2/fp@0,0/ssd@w21000004cf7fe655,0
      29. c4t1d0 <SUN36G cyl 24620 alt 2 hd 27 sec 107>
          /pci@9,600000/SUNW,qlc@2/fp@0,0/ssd@w21000004cf8f57c1,0

We can now proceed with the hardware replacement of the disk.


# luxadm -e offline /dev/rdsk/c4t0d0s2
# luxadm remove_device -F /dev/rdsk/c4t0d0s2

   WARNING!!! Please ensure that no filesystems are mounted on these device(s).
   All data on these devices should have been backed up.
   The list of devices which will be removed is:
     1: Device name: /dev/rdsk/c4t0d0s2
        Node WWN:    20000004cf7fe655
        Device Type:Disk device
        Device Paths:
        /dev/rdsk/c4t0d0s2
   Please verify the above list of devices and
   then enter 'c' or <CR> to Continue or 'q' to Quit. [Default: c]:
   stopping:  /dev/rdsk/c4t0d0s2....Done
   offlining: /dev/rdsk/c4t0d0s2....Done
   Hit <Return> after removing the device(s).
   ............
   Device: /dev/rdsk/c4t0d0s2 Removed.

  Note: For the v880 ONLY, you may remove the device by enclosure name and slot number.
  For example: # luxadm remove_device FCloop,s1

Physically replace the disk.

The next command will create the devices for the new drive.
Alternatively, you can use the # devfsadm -Cv to create the devices.
Depending on your version of solaris, the picld daemon may make
them for you as well. If all these fail, reconfiguration reboot
will solve the problem. Note: For the v880 ONLY, you may add the
device by enclosure name and slot number.
For example: # luxadm insert_device FCloop,s1

# luxadm insert_device
     Please hit <RETURN> when you have finished adding Fibre Channel Enclosure(s)/Device(s):
     Waiting for Loop Initialization to complete...
     New Logical Nodes under /dev/dsk and /dev/rdsk :
        c4t0d0s0
        c4t0d0s1
        c4t0d0s2
        c4t0d0s3
        c4t0d0s4
        c4t0d0s5
        c4t0d0s6
        c4t0d0s7

Lets get the disk recognized and labeled in solaris.

# format c4t0d0 
   label <Enter> y <Enter>

Finish up all repairs in Veritas Volume manager.This next command
will initialize the drive. There are many different initialization
parameters (especially in later versions of veritas volume manager)
You must make sure the disk is initialized in the exact same fashion
as before. An excellent verification of this is to compare the VTOC
of the drive with other disks in the diskgroup. If your vtoc differs
after initialization, see the man page on vxdisksetup to re-initialize
correctly.



# vxdctl enable
# /usr/lib/vxvm/bin/vxdisksetup -i c4t0d0 format=sliced
# vxdg -g rootdg -k adddisk rootdisk=c4t0d0s2
# vxrecover -g rootdg rootdisk
# /usr/lib/vxvm/bin/vxbootsetup  -g rootdg  rootdisk

As an alternative to the vxdisksetup, vxdg, vxrecover and vxbootsetup
commands, you may use the # vxdiskadm utility, option 5, to restore the
disk into the diskgroup. When complete, outputs should all report normal.

#vxprint -ht
   dm rootdisk     c4t0d0s2     auto     66190    71057844 -
   dm rootmirror   c4t1d0s2     auto     66190    71057844 -

   v  rootvol      -            ENABLED  ACTIVE   25166079 ROUND     -        root
   pl rootvol-01   rootvol      ENABLED  ACTIVE   25166079 CONCAT    -        RW
   sd rootdisk-02  rootvol-01   rootdisk 29360906 25166079 0         c4t0d0   ENA
   pl rootvol-02   rootvol      ENABLED  ACTIVE   25166079 CONCAT    -        RW
   sd rootmirror-01 rootvol-02  rootmirror 0      25166079 0         c4t1d0   ENA

   v  swapvol      -            ENABLED  ACTIVE   4194828  ROUND     -        swap
   pl swapvol-01   swapvol      ENABLED  ACTIVE   4194828  CONCAT    -        RW
   sd rootdisk-01  swapvol-01   rootdisk 25166078 4194828  0         c4t0d0   ENA
   pl swapvol-02   swapvol      ENABLED  ACTIVE   4194828  CONCAT    -        RW
   sd rootmirror-02 swapvol-02  rootmirror 25166079 4194828 0        c4t1d0   ENA

# vxdisk list
   DEVICE       TYPE            DISK         GROUP        STATUS
   c4t0d0s2     auto:sliced     rootdisk     rootdg       online
   c4t1d0s2     auto:sliced     rootmirror   rootdg       online

Attachments

This solution has no attachment