---------------------------------------------------------------------
- Sun Proprietary/Confidential: Internal Use Only -
---------------------------------------------------------------------  
                            FIELD INFORMATION NOTICE
                  (For Authorized Distribution by SunService)

FIN #: I0837-1

Create Date: Jun/14/02

Keywords: 

Sun Fire 15K servers cannot boot from Sun StorEdge T3/T3+ Arrays due to an issue with the Sun FCP SCSI Device Driver

SunAlert: No

Top FIN/FCO Report: No

Products Reference: T3 Storage Array on F15K

Product Category: Server / Service

Product Affected: 

Systems Affected:
-----------------
Mkt_ID      Platform    Model     Description        Serial Number
------      --------    -----     -----------        -------------
  -           F15K       ALL      Sun Fire 15K             -
 

X-Options Affected:
-------------------
Mkt_ID         Platform   Model   Description             Serial Number
------         --------   -----   -----------             -------------
F15K-T3B-RK-1	   -	    -     T3BES-RK FOR F15K             -
F15K-T3B-RK-2      -        -     T3BES-RK FOR F15K             -
F15K-T3B-RK-3      -        -     T3BES-RK FOR F15K             - 
T3BWG-RR-11-327R5  -        -     T3BWG 72" RCK 1X9X36G         -
T3BWG-RR-11-660R5  -        -     T3BWG 72" RCK 1X9X73G         -
XT3BWG-RR-11-327   -        -     T3BWG RR 1X9X36G              -
XT3BWG-RR-11-660   -        -     T3BWG RR 1X9X73G              - 
T3BES-RR-22-655R5  -        -     T3BES 72" RCK 2X9X36G         -
T3BES-RK-22-655    -        -     T3BWG 72" RCK 1X9X73G         -
T3BES-RK-88-2620   -        -     T3BWG 72" RCK 8X9X36G         -
XT3BES-RR-22-655   -        -     T3BES RR 2X9X36G              -
XT3BES-RK-22-655   -        -     T3BES 72" RCK 2X9X36G         -
XT3BES-RK-88-2620  -        -     T3BES 72" RCK 8X9X36G         -
T3BES-RR-22-1321R5 -        -     T3BES 72" RCK 2X9X73G         -
T3BES-RK-22-1321   -        -     T3BES 72" RCK 2X9X73G         -
T3BES-RK-88-5284   -        -     T3BES 72" RCK 8X9X73G         -
XT3BES-RR-22-1321  -        -     T3BES RR 2X9X73G              -
XT3BES-RK-22-1321  -        -     T3BES 72" RCK 2X9X73G         -
XT3BES-RK-88-5284  -        -     T3BES 72" RCK 8X9X73G         - 
6727A              -        -     PCI FC HOST ADAP CTO          -
6799A              -        -     FC-AL TARGET EMULATOR 2 PRT   -
X6727A             -        -     PCI DUAL FC-ADAPTER           -
X6799A             -        -     FC-AL TARGET EMULATOR 2 PRT   -

Parts Affected: 

Part Number   Description   Model
-----------   -----------   -----
     -             -          -

References: 

BugId: 4607469 - System hangs while booting after installing Leadville
                 IVj patches.

Issue Description: 

Sun Fire 15K domains cannot boot from Sun StorEdge T3/T3+ Arrays due 
to a problem with versions 4 and 5 of the Sun FCP SCSI Device Driver.  
Attempts to boot from the T3/T3+ array will hang and the domain will 
become unavailable unless a workaround is performed.

Any F15K domain using a T3/T3+ array as a boot device with Sun FCP SCSI
Device Driver version "4j" or newer under Solaris 8 is affected.  Either 
single or partner pair T3/T3+ configurations are affected.

The current version of the FCP SCSI Device Driver can be determined 
with the following command:

   # modinfo | egrep '(SunFC|mpxio|scsi_vhci)'

Sample output:

   34 78034000   fe94 150   1  fcp (SunFC FCP v5.e-2-1.17)
   35 78044000   6f28   -   1  fctl (SunFC Transport v5.e-2-1.16)
   36 7804a000   49ac   -   1  mpxio (MDI Library v5.e-1-1.7)
   37 78050000   7a10 234   1  scsi_vhci (SCSI vHCI Driver v5.e-1-1.7)
   39 78084000  10c23 149   1  fp (SunFC Port v5.e-2-1.18)
   41 780f6000  2db28 153   1  qlc (SunFC Qlogic FCA v5.e-2-1.16)

The above example shows the FCP SCSI Device Driver at version 5e.

When the failure occurs, the domain will hang during the boot process.
The hang will usually occur at this point:

   SunOS Release 5.8 Version Generic_108528-13 64-bit
   Copyright 1983-2001 Sun Microsystems, Inc.  All rights reserved. 

Root cause analysis shows that the boot process is hanging due to a
deadlock condition.  There are three threads in cyclic dependency.  As 
a result, device enumeration cannot complete and the root file system
cannot be mounted.

A permanent resolution to this problem will be incorporated in FCP SCSI
Device Driver 6.1, which is due for release sometime in July 2002.  Until
this version is released, follow the suggested workaround given in the
Corrective Action section below.

Implementation: 

---
         |   |   MANDATORY (Fully Proactive)
          ---
 
 
          ---
         |   |   CONTROLLED PROACTIVE (per Sun Geo Plan)
          ---
 
 
          ---
         | X |   REACTIVE (As Required)
          ---

Corrective Action: 

The following recommendation is provided as a guideline for authorized
Enterprise Services Field Representatives who may encounter the above
mentioned problem.

Configure Sun Fire 15K domains as shown below in order to boot from 
either a single or partner pair T3/T3+ configuration:
 
---------------------------------
Single Brick (WG) T3 Boot Devices
----------------------------------
   Booting from a single brick (WG) T3 is supported only
   if the following items are true:
 
   1. The StorEdge T3 WG is connnected to port 2 of a Dual Fibre Channel
       Network Adapter [Crystal+], X6727A  (Sun p/n 375-3030).
 
   2. Port 1 on the Crystal+ card must *not* have mpxio explicitly 
       disabled in the host system's qlc.conf file.

       Use the following procedure to insure that mpxio (StorEdge Traffic 
       Manager) is not disabled for Port 1:

       Use "ls -l /dev/fc" to display the Fiber Channel devices.

       Example Output:

       lrwxrwxrwx 1 root root 49 Apr 17 18:14 fp0 -> ../ 
    ../devices/pci@6,2000/SUNW,qlc@2/fp@0,0:devctl 
       lrwxrwxrwx 1 root root 49 Apr 17 18:14 fp1 -> ../ 
    ../devices/pci@7,2000/SUNW,qlc@2/fp@0,0:devctl 
       lrwxrwxrwx 1 root root 49 Apr 17 18:14 fp0 -> ../ 
    ../devices/pci@6,2000/SUNW,qlc@2/fp@0,0:devctl 
       lrwxrwxrwx 1 root root 49 Apr 17 18:14 fp1 -> ../ 
    ../devices/pci@7,2000/SUNW,qlc@2/fp@0,0:devctl 
       lrwxrwxrwx 1 root root 49 Apr 17 18:14 fp2 -> ../ 
    ../devices/pci@a,2000/SUNW,qlc@2/fp@0,0:devctl 
       lrwxrwxrwx 1 root root 49 Apr 17 18:14 fp3 -> ../ 
    ../devices/pci@b,2000/SUNW,qlc@2/fp@0,0:devctl 
       lrwxrwxrwx 1 root root 50 Apr 17 18:14 fp4 -> ../ 
    ../devices/pci@12,2000/SUNW,qlc@2/fp@0,0:devctl 
       lrwxrwxrwx 1 root root 56 Apr 17 18:14 fp5 -> ../ 
    ../devices/pci@13,2000/pci@2/SUNW,qlc@4/fp@0,0:devctl 
       lrwxrwxrwx 1 root root 56 Apr 17 18:14 fp6 -> ../ 
    ../devices/pci@13,2000/pci@2/SUNW,qlc@5/fp@0,0:devctl 

     The last two entries have two /pci elements. This is a dual PCI HBA.
     The rest of the entries do not have additional PCI bridges and are
     Single PCI HBAs.

     To explicitly enable mpxio on port 1, enter the following line in 
     qlc.conf:

    name="qlc" parent="/pci@13,2000/pci@2" unit-address="4" mpxio-disable="no"

   3. Port 2 on the Crystal+ card *must* have mpxio explicitly
      disabled in the host system's qlc.conf file.

     To explicitly disable mpxio on port 2, using the above example,
     enter the following line in qlc.conf:

   name="qlc" parent="/pci@13,2000/pci@2" unit-address="5" mpxio-disable="yes"

NOTE: In a non-fabric environment, whenever the "mpxio-disable" field in 
      scsi_vhci.conf, qlc.conf or T3's mp-support field is changed, the 
      host must go through a reconfiguration reboot (boot -r).

---------------------------------
Partner Pair (ES) T3 Boot Devices
---------------------------------
   Booting from a StorEdge T3 partner pair (ES) configuration is
   supported only if the following items are true:
 
    1. Both connections from the host to the T3 partner pair are through
       Dual Fibre Channel Network Adapter [Crystal+] , X6727A  
       (Sun p/n 375-3030).
 
    2. Only port 2 on each Crystal+ card is connected to the partner 
       pair being used as the boot device.
 
    3. Port 1 on each of the two Crystal+ cards must *not* have
       mpxio explicitly disabled in the host system's qlc.conf file.
 
    4. Port 2 on the Crystal+ card *must* have mpxio explicitly
       disabled in the host system's qlc.conf file.

       See the example above under Single Brick T3 Boot Devices for 
       more information on how to enable and disable mpxio on a 
       per-port basis.

    5. On the T3 array, multi-pathing support can only be set to one 
       of the following options (using the T3 'sys' command):
 
          mp_support=rw
 
       OR
 
          mp_support=none

Comments: 

None
 
============================================================================

Implementation Footnote: 

i)   In case of MANDATORY FINs, Enterprise Services will attempt to
     contact all affected customers to recommend implementation of
     the FIN.
 
ii)  For CONTROLLED PROACTIVE FINs, Enterprise Services mission critical
     support teams will recommend implementation of the FIN  (to their
     respective accounts), at the convenience of the customer.
 
iii) For REACTIVE FINs, Enterprise Services will implement the FIN as the
     need arises.
----------------------------------------------------------------------------
All released FINs and FCOs can be accessed using your favorite network
browser as follows:
 
SunWeb Access:
--------------
* Access the top level URL of http://sdpsweb.ebay/FIN_FCO/
 
* From there, select the appropriate link to query or browse the FIN and
  FCO Homepage collections.
 
SunSolve Online Access:
-----------------------
* Access the SunSolve Online URL at http://sunsolve.Corp/
 
* From there, select the appropriate link to browse the FIN or FCO index.
 
Internet Access:
----------------
* Access the top level URL of https://infoserver.Sun.COM
--------------------------------------------------------------------------
General:
--------
* Send questions or comments to [email protected]
--------------------------------------------------------------------------