Goal

Solution

Contents

• Background

• Creating a bay_labels Configuration File

  • About bay_labels Configuration File

  • Format of bay_labels Configuration File and Examples

  • How to Create bay_labels Configuration File

• Creating a Custom Solaris Distribution that Contains the bay_labels files

• Create AI or LiveCD Distro Constructor Image that Contains your Newly Created IPS Package

The bay_labels file captures information about how the physical serial links (also known as PHY's) are wired/routed from either a controller on the main board or a plug-in HBA card to the disk drive bays. This file is used in the FMA topology discovery process to add location content to the libdevinfo(3LIB), which in turn gets consumed by various installers and commands to provide location information. For example, the text installer and GUI installer would be able to display location (label) information for each target candidate so that the user can select the desired device for installation based on location information. The automated installer can specify label name when "receptacle" type is selected as in the following example:

      </disk>

 The name of the bay_labels configuration file file needs to be <PLATFORM-NAME>,bay_labels. How to find the <PLATFORM-NAME> is explained below, and note there is a comma (,) between <PLATFORM-NAME> and bay_labels. You can re-use the same bay_labels configuration file if all your servers are the same <PLATFORM-NAME>.

The following assumptions are made when a bay_labels configuration file is re-used for multiple servers:

• The servers have the same <PLATFORM-NAME>

• The servers have the same number of supported disk slots.

• The servers have the same SAS 2.0 HBA/controller.

• The cabling between the SAS 2.0 HBA/controller and the disk bay is exactly the same for the servers.

Format of bay_labels Configuration File and Examples

Two key pieces of information in the bay_labels configuration file need attention that requires a special procedure to create:

• The Label (silkscreen labels on printed on drive bay)

• The PHY (phy-num) number

Example 1

This example is for a simple 8-slot disk bay with Intel SAS 2.0 SCU in the system:

$ cat /usr/platform/i86pc/lib/fm/topo/maps/SUN-FIRE-X4275-M3,bay_labels
# Product : HBA : HBA Instance : Chassis Name : Chassis S/N : PHY : Label
# -----------------------------------------------------------------------
SUN-FIRE-X4275-M3:scu:0:SYS:-:0:HDD0
SUN-FIRE-X4275-M3:scu:0:SYS:-:1:HDD1
SUN-FIRE-X4275-M3:scu:0:SYS:-:2:HDD2
SUN-FIRE-X4275-M3:scu:0:SYS:-:3:HDD3
SUN-FIRE-X4275-M3:scu:0:SYS:-:4:HDD4
SUN-FIRE-X4275-M3:scu:0:SYS:-:5:HDD5
SUN-FIRE-X4275-M3:scu:0:SYS:-:6:HDD6
SUN-FIRE-X4275-M3:scu:0:SYS:-:7:HDD7

Example 2

This example is for a 7-slot disk bay with LSI SAS 2.0 HBA, mpt_sas, in the system.

$ cat /usr/platform/i86pc/lib/fm/topo/maps/PowerEdge-T710,bay_labels
# Product : HBA : HBA Instance : Chassis Name : Chassis S/N : PHY : Label
# -----------------------------------------------------------------------
PowerEdge-T710:mpt_sas:0:SYS:7J81VK1:1:HDD6
PowerEdge-T710:mpt_sas:0:SYS:7J81VK1:2:HDD5
PowerEdge-T710:mpt_sas:0:SYS:7J81VK1:3:HDD4
PowerEdge-T710:mpt_sas:0:SYS:7J81VK1:4:HDD3
PowerEdge-T710:mpt_sas:0:SYS:7J81VK1:5:HDD2
PowerEdge-T710:mpt_sas:0:SYS:7J81VK1:6:HDD1
PowerEdge-T710:mpt_sas:0:SYS:7J81VK1:7:HDD0

How to Create bay_labels Configuration File

If none of the bay_labels files provided in the intended directory will work for your system configuration, use the following steps to create a bay_labels configuration file.

Note: To create a bay_labels configuration file, you need to have physical access to the system. Then, you can deploy that file to other systems that have the same configuration.

The First Phase:

1. Power off the server and unplug all disk drives.

2. Insert all disk drives to the disk drive bay one by one while recording the WWN of each drive and the corresponding drive-bay label (silkscreen) the drive is inserted into. You should have a collection of pairs with (WWN, bay_labels).

5000C50003B01B7B HDD0
5000C5003A7939A1 HDD1
5000CCA0120FA511 HDD2
.....

3. Boot Oracle Solaris 11 with the text or GUI Installer. You don't need to do the installation. Start a terminal window with a command line window or exit out of text installer to a shell command prompt.

4. Use the format(1M) command to list the disk drives for all slots. Here's is an example of the format output showing the c[x]t[y]d[z] decoding for the third entry c[0]t[5000C50003B01B7B]d[0].

Note that the SAS disk drive normally has two serial ports, each has a near identical SAS address. They differ only in the very last hexadecimal digit depending on implementation details. The explicit device path will be listed for the drive without MPxIO support while the /scsi_vhci/ path will represent the logical path when MPxIO is enabled.

You might have the location information like this /dev/chassis/SYS/HDD0/disk for the disk drive. But, for now, do not trust that information until you have completed the verification process.

Searching for disks...done

AVAILABLE DISK SELECTIONS:
       0. c7t5000CCA0120FA511d0 
          /pci@0,0/pci8086,3c02@1/pci8086,1d74@0/pci8086,1d3f@8/pci8086,1d68@0/
             iport@1/disk@w5000cca0120fa511,0
       1. c8t5000C5003A7939A1d0 
          /pci@0,0/pci8086,3c02@1/pci8086,1d74@0/pci8086,1d3f@8/pci8086,1d68@0/
             iport@4/disk@w5000c5003a7939a1,0
       2. c0t5000C50003B01B7Bd0 
          /scsi_vhci/disk@g5000c50003b01b7b
Specify disk (enter its number): 

5. For each disk device listed in the format command output, use the following technique to get the phy-num value:

#DISK=/dev/dsk/c[x]t[y]d[z]
#PHYNUM=`prtconf -v ${DISK}s2 | awk '/phy-num/ {print;getline;print}' | grep value`
#echo $PHYNUM | awk -F"=" '{print $2}'| uniq

Basically, you invoke the prtconf command to get the device's phy-num value from the output. The phy-num and its value appear on two consecutive lines.

6. Record phy-num value with each device identified through WWN, and its corresponding label in the bay_labels configuration file. Now, you have collected a table with each row showing WWN device name, phy-num value, and the silkscreen label string for this particular system.

c0t5000C50003B01B7Bd0 00000000 HDD0
c8t5000C5003A7939A1d0 00000001 HDD1
c7t5000CCA0120FA511d0 00000002 HDD2
.....

The Second Phase:

Properly formatted bay_labels files, which may be copied and used for your initial template, can be found in the following Solaris 11 directory /usr/platform/<PLATFORM>/lib/fm/topo/maps/.

You will need to copy, rename, and edit one of these files in order to create a new bay_label file for your specific system. Make sure that your drive bay "label" information and “Product Name” string matches that of the targeted server system.

You can get "Product Name" string by using the the "fmtopo" command. The following command line example may be used to get the target system’s product ID.

#root@server1:~# ./fmtopo | head -5 | grep product-id  | cut -d : -f 3
product-id=SUN-FIRE-X4275-M3 

The “fmtopo” command can be found in /usr/lib/fm/fmd directory.

Below is a brief explanation of the various required fields in the bay_labels file.

# Product : HBA : HBA Instance : Chassis Name : Chassis S/N : PHY : Label

• The "Product" ID string can be obtained using the “fmtopo” command (see example above).
• The "HBA" is the driver name of the HBA, which can be scu, mpt_sas, or pmcs.
• "HBA Instance" number is normally 0 unless you have multiple HBA's in the system.
• The "Chassis Name" is normally "SYS", which implies internal direct attached disk drives. Disk drives attached through external SAS expanders do not need this bay_labels configuration file.
• The "Chassis S/N" is normally a "-" string.
• The "PHY" and "Label" pairs should be imported from the file created by the manual procedure in the previous steps.

The Final Testing and Verification Phase:

You can use "svcadm restart fmd" to trigger the topology discovery process without rebooting the server. You can then use "croinfo" and "format" commands to verify whether the labels match the actual device placement. You might need to wait several seconds since the FMA daemon topology discovery process cannot be completed instantaneously. Other commands including "smbios" should be useful to confirm and validate the content of the configuration file.

Using a new bay_labels file with the Solaris 11 Installer:

In order for the bay_labels file to be useful during install it needs to be contained in the install media. To accomplish this, create a Solaris IPS package that contains your bay_labels files and publish it to an accessible package repository. The IPS package should install the bay_labels file into the /usr/platform/<PLATFORM>/lib/fm/topo/maps/ directory. The Solaris Distro Constructor utility [distro_const(1M)], can then be used to add this IPS package to a custom LiveCD or Automated Installer (AI) install image. (See the next section) If doing an AI install, the AI manifest needs to be modified to also install the IPS package.

Creating a Custom Solaris Distribution that Contains the bay_labels files

The following section describes how to create an IPS package that contains your croinfo map files and publish it to your package repository.

4. Modify all the file entries in the manifest so the path points to the proper location. For example,  path=/usr/platform/i86pc/lib/fm/topo/maps/.

Create AI or LiveCD Distro Constructor Image that Contains Your Newly Created IPS package

Install a system with the Oracle Solaris 11 build from which you want to create your Distro Constructor image.

1. Add the distro constructor package.

2. Copy desired manifest file for customizing. The example manifests are in /usr/share/distro_const.

For LiveCD:

For AI:

For AI installs make sure you modify your AI manifest file to also install your package (include additional publisher/repo if needed).

This solution has no attachment