Sun Storage 2530 Array
 
Sun Storage 2540 Array
 

GCS>Sun Microsystems>Storage - Disk>Modular Disk - 2xxx Arrays
 

This document provides simple definitions for terms specific to the SAS protocol. A thorough understanding these terms is required before attempting to troubleshoot any SAS related issues.

A Phy contains one transceiver (below).

A physical link attaches two phys together

Please see named 100000000000018900000154C6533EEA.png below.

SAS bandwidth

For each SAS Link there is one transmit and one receive running at 3Gb/s. Full duplexing allows for data to be received and transmitted at the same time, which means that 6Gb/s full-duplex is possible.

Both directions must use the same physical link rate.

Peak bandwidth can be (and usually is) greatly increased using port bonding or port aggregation which in SAS terminology is referred to as a wide link.

Please see named 1000000000000185000000F7B8CFA49A.png below.

SAS: What is a port

Ports contain Phys (below). Each SAS port has a worldwide unique 64-bit address. In SAS, the 64-bit SAS address serves as the SCSI port identifier. Of course this port identifier must be unique within a SCSI domain. This 64-bit port address is analogous to the fibre channel Port Worldwide Number.

Please see named 100000000000011D0000007EDB601949.png below.

SAS: Wide ports

A wide port is created if there is more than one physical link in the port (below). Wide ports provide port aggregation (sometimes referred to as 'bonding' or 'trunking') for speeds up to 12Gb/s (24Gb/s full-duplex). Wide ports are configured automatically, the hardware detects the number of physical links so no user input is needed.

With multiple links a command may be sent down one link and corresponding data may be returned on an alternate link (SAS uses available resources).

For example the following link is x2 wide that is there are 2 phys on either side of the link. The port peak bandwidth of a 2 phy HBA is 6Gb/s half duplex or 12Gb/s full duplex.

Please see named 1000000000000127000000E08C1D6B96.png below.

SAS Addresses

SAS addresses may be broken down into the following three main components:

Please see named 1000000000000350000000EE5464479E.png below.

Least significant 12 bits used to reflect various entities on the controller:

- Controller SAS Base Address (BA): the address created when all 12 bits are set to 0b

- Addresses starting at BA assigned to on-board IOC ASIC, consuming one address per PHY (IOC_PHYS)

- Addresses starting at BA + IOC_PHYS assigned to Host Interface Card, consuming one address per PHY (HIC_PHYS)

- Addresses starting at BA + IOC_PHYS + HIC PHYS assigned to controller-resident SAS Expander, consuming one address per Expander PHY, plus one address per Expander Virtual PHY, plus one address per potential SATA drive

SAS: What is a device

SAS devices contain ports

Each phy within a device has a unique identifier, for example phy 2.

According to the SAS standard the topology shown below is perfectly legal the physical limitations on device connectivity are based on available cable types (which are usually 4 phy wide). The other important restriction placed on connectivity is that devices should not create a looped topology.

Please see named 10000000000001610000018039F1EB39.png below.

SAS: Sample end devices

There are several possible permutations and combinations for SAS connectivity between a target and an initiator. The end user has no control over this behaviour, nevertheless, it is worthwhile exploring the possibilities:

HBA 8 phys

option (i) One SAS address for all 8 phy

option (ii) One SAS address for 4 phys, another address for the other 4 phys

option (iii) 8 SAS addresses

Disk Drive 2 phys

option (i) Separate SAS address for each phy

SAS: Understanding a real example

In the following output, one port spans four phys. And note that the port WWN then changes by a factor of 4 on the last byte. Of course this last byte is in hexadecimal, so exercise caution in interpreting raw WWNs.

   # grep WWN /var/adm/messages

   Aug 3 12:20:01 kangaroo mpt1: Port 0/PHYs 0-3 (wide-port) WWN: 0x500605b000244410

   Aug 3 12:20:01 kangaroo mpt1: Port 1/PHYs 4-7 (wide-port) WWN: 0x500605b000255414

   Aug 3 12:20:02 kangaroo mpt3: Port 0/PHYs 0-3 (wide-port) WWN: 0x500605b0000af020

   Aug 3 12:20:02 kangaroo mpt3: Port 1/PHYs 4-7 (wide-port) WWN: 0x500605b0000af024

The above syslog output represents the output for x2 dual-ported SAS HBAs(on the server) connecting to a total of 4 (physical) ports on the target (which happens to be an STK2530). The diagram following represents only the first dual-ported HBA connection with the SAS target.

Interpreting the output:

       Please see named 1000000000000299000001B7F9B2937A.png below.

SAS Connectors

Hickory and Sun SAS HBAs use Mini-SAS connectors (below) sometimes also referred to as iPASS connectors. Other SAS connectors, not used for Hickory, resemble Infiniband connectors.

Please see named 100000000000012C000000F0A57927E5.png below.

SAS Connector Types

Name Description

________________________________________________________

SFF 8482 form factor compatible with SATA

SFF 8484 hi-density int. connector <= 4 devices

SFF 8470 a.k.a. Infiniband connector

SFF 8087 a.k.a. iPASS internal 4x connector

SFF 8088* a.k.a. iPASS external 4x connector

Sun SAS HBAs

Name Type SSID

________________________________________________________

Dione PCI-X 3Gb/s SAS HBA - 8 Port 0x3160

Pandora PCI-Express 3Gb/s SAS HBA - 8 Port 0x3170

Prometheus PCI-Express SAS RAID HBA - 8 Port 0x0287

Sun SAS storage

Host Model# VID / PID LSI sub

Connect Codename model

________________________________________________________

FC ST2540 SUN/LCSM100_F Mary Jane 5

SAS ST2530 SUN/LCSM100_S Keystone 58

iSCSI ST2510 SUN/LCSM100_I Breckenridge 60

ESM* ST2500 SUN/LCSM100_E Shea n/a

*No direct host-connect for ESM module

SAS-related Links

SAS HBA Support Matrix

http://twiki.sfbay/bin/view/Main/HBASupportMatrix#TableSASSCSI

HBA Engineering Twiki

http://twiki.sfbay.sun.com/bin/view/Main/StorageConnEng

NSG Twiki

http://nsgtwiki.sfbay.sun.com/twiki/bin/view

T10 SAS Drafts

http://www.t10.org/drafts.htm#SCSI3_SAS

MPxIO/mpt Information

http://disk.prc/work/projects/mpxio_mpt

SCSI International Trade Association

http://scsita/org

Sun/Seagate SAS presentation

http://www.seagate.com/specials/sac/sun/

References:

T10/1760-D, Revision 4

16 May 2006

Information technology - Serial Attached SCSI - 2 (SAS-2)

SAS Technology TOI

David Hollister

Staff Engineer

Connectivity Solutions

Hickory TOI, Sun Microsystems 2007

Submitted by: Terence Donoghue

This solution has no attachment