Sun SPARC Enterprise M9000-64 Server
 
Sun SPARC Enterprise M9000-32 Server
 
Sun SPARC Enterprise M8000 Server
 
Sun SPARC Enterprise M3000 Server
 
Sun SPARC Enterprise M4000 Server
 
Sun SPARC Enterprise M5000 Server
 

PLA-Support>Sun Systems>SPARC>Enterprise>SN-SPARC: Mx000
 
.Old GCS Categories>Sun Microsystems>Servers>OPL Servers
 

Applies to:

Goal

This Document will describe OPL memory subsystem, configuration rules and restrictions.

Fix

Introduction

In the Mx000 (OPL) Servers resources are located on 1 to 16 Physical System Boards (PSB). A Physical System Board (PSB) consists of up to four (4) CPUM-CHIP's, up to 32 DIMM's and one IOU resource (optional).
I/O resources on an IOU are tied to a specific CPU/Memory resource, and that CPU, Memory & I/O resource are assigned to a XSB (eXtended System Board). The XSB can then be assigned to a domain. To be usable for a domain, a PSB must first be configured in either UNI-XSB or QUAD-XSB mode.

A M3000 PSB contains a fixed number of pieces of hardware:

• A single CPUM-CHIP
• Up to 4 or 8 DIMM's
• IO

A M4000 - PSB contains of up to seven pieces of hardware:

• Up to two CPUM (each containing two CPUM-CHIP's for a total of four physical CPU's).
• Up to four MEMB's (each containing 4 or 8 DIMM's ; up to 32 total physical DIMM's)
• One IOU (IOU's are needed to access internal devices DVD, Tape, Disk, Network)

A M5000 - PSB contains of up to seven pieces of hardware:

• Up to two CPUM (each containing two CPUM-CHIP's for a total of four physical CPU's).
• Up to four MEMB's (each containing 4 or 8 DIMM's ; up to 32 total physical DIMM's)
• One IOU (Optional, at least one required per domain; IOU's are needed to access internal devices DVD, Tape, Disk, Network)

The M4000 has 1 PSB and the M5000 has 2 PSB.

A M8000/M9000 PSB contains up to two pieces of hardware:

• One CMU (each CMU has 2 or 4 CPUM-CHIP's and 16 or 32 DIMM's)
• One IOU (Optional, at least one required per domain)

In UNI-XSB mode, we get 1 XSB consisting of the whole PSB.
In QUAD-XSB mode, we get up to 4 XSB's, consisting each of one quarter of a PSB.
The M3000 (Ikkaku) can be viewed as a single QUAD-XSB (1 CPUM-CHIP, 8 DIMMs).

The Mx000 (OPL) Servers use DDR2 DIMMs. 1, 2, 4, or 8 GB types can be used.
OPL uses Memory Groups. XSBs can access 2 groups, Group A and Group B.
Group A must always be populated, Group B can optionally be populated to increase memory capacity.

OPL automatically configures memory interleaving to obtain the best performance.
Interleaving takes place within a XSB (UNI or QUAD).
All DIMMs within a XSB are included in a single interleaving scheme, there is only one interleave factor defined inside a XSB. This has the consequence that the number of DIMMS in a Group can only be 4, 8 or 16 and that the number of DIMMs in Group B must be either 0 or the same as the number of DIMMs in Group A
Populating Group B does not affect interleaving or mirroring, it only increases the memory capacity.

Note: Interleave factor of 3 is not possible, therefore XSBs with 12 DIMMS (or 24 DIMMs ) are not permitted.
Note: By definition, a QUAD-XSB can only hold 4 DIMMs in Group A and optionally 4 DIMMS in Group B.

Memory can be configured in mirror mode for higher reliability. (except M3000)
In mirrored mode, both the the memory capacity and the the interleave factor are halved.

The minimal hardware required for a functioning XSB (UNI or QUAD) is to have at least 1 CPUM-CHIP and at least 4 DIMMs in Group A.

DIMM naming scheme and physical location

The DIMM names as output by the showhardconf command and as named on the FRUs is as follows:

Applicable memory population rules

The rules are summarized in the 5 tables below and the additional Group rules.
The gray areas depict the hardware that can be installed.
It is important to understand that the rules apply per XSB.

Note: PSB1 only exists in M5000.

*Note: Install these first.
Note: PSB1 only exists in M5000.
Note: Each PSB can individually be set to UNI-XSB or QUAD -XSB mode.

Note: Each PSB can individually be set to UNI-XSB or QUAD -XSB mode.
Note: For architectural reasons, M8000 / M9000 do not allow memory mirroring in QUAD-XSB mode.

Additional Group Rules

within a XSB (UNI or QUAD), the following rules apply in addition to the above 4 tables:

1. All DIMMs within a Group (A or B) must be of the same size / rank

2. The size of the DIMMs in Group B must be smaller than or equal the size of the DIMMs in Group A.

3. The number of DIMMs in Group B must be either 0 or the same as the number of DIMMs in Group A

Additional considerations

Implication of the above rules for M4000 / M5000

1. Half populated MEMB, with only Group A populated are supported.

2. In UNI-XSB mode, MEMBs must be installed in powers of 2 (1,2 or 4) per XSB.
   3 MEMBs are not possible, due to the single interleaving scheme used on an XSB.

   Note: In a M5000, there are two PSBs, each of which corresponds to a separate and independent XSB.

3. It is permitted to have 3 MEMBs on a PSB, as long as the PSB is configured in QUAD-XSB mode.

4. The minimum requirement for UNI-XSB mode is CPUM#0 / MEMB#0 (and CPUM#2 / MEMB#4)

5. A CPUM-CHIP on a CPUM without its associated MEMB and vice-versa cannot be used in QUAD-XSB mode.
   The setupfru command will report Operation has completed. However, a configuration error was detected. If you configure a PSB in QUAD-XSB mode and one of the QUAD-XSB is made of a CPUM-CHIP and no corresponding MEMB or vice-versa.
   In this case, "showboards" will report the QUAD-XSB as "Unmount".

6. XSBs can have different memory configurations.

7. Only identically configured QUAD-XSBs can be reconfigured into one UNI-XSB

8. Filling group B does not increase the interleaving factor, thus does not increase memory performance.
   Only the memory capacity is increased.

Memory upgrade

The rule requiring that Group A contains the DIMMs with the higher capacity may dictate that in case of a memory upgrade, DIMMs from Group A must be moved to Group B and the new higher capacity DIMMs be installed in Group A.

Performance considerations

UNI-XSB mode allows higher memory performance because it can use higher interleave factors. If all resources of a PSB are needed in one domain, then it is advisable to configure the PSB in UNI-XSB mode.

RAS considerations

Memory mirroring is an effective way to protect against memory failures. Configuring an XSB for memory mirroring halves the available memory capacity. It also halves the interleave factor. Memory mirroring should therefore best be used in UNI-XSB mode.

Best Practice

Try to avoid mixed configurations. It is advisable to have all identical MEMBs in a PSB of a M4000 / M5000. This allows to change between UNI-XSB and QUAD-XSB mode without loosing maximum performance or having systems containing unusable hardware.

If the capability to switch between UNI-XSB and QUAD-XSB mode is desired, then both the rules for UNI-XSB and those for QUAD_XSB must be followed. The supported M8000 / M9000 configurations enforce this.

How to identify DIMMs inside a system from XSCF

The showhardconf command run on the XSCF display the type of DIMMs

Type 1A = 1GB / rank1
Type 1B = 1GB / rank2
Type 2A = 2GB / rank1
Type 2B = 2GB / rank2
Type 4A = 4GB / rank1
Type 4B = 4GB / rank2
Type 8B = 8GB / rank2

Steps to Follow
Follow these steps to configure OPL memory.
A supported memory configuration must be built according to the above "Applicable memory population rules" section.

1. For every PSB, use the appropriate table:
• M3000
• UNI-XSB mode M4000 / M5000
• QUAD-XSB mode M4000 / M5000
• UNI-XSB mode M8000 / M9000
• QUAD-XSB mode M8000 / M9000
2. For every PSB, follow the additional group rules.

Product
M3000
M4000
M5000
M8000
M9000

This solution has no attachment