Sun Fire[TM] 3800, 4800, 4810, 6800, E4900, and E6900: Sun Fire Segments, Domains, and Power Grids

Asset ID:	1-71-1316607.1
Update Date:	2012-10-04
Keywords:

Solution Type Technical Instruction Sure

Solution 1316607.1 : Sun Fire[TM] 3800, 4800, 4810, 6800, E4900, and E6900: Sun Fire Segments, Domains, and Power Grids

Applies to:

Sun Fire 6800 Server - Version Not Applicable and later
Sun Fire E6900 Server - Version Not Applicable and later
Sun Fire 4800 Server - Version Not Applicable and later
Sun Fire 3800 Server - Version Not Applicable and later
Sun Fire E4900 Server - Version Not Applicable and later
Information in this document applies to any platform.

Goal

The aim of this document is to describe:

Segments
Domain
Power Grids

for Serengeti platforms

Fix

Legenda:

Name	Also called
Repeater	RP / FS / Fireplane Switch
Partition	Segment
Serengeti	Sun Fire
MAC	Media Access Control The serial number burned into Ethernet and Token Ring adapters that identifies that network card from all others

Note:
Sun Fire 6800 images and tables also covers E6900 system
Sun Fire 4810/4800/3800 images and tables also covers E4900 system

Domain Allocations for Sun Fire E6900/6800/4810/E4900/4800/3800

In a Sun Fire 6800/6900, a single Segment mode requires 4 Fireplane Switches; and in a dual Segment mode requires 2 Fireplane Switches/Segment.
The other platforms in single Segment mode requires 2 Fireplane Switches; and in dual Segment mode requires 1 Fireplane Switch/Segment.

Notes specific to E4900/4810/4800/3800:

Domain C with MAC B in the Dual Segment E4900/4810/4800/3800 is not a typo; see What about the MAC?
When Domain B -> Domain C only the MAC and hostID are migrated. The Domain Settings are not migrated.

Domain B and C should match: use setupdomain at the domain shell.
Domain B and C ACL should match: use setupplatform at a platform shell.

Effects of Segmenting and Adding Multiple Domains

Effects creating two Segments

Data Bandwidth across Fireplane is reduced to half (4.8GB/s to 2.4GBs).
Each Segment is logically independent from the other; i.e. The resources from one Segment will not affect the other.

Effects of creating two Domains in a Segment:

Domain Snooping Bandwidth is reduced to half (9.6GB/s to 4.8GB/s).
Two Domains in a Segment share a common resource: a set of Fireplane Switches.
Within a Segment, a failure in one Domain can influence the other Domain.

If any Fireplane Switch Fails

Associated Domains cannot be brought up.

Components such as system clocks, power and cooling are fully redundant. The system interconnect is redundantly configurable.

From our point of view, redundant Fireplane Switches means the system can be reconfigured to avoid using the failed Fireplane Switch.

Recommendations

Allocate Domain Resources on the Power Grid boundries.
Try to use the above configurations which gives you 100% Domain Snooping Bandwidth.
If creating two Domains then Segment the machine and create one Domain / Segment; this will increase Reliability and Availability.

Keep in mind:

With the varying number of system configuration effecting Domain Snooping Bandwidth and/or Data Bandwidth across the Fireplane, does not necessarily mean applications will run slower.
These are maximum performance numbers. If your application performance is not bound by these characteristics, Segmenting a Sun Fire and/or creating two Domains in a Segment will likely have little or no effect whatsoever on your application.
Consider what recommended means. This definition will vary widely; it is based almost entirely on what the customer is trying to achieve.

What about the MAC?

With the current implementation, the Media Access Control (MAC) addresses and HostIds for each of the four domains on Sun Fire 6800/E6900 are fixed. Although all four domains can run simultaneously, if a Solaris image moves from Domain A to Domain C it will see a different MAC address and HostId. Obviously this is not the desired behavior.

On the other platforms, Sun Fire E4900/4810/4800/3800, the ID board only contains two MAC addresses for the domains. The first is reserved exclusively for Domain A, the second is assigned to Domain B or C depending on Segmentation mode. It is not possible to use Domain D on these systems, nor is it possible to have both B and C running at the same time.

Resource Allocation along Power Grids

When configuring a system keep in mind the physical layout of the system.

Domain A or B cannot be brought up
SB0, SB2, SB4, IB6, or IB8 cannot be powered up

Power Grid 1 Fails (only on Sun Fire 6800/E6900)

Domain C or D cannot be brought up
SB1, SB3, SB5, IB7, or IB9 cannot be powered up

SBx and IBx boards can be allocated to any Domain in any Segment:

Try and keep the resource allocation on the same Power Grid.
Creating a Domain with resources from another Power Grid reduces the Reliability and Availability.

Attachments

This solution has no attachment