Sun Blade 2500 Workstation
 
Sun Fire V240 Server
 
Sun Fire V210 Server
 
Sun Blade 1500 Workstation
 

GCS>Sun Microsystems>Desktops>Workstations
 
GCS>Sun Microsystems>Servers>Entry-Level Servers
 

This is discussed in <Document: 1006000.1> and you are encouraged to read this before continuing any further.

However circumstances may arise where it is necessary or desirable to force the operational parameters on an interface.

The purpose of this document is to show the methods which may be employed in order to configure the "bge" gigabit Ethernet driver in order to achieve this.  

Note : The document contains many references to pages within the Solaris[TM] reference manual. If you are not sure how to view these "man pages" you should consult the entry which describes the use of the man(1) command by running,

# /usr/bin/man -M /usr/man -s1 man

1. Introduction

The name "bge" refers to the fourth generation of Sun Microsystems' gigabit Ethernet products which includes a number of on-board interfaces and a GLD (see gld(7D)) based STREAMS driver. It operates on both fibre and copper media, supports VLANs and is capable of 1000 Mbit/s (or 1 Gbit/s) maximum speed.

If the kernel module is loaded you can determine the version of the bge(7D) driver installed on your system using the modinfo(1M) command, e.g.,

# /usr/sbin/modinfo | /usr/bin/grep bge
80 1032f15e   8a90 195   1  bge (BCM570x driver v0.22)

Alternatively, if you know the location of the driver module, you can use the strings(1) command as shown in the following example.

# /usr/bin/strings /platform/sun4u/kernel/drv/sparcv9/bge | /usr/bin/grep BCM
BCM570x driver v0.23

N.B .: For correct operation of the interface you should allow both ends of the link to auto-negotiate or force both ends to use the same settings. Failure to comply with this may result in poor network performance or loss of the link which is often misinterpreted as a hardware fault. By default the interface will attempt to auto-negotiate with the link partner.

There are two ways to modify the operational parameters on these interfaces:

Both of these methods are described in this document.

While the ndd command may be used to achieve the same results as the driver configuration file its use is less rigorous leading to greater scope for error. Furthermore, the use of ndd to make static changes is not recommended by Sun Microsystems. By "static" we mean changes which will remain in place even if the system is rebooted.

Note : In the past it has also been possible to use the /etc/system file to configure NIC parameters but this cannot be done with the "bge" driver.

2. Device Instance Numbers

The /etc/path_to_inst file records mappings of physical device names to instance numbers. Each line in this file contains the following three fields.

"physical-name" instance-number "driver-binding-name"

This is described in the man page for path_to_inst(4).

For example, on a Sun Fire V210 or V240 Server we see the following entries for the bge driver binding name.

# /usr/bin/grep '"bge"' /etc/path_to_inst
"/pci@1f,700000/network@2" 0 "bge"
"/pci@1f,700000/network@2,1" 1 "bge"
"/pci@1d,700000/network@2" 2 "bge"
"/pci@1d,700000/network@2,1" 3 "bge"

These are the onboard interfaces and will be reported as bge0, bge1, bge2 and bge3 in the output from the ifconfig -a command if they are plumbed.

We will look at this file in more detail when we show examples of how to use the driver configuration file in section 4.2.

Note : The /etc/path_to_inst file is maintained by the Solaris[TM] Operating System and should not be edited without first consulting Sun Services.

3. Using the ndd(1M) Command

3.1 How to List the Available Parameters

In order to view the complete list of operational parameters available for a specific interface (device instance) you can run the following ndd(1M) command as the super-user.

# /usr/sbin/ndd /dev/bge<instance-number> \ 

where <instance-number> should be replaced by the instance number shown in the /etc/path_to_inst file for the device in which you are interested.

For example,

# /usr/sbin/ndd /dev/bge0 \
(read only)
autoneg_cap                   (read only)
pause_cap                     (read only)
asym_pause_cap                (read only)
1000fdx_cap                   (read only)
1000hdx_cap                   (read only)
100T4_cap                     (read only)
100fdx_cap                    (read only)
100hdx_cap                    (read only)
10fdx_cap                     (read only)
10hdx_cap                     (read only)
adv_autoneg_cap               (read and write)
adv_pause_cap                 (read and write)
adv_asym_pause_cap            (read and write)
adv_1000fdx_cap               (read and write)
adv_1000hdx_cap               (read and write)
adv_100T4_cap                 (read only)
adv_100fdx_cap                (read and write)
adv_100hdx_cap                (read and write)
adv_10fdx_cap                 (read and write)
adv_10hdx_cap                 (read and write)
lp_autoneg_cap                (read only)
lp_pause_cap                  (read only)
lp_asym_pause_cap             (read only)
lp_1000fdx_cap                (read only)
lp_1000hdx_cap                (read only)
lp_100T4_cap                  (read only)
lp_100fdx_cap                 (read only)
lp_100hdx_cap                 (read only)
lp_10fdx_cap                  (read only)
lp_10hdx_cap                  (read only)
link_status                   (read only)
link_speed                    (read only)
link_duplex                   (read only)
link_autoneg                  (read only)
link_rx_pause                 (read only)
link_tx_pause                 (read only)
loop_mode                     (read only)

Notice that some of these parameters may be changed by the super-user (read and write) and that some have values which are set by the driver (read only). The parameters listed by this command will depend upon the version of the bge(7D) driver installed. The example output shown above was produced by the following driver version and earlier versions may not list so many parameters.

# /usr/sbin/modinfo | /usr/bin/grep bge 103  13f27f5   b13b 202   1  bge (BCM579x driver v0.39)

In order to force the speed and duplex settings on an interface you must set the adv_autoneg_cap parameter to '0'.

It is advisable to set one, and only one, of the following parameters to '1' depending on whether you want to force 1000 Mbit/s full duplex, 1000 Mbit/s half duplex, 100 Mbit/s full duplex, 100 Mbit/s half duplex, 10 Mbit/s full duplex or 10 Mbit/s half duplex respectively, the others should be set to '0'.

adv_1000fdx_cap
adv_1000hdx_cap
adv_100fdx_cap
adv_100hdx_cap
adv_10fdx_cap
adv_10hdx_cap

The ramifications of setting more than one of these parameters to '1' are dependent upon the revision of the driver being used and these guidelines should be followed irrespective of the method of configuration which you have chosen to use.

Note : The man page for the ndd(1M) command states the following.

"The parameters supported by each driver may change from release to release. Like programs that read /dev/kmem, user programs or shell scripts that execute ndd should be prepared for parameter names to change."

3.2 Examining Current Parameter Values

3.2.1 Command Syntax

In order to examine the current value of a parameter for a specific interface we run the ndd command with the following syntax as the super-user.

# /usr/sbin/ndd /dev/bge<instance-number> <parameter-name>

For example,

# /usr/sbin/ndd /dev/bge1 adv_autoneg_cap
1

3.2.2 An Example Script

The following Bourne Shell script may be used to query the current value of each parameter.

#!/bin/sh
#
PATH=/usr/bin:/usr/sbin
usage="usage: `basename $0` <instance-number>"
if [ $# -ne 1 ]; then
echo $usage
exit 2
fi
for p in `ndd /dev/bge$1 \  | awk '{print $1}' | grep -v \ `
do
echo "$p: `ndd /dev/bge$1 $p`"
done
exit 0

If we name this script "bge_query" and place it in the /tmp directory with appropriate execute permissions it may be run by the super-user as follows.

# /tmp/bge_query <instance-number>

where <instance-number> should be replaced with the device instance for the interface which we wish to query.

For example,

# /tmp/bge_query 0
autoneg_cap: 1
pause_cap: 1
asym_pause_cap: 1
1000fdx_cap: 1
1000hdx_cap: 1
100T4_cap: 0
100fdx_cap: 1
100hdx_cap: 1
10fdx_cap: 1
10hdx_cap: 1
adv_autoneg_cap: 1
adv_pause_cap: 1
adv_asym_pause_cap: 1
adv_1000fdx_cap: 1
adv_1000hdx_cap: 1
adv_100T4_cap: 0
adv_100fdx_cap: 1
adv_100hdx_cap: 1
adv_10fdx_cap: 1
adv_10hdx_cap: 1
lp_autoneg_cap: 1
lp_pause_cap: 0
lp_asym_pause_cap: 0
lp_1000fdx_cap: 0
lp_1000hdx_cap: 0
lp_100T4_cap: 0
lp_100fdx_cap: 0
lp_100hdx_cap: 1
lp_10fdx_cap: 0
lp_10hdx_cap: 1
link_status: 1
link_speed: 100
link_duplex: 1
link_autoneg: 1
link_rx_pause: 0
link_tx_pause: 0
loop_mode: 0

So, the values shown above are the current settings for bge0.

3.2.3 Interpreting the Values

The important parameters and their meaning are shown in table 1 below. The parameters whose names begin with "link_" show the mode in which the link is currently operating while those beginning "adv_" show the capabilities which the interface is advertising on the link.

N.B . The parameter values and their meaning are liable to change between versions of the driver and should not be relied upon in user programs or shell scripts. Interpretation of the parameters if the link is down (link_status = 0) is beyond the scope of this document.

Table 1 - Interpretation of Parameter Values

Notes:

3.3 Changing Parameter Values

In order to change the value of a parameter for a specific interface we run the ndd command with the following syntax as the super-user.

# /usr/sbin/ndd -set /dev/bge<instance-name> <parameter-name> <value>

For example, to set the value of adv_100fdx_cap to '0' for bge2 we would use,

# /usr/sbin/ndd -set /dev/bge2 adv_100fdx_cap 0
#

3.4 Dynamic Changes Using ndd

To illustrate a dynamic change let's assume that we want to force bge0 to operate at 100 Mbit/s full duplex. In order to do this we would need to run the following commands.

# /usr/sbin/ndd -set /dev/bge0 adv_1000fdx_cap 0
# /usr/sbin/ndd -set /dev/bge0 adv_1000hdx_cap 0
# /usr/sbin/ndd -set /dev/bge0 adv_100fdx_cap 1
# /usr/sbin/ndd -set /dev/bge0 adv_100hdx_cap 0
# /usr/sbin/ndd -set /dev/bge0 adv_10fdx_cap 0
# /usr/sbin/ndd -set /dev/bge0 adv_10hdx_cap 0
# /usr/sbin/ndd -set /dev/bge0 adv_autoneg_cap 0

Any changes made at the command line in this manner will be lost if the system is rebooted.

3.5 Static Changes Using ndd (Not recommended)

In order to make static changes in versions of Solaris[TM] prior to Solaris 10 you can place the necessary commands in a startup script. By "static" we mean that the changes will remain in place across system reboots.

For example, let's assume that we want to force bge0 to operate at 100 Mbit/s full duplex. We can create a file named "net-tune" in the /etc/init.d directory with the following content.

#!/sbin/sh
#
PATH=/sbin:/usr/bin:/usr/sbin
echo "Configuring Network Interfaces"

case "$1" in

'start')
ndd -set /dev/bge0 adv_1000fdx_cap 0
ndd -set /dev/bge0 adv_1000hdx_cap 0
ndd -set /dev/bge0 adv_100fdx_cap 1
ndd -set /dev/bge0 adv_100hdx_cap 0
ndd -set /dev/bge0 adv_10fdx_cap 0
ndd -set /dev/bge0 adv_10hdx_cap 0
ndd -set /dev/bge0 adv_autoneg_cap 0
;;
esac
exit 0

Then we create a symbolic link to this file from the /etc/rc2.d directory, i.e.,

# /usr/bin/ln -s /etc/init.d/net-tune /etc/rc2.d/S68net-tune

The script will then be run during the boot sequence and force bge0 to the desired mode of operation.

Note : In Solaris 10 the init.d(4) man page states the following.

"The service management facility (see smf(5)) is the preferred mechanism for service  initiation and termination. The init.d and rc .d directories are obsolete, and are provided for compatibility purposes only."

4. Using the Driver Configuration File

IMPORTANT NOTE: The bge NIC driver configuration file is considered a  non-editable  file by the Solaris package framework up through and including Solaris 10 Update 4.  This is also true for the majority of driver configuration files in Solaris.  Changes to  non-editable  Solaris files will survive reboots, thus they are considered static in that respect.   However, if a patch or a Solaris update delivers a new version of a  non-editable  file, the old file will be silently overwritten without merging any user-created content.

4.1 Where to Place the File

General information on driver configuration files can be found in the man page for driver.conf(4). The bge configuration file is named "bge.conf" and you may see documentation which states that it should be placed in the /platform/sun4u/kernel/drv directory. The reasons for this choice of location are discussed in the following InfoDoc which you may wish to consult.

Document ID:
 7
9861


Title:
 On Where to Place your Driver Configuration File

4.2 Specifying Entries

The syntax for a single entry in the file may take one of two forms:

The first form may be used to define settings for a particular device instance while the second form may be used to make global settings. Any number of property-name/value pairs may be specified in an entry and items within the entry may be separated by any number of newlines, spaces or tabs.

Global properties created using the second form may be overridden by entries with the first form.

Comments may be included in the file by specifying a '#' character at the beginning of the comment string which extends for the rest of the line.

4.2 Example File Content

Example 1

As a first example let's revisit our Sun Fire V240 Server which has the following bge device instances listed in the /etc/path_to_inst file previously discussed in section 2.

"/pci@1f,700000/network@2" 0 "bge"
"/pci@1f,700000/network@2,1" 1 "bge"
"/pci@1d,700000/network@2" 2 "bge"
"/pci@1d,700000/network@2,1" 3 "bge"    <--- bge3

To force all of the interfaces to run at 100 Mbit/s full duplex with the exception of bge3 which we want to force to run at 1000 Mbit/s (1 Gbit/s) full duplex we could place the following lines in the bge.conf file.

#
# Entries for global settings
#
adv_autoneg_cap=0;
adv_1000fdx_cap=0;
adv_1000hdx_cap=0;
adv_100fdx_cap=1;
adv_100hdx_cap=0;
adv_10fdx_cap=0;
adv_10hdx_cap=0;
#
# Entry to override global settings for bge3
#
name="bge" parent="/pci@1d,700000" unit-address="2,1"
adv_100fdx_cap=0
adv_1000fdx_cap=1;

The important points to note here are:

These parameter settings will take effect following a system reboot, only the ndd(1M) command may be used to effect dynamic changes. In this example the bge1 and bge2 interfaces will adopt the default behaviour and attempt to auto-negotiate with the link partner.

The bge(7D) driver will also recognise the property names "speed" and "full-duplex". Auto-negotiation will be disabled if either is set so you should never specify one of these properties without the other.

We could place the following lines in the bge.conf and they would have the same effect as those shown above.

#
# Global settings
#
speed=100;
full-duplex=1;
#
# Entry to override global settings for bge3
#
name="bge"
parent="/pci@1d,700000"
unit-address="2,1"
speed=1000;

The "speed" property may be given a value of 10, 100 or 1000 and the "full-duplex" property should be given a value of '0' or '1' depending on whether you wish to specify half or full duplex respectively.

Once the system has successfully rebooted you can check the parameter settings using the ndd command as described in section 3.2.

Example 2

As a second example, let's assume that we would like to force all of the bge interfaces on our V240 Server to operate at 1000 Mbit/s full duplex. In this case we need only place the following entry in the bge.conf file.

adv_autoneg_cap=0 adv_1000fdx_cap=1 adv_1000hdx_cap=0
adv_100fdx_cap=0 adv_100hdx_cap=0 adv_10fdx_cap=0 adv_10hdx_cap=0;

The items in this entry may be placed on a single line or spread over several.

It is also permissible to place each item in individual entries on separate lines, e.g.,

adv_autoneg_cap=0;
adv_1000fdx_cap=1;
adv_1000hdx_cap=0;
adv_100fdx_cap=0;
adv_100hdx_cap=0;
adv_10fdx_cap=0;
adv_10hdx_cap=0;

As an third alternative we could use the "speed" and "full-duplex" properties and place the following entries in the file.

speed=1000;
full-duplex=1;

Example 3

As a final example let's assume that we want to force bge0 to operate at 100 Mbit/s  full duplex on our V240 Server while allowing the other bge interfaces to auto-negotiate.

Referring to the /etc/path_to_inst device instances shown in example 1 we can place the following entry in our bge.conf file.

name="bge" parent="/pci@1f,700000" unit-address="2"
adv_autoneg_cap=0 adv_1000fdx_cap=0 adv_1000hdx_cap=0
adv_100fdx_cap=1 adv_100hdx_cap=0 adv_10fdx_cap=0 adv_10hdx_cap=0;

Alternatively, we could use the "speed" and "full-duplex" properties, e.g.,

name="bge" parent="/pci@1f,700000" unit-address="2" speed=100 full-duplex=1;

The Platform Software Architecture Review Committee (PSARC) is
chartered by the Systems Architecture Council (SAC) to review
architectural aspects of platform software. PSARC reviews the
architecture of all types of platform software shipped by Sun
Microsystems, and is not limited to Solaris.

The need for a standard set of 802.3 MII/GMII physical layer
statistics which apply to all MII/GMII Ethernet drivers is
described in case PSARC/2003/581 which may be viewed at the
following URL.

http://sac.sfbay.sun.com/Archives/CaseLog/arc/PSARC/2003/581/

The following is an extract taken from the case notes.

The kstat interface presents a clean, simple, and effective
location to obtain physical layer statistics from the driver to the
user layer. Therefore the use of 'ndd' for read only statistics
should be discouraged in favor of using kstats. The set of
statistics this case addresses is the MII/GMII physical layer
statistics. There's already a list of Physical statistics which
have been part of the set of 'ndd' variables as read only ndd
parameters. Those statistics date back to the implementation of
'hme' and should be moved to 'kstats' to facilitate a uniform
method of debugging link problems.

Recently those physical layer statistics have moved to 'kstats'
interface with the introduction of 'ce', this move was required to
allow for greater configurability provided by GMII interface. This
has been well received by users of Cassini and sets an excellent
precedent to follow for all other NIC cards both legacy and
future.

 Author: Neil Cheney

 May 2005

 

 

This solution has no attachment