Sun SPARC Enterprise M8000 Server
 
Sun SPARC Enterprise M4000 Server
 
Sun SPARC Enterprise M3000 Server
 
Sun SPARC Enterprise M9000-32 Server
 
Sun SPARC Enterprise M5000 Server
 
Sun SPARC Enterprise M9000-64 Server
 

PLA-Support>Sun Systems>SPARC>Enterprise>SN-SPARC: Mx000
 
.Old GCS Categories>Sun Microsystems>Servers>OPL Servers
 

Applies to:

Goal

This doc describes the various ways to gather diagnostic data from either the XSCF or the domain and XSCF combined.

It applies to all M-series servers, and describes the usage of Explorer 7.0 or newer, on both Solaris 10 and Solaris 11.

Fix

Running snapshot on Sun SPARC Enterprise Mx000 (OPL) eXtended System Control Facilty ( XSCF ).

Snapshot stores the collected data on a remote network  host or on  an  external media device, based upon the use of the -t or -d option. To store the collected data on a remote network  host  using  the -t option,  the command line must specify a host name (or IP address), a target directory on the remote  network host,  and the user name of a user on the remote host.  When storing  data on a remote network host, snapshot opens a network connection using SSH to act as a data pipe to the remote file.

Example of running snapshot to a remote host:   XSCF> snapshot -L F -t <username>@<hostname or IP address>:<location to write to>
  XSCF> snapshot -L F -t [email protected]:/home/joe/logs/x
Example of running snapshot to a USB device:
   XSCF>snapshot -L F -d usb0

Bare in mind, that the external media device connected to the XSCF's USB port, is expected to have a partition 1, formatted with the FAT32 filesystem.
The external USB device can have multiple partitions, as long as partition 1 is FAT32. That partition will then be used by the snapshot command.

Running explorer on Sun SPARC Enterprise Mx000 (OPL) Servers

Like on any other Sun Server, you can run Explorer on the Solaris instance(s) on that server. When you run explorer with the option -xscfextended, it is capable of collecting a Snapshot on the specified XSCF('s). The Snapshot data will be in the xscf sub-directory of the host's explorer. It does make sense when running an explorer on a Mx000  Server domain, to run it with -xscfextended, that way a single explorer data file contains information about the domain, and its main XSCF. However, to collect the XSCF snapshot via explorer, the explorer does not need to run on the platform containing that XSCF. As long as explorer can reach the XSCF over the network, it is capable or collecting the snapshot data via the -xscfextended option.

In this document, we will provide an example of running explorer on a domain of a Sun SPARC Enterprise M8000.  It gathers both the domain info and the eXtended System Control Facility ( XSCF ) info, via ssh. The system that the explorer package is loaded on and runs on, is referred to as the Explorer host. The Explorer host can be a dedicated host from which to collect explorers, but it can also be the domain. In this case our M8000 domain is the explorer host.

The explorer version must be 7.0 or above for Mx000 machines. We strongly recommend to run the latest version of Explorer, which is currently 7.0.  Explorer may be downloaded from "Oracle Services Tools Bundle (STB) - RDA/Explorer, SNEEP, ACT (Doc ID 1153444.1)"  .

NOTE: By choosing the Mx000 Solaris domain as the explorer host it is assumed it is healthy enough to stay up long enough for the customer to run the explorer and then get the explorer output and send to Oracle for analysis. If the domain will not boot or stay up, consider using another machine as the explorer host or simply gather just the XSCF snapshot manually as described in the first section of this document, "Running snapshot on Sun SPARC Enterprise Mx000 (OPL) eXtended System Control Facility ( XSCF )" .

Summary of the steps to get explorer to collect XSCF snapshot

1.  Find an existing (or create a new) explorer user login on the xscf .
2.  Verify explorer user login on XSCF was added and the correct permissions were applied.
3.  As root on the explorer host, establish the known_hosts entry on the explorer host for the XSCF.
4.  As root on explorer host, generate new dsa key, with empty passphrase .
5.  Copy the newly generated id_dsa.pub content from the explorer host .
6.  Login to the xscf with the XSCF explorer login to add the public key generated in step 4 .
7.  Setup archiving from the XSCF to the explorer host .
8.  Login to the xscf with the XSCF explorer login and test connectivity via snapshot -T -D.
9.  As root on the explorer host run ./explorer -w xscfextended,default.
10.  When prompted ; Supply the IP address of the xscf.
11.  When prompted ; Supply the user login on the xscf.

Step 10 and 11 can be prevented from happening, by editing the file /etc/opt/SUNWexplo/xscfinput.txt, and add host/login combination of the XSCF('s) you want to collect a snapshot on:

bash-3.00# more xscfinput.txt
# Input file for extended data collection
# Format is HOST USER [PASSWORD]
# Include password for telnet connection (not recommended)
# Omit password for SSH connection using public key authentication
# Explorer requires the mode of xscfinput.txt is 0400 or 0600
10.4.145.76   explorer   
10.4.145.89   explorer

Explorer output will be in /opt/SUNWexplo/output (unless some other default location has been specified), and the xscf snapshot portion is in /opt/SUNWexplo/output/explorer.809210d0.v4u-m8000a-dom00-2011.05.03.15.50/xscf/10.4.145.76 .

# pwd
/opt/SUNWexplo/output/explorer.809210d0.v4u-m8000a-dom00-2011.05.03.15.50/xscf/10.4.145.76
#ls
snapshot.err
snapshot.out
v4u-m8000a-xscf0_10.4.145.76_2011-05-03T23-02-48.zip

Before diving into the details, here is a brief description of snapshot, log archiving and explorer.

Snapshot is a command on the XSCF that gathers all the XSCF error logs etc. and copies them to a user specified location. This location can be a USB drive plugged into the XSF USB port, or a user specified network location. To automate the XSCF snapshot collection using Explorer, you need to configure the Log Archiving functionality. Log Archiving itself does not have to be enabled for explorer to be successful in collecting the Snapshot, if Log Archiving is configured correctly, but set to disabled, you can still use explorer to collect your XSCF snapshot.

Log archiving on the XSCF allows the XSCF to periodically copy all it's logs to a specified location. Log archiving is setup on the XSCF via the setarchiving command.
Persistent storage  space  on the Service Processor is limited. Some logs may grow to the point where old log  entries must  be  overwritten  or  deleted. Log archiving allows the user to  set  up  the  Service  Processor  to  automatically archive its log data on a remote host.
Please note Logs archived on the archive host should be  rotated at  regular intervals to avoid loss of log information.
Use logadm(1M) to configure log rotation on a system that runs the Oracle Solaris OS .

Explorer starts the snapshot command on the XSCF, with the -T and the -D option. The -D option will dictate in which directory the secure temporary files will be stored by explorer. Please check the explorer manual page for the default location used by Explorer, these locations depend on Explorer being installed as an IPS package or a SVR4  package. To override this default location, set the variable EXP_TMP to an alternative location, use the value of EXP_TMP when testing Log Archiving in step 8. For documentation purposes , we will refer to ${EXP_TMP} where needed. Please note , that starting with Explorer 7.0, ownership of directory ${EXP_TMP} must be root, and permissions of ${EXP_TMP} must be 0700.  More details on Explorer can be found in the Oracle® Explorer Data Collector User’s Guide .

Detailed steps to get explorer to collect XSCF snapshot

On the XSCF there needs to be a user that has platadm privileges. For this detailed explanation, we choose "explorer" as the XSCF explorer login. In the example, the IP address of the XSCF is 10.4.145.76 , the explorer host IP address is 10.4.145.87 .

1. Find an existing (or create a new) explorer user login on the XSCF.

If such a user does not exist, add a user login for explorer on the XSCF. As explorer runs, it will prompt for this user (i.e. 'Remote username:' ), unless /etc/opt/SUNWexplo/xscfinput.txt has been filled with host/login combination to automate the process. 

 

   XSCF> adduser explorer
   XSCF> setprivileges explorer platadm
   XSCF> password explorer
   New XSCF password:
   Retype new XSCF password:

2.  Verify explorer user login on XSCF was added and the correct permissions were applied.

   XSCF> showuser explorer
   User Name: explorer
   UID: 100
   Status: Enabled
   Minimum: 0
   Maximum: 99999
   Warning: 7
   Inactive: -1
   Last Change: May 03, 2011
   Password Expires: Never
   Password Inactive: Never
   Account Expires: Never
   Privileges: platadm

3. As root on the explorer host, establish the known_hosts entry on the explorer host for the XSCF.
 
As root on the explorer host, manually ssh into the XSCF, using the XSCF explorer login one time, to establish the known_hosts entry on the explorer host for the XSCF. Remember,  explorer must be run as user root, hence the use of user root.

Note: Answer yes the first time ssh connects into the XSCF.

   # ssh -l explorer 10.4.145.76
   The authenticity of host '10.4.145.76 (10.4.145.76)' can't be established.
   RSA key fingerprint is 80:c0:74:eb:4d:89:5b:b0:f0:ee:7b:44:72:2f:43:b8.
   Are you sure you want to continue connecting (yes/no)? yes

  Warning: Permanently added '10.4.145.76' (RSA) to the list of known hosts.
   [email protected]'s password:

   Last login: Thu May  3 15:37:41 2011 from somehost.oracle.com

   XSCF>

4. As root on explorer host, generate a new dsa key, with empty passphrase.

We need the key on the XSCF, to properly configure the Log Archiving, which is needed for the Snapshot SSH transport.
Leave the passphrase empty. Depending on the SSH setup on your explorer host, the location of the generated keys can differ. Also take note, XSCF is only capable of using SSH type 2.

# ssh-keygen -t dsa
Generating public/private dsa key pair.
Enter file in which to save the key (//.ssh/id_dsa):
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in //.ssh/id_dsa.
Your public key has been saved in //.ssh/id_dsa.pub.
The key fingerprint is:
9a:ed:5b:4d:05:92:8b:b1:36:f2:f7:a0:00:86:86:bf root@host

5.  Copy the newly generated id_dsa.pub content from the explorer host .

Copy the content of id_dsa.pub. The key is a single line in the text file id_dsa.pub, you can copy it either with or without the trailing newline.

# pwd
/.ssh
# cat id_dsa.pub
ssh-dss AAAAB3NzaC1kc3MAAACBAILP6i1v7rBPc57vTdzfvwQcDJ/qpUcRgZG6i5QvIvBhCQ6U
MeAdyDlCMXsXNgOWKmNLgbM0wLsHrcXuxzYxSwv4ArzuK8gh23t3XT5PsKeIvSvQFWieArWcW4Fc
EGYMs3JwwCjwFLeIpdKW5HoJa78uH5mHFuWv7EJdA73iNHAAAAFQDD/fmQPy3TdHX0iEcm+XcVOi
znQAAAIAuZHPrey2IbhiDIiEp5hB/09i7zW2j0peky2Q8WgMPrs7ABlX00L2p0/3wAm1KgNt87vF
9ZAaj12ikV5/66s7EP9o2Yp5kfvEMLvisyUvRqbct5D7JRySLcTESG0JjunpcJcM6nfT1Sj4iG3q
/85zbMEsP8GY86o2QJ7tbBQogAAAIA2CGJwSyGlkQVWkiozLzb/4QchEqtiJqXSrSXHRk8QW9ge
KAsnsH+poT5J+j9PQx6XyTFjTLC2m+IGXDxaIDkTPwCNt2nAYP+5NJOjtj7eUTWLuD5WiruLN1k
OQgiQbTLGrdVrXzSs0pMQY6ZM1JOPoHLS2vAyPvgAEYi8ZXVw==root@v4u-m8000a-dom00
#

6. Login to the xscf with the XSCF explorer login, to add the public key generated in step 4, and copied in step 5.


From the explorer host, as root, run ssh -l explorer to login to the XSCF. Then, on the XSCF, add the public key by running setssh -c addpubkey .
In the example below, I copied all of the key, including the trailing newline, then hit <return>, followed by <ctrl-d>.

ssh -l explorer 10.4.145.76Last login: Thu May  3 18:37:41 2011 from somehost.oracle.com
XSCF> setssh -c addpubkey
   Please input a public key:
   ssh-dss AAAAB3NzaC1kc3MAAACBAILP6i1v7rBPc57vTdzfvwQcDJ/qpUcRgZG6i5QvIvBhCQ6U
   MeAdyDlCMXsXNgOWKmNLgbM0wLsHrcXuxzYxSwv4ArzuK8gh23t3XT5PsKeIvSvQFWieArWcW4Fc
   EGYMs3JwwCjwFLeIpdKW5HoJa78uH5mHFuWv7EJdA73iNHAAAAFQDD/fmQPy3TdHX0iEcm+XcVOi
   znQAAAIAuZHPrey2IbhiDIiEp5hB/09i7zW2j0peky2Q8WgMPrs7ABlX00L2p0/3wAm1KgNt87vF
   9ZAaj12ikV5/66s7EP9o2Yp5kfvEMLvisyUvRqbct5D7JRySLcTESG0JjunpcJcM6nfT1Sj4iG3q
   /85zbMEsP8GY86o2QJ7tbBQogAAAIA2CGJwSyGlkQVWkiozLzb/4QchEqtiJqXSrSXHRk8QW9ge
   KAsnsH+poT5J+j9PQx6XyTFjTLC2m+IGXDxaIDkTPwCNt2nAYP+5NJOjtj7eUTWLuD5WiruLN1k
   OQgiQbTLGrdVrXzSs0pMQY6ZM1JOPoHLS2vAyPvgAEYi8ZXVw==root@v4u-m8000a-dom00
<return>
<ctrl-d>

At this point, as user root logged in on your explorer host, you should be able to ssh into the XSCF using the XSCF explorer login , without being prompted for a password. If you get prompted for a password, first fix that before continuing, because if "ssh -l explorer 10.4.145.76" prompts for a password, the -xscfextended option of explorer will not work.

# id
uid=0(root) gid=0(root)
# ssh -l explorer 10.4.145.76
Last login: Thu May  3 18:38:15 2011 from somehost.oracle.com
XSCF>

7. Setup archiving from the XSCF to the explorer host .

Log Archiving itself does not have to be enabled for explorer to be successful in collecting the Snapshot, if Log Archiving is configured correctly, but set to disabled, you can still use explorer to collect your XSCF snapshot.  It is up to the customer to enable or disable Log Archiving . With the Explorer 7.0 (or newer) restrictions on ownership and permissions of the temporary directory for secure files (${EXP_TMP}, you need to setup archiving for user root. If you enter an invalid password for the archiving username, or if that user has no write-access to the provided location, enabling of archiving will fail.

Also, for the explorer host to be able to receive the snapshot output, ssh-ing into the explorer host with be allowed for user root.

 

   XSCF> setarchiving disable
   XSCF> setarchiving -k download -r -t [email protected]:/usr/rob
   Fingerprint: 27:5e:08:5a:34:7c:3e:7b:45:d8:75:7a:43:21:8a:d2
   Accept this public key (yes/no)  yes
   Enter ssh password for [email protected]:

  XSCF> setarchiving enable
   Testing the archiving configuration...
   Logs will be archived to 10.4.145.87

8. Login to the xscf with the XSCF explorer login and test connectivity via snapshot -T -D ${EXP_TMP}.

   NOTE: The '-T' Specifies executing snapshot in SSH target mode using the value for user@host:directory previously set using setarchiving.

   NOTE: The '-D' Specifies the location of the secure temporary files, as used by explorer's xscfextended module
 
   As the xscf user login for explorer use:

   XSCF> snapshot -T -D ${EXP_TMP}
   Setting up ssh connection to [email protected]...
   Collecting data into [email protected]:/opt/SUNWexplo/tmp/v4u-m8000a-xscf0_10.4.145.76_2007-05-04T22-26-13.zip
   Data collection complete
   XSCF>

9. As root on the explorer host run ./explorer -w xscfextended,default.

Oracle has successfully tested this procedure on both Oracle Solaris 10 and Solaris 11, with STB7.0 (containing Explorer 7.0).

If you are running an Explorer version earlier then Explorer 6.10, you could hit Bug 13831855 (Update) [1330:STB:Status 10] EXPLORER -W XSCFEXTENDED FAILS TO CONNECT SNAPSHOT: FAILED TO CREATE REMOTE FILE . If you do, it is needed to change the file permission on /opt/SUNWexplo/tmp, otherwise the SSH daemon on the explorer host will fail to collect the snapshot. This change of file permission needs to happen regardless the fact if you are going to automate the snapshot collection via explorer.

# chmod 1777 /opt/SUNWexplo/tmp
# cd /opt/SUNWexplo/bin
# ./explorer -w xscfextended,default
   May 03 06:56:02 v4u-m8000a-dom00[25803] explorer: explorer ID: explorer.809210d0.v4u-m8000a-dom00-2007.05.03.13.56
   May 03 06:56:02 v4u-m8000a-dom00[25803] patch: RUNNING
   May 03 06:56:02 v4u-m8000a-dom00[25803] pkg: RUNNING
   May 03 06:56:22 v4u-m8000a-dom00[25803] sysconfig: RUNNING
.
.
May 03 06:56:52 v4u-m8000a-dom00[25803] xscfextended: RUNNING
Type return to end the list Remote hostname or IP address:

10. When prompted ; Supply the IP address of the xscf.

If you have not filled /etc/opt/SUNWexplo/xscfinput.txt with host/login combinations to automate the snapshot collection via explorer, you will be prompted to enter the remote hostname or IP address of the XSCF. The xscfextended portion runs about 4 minutes, after which it will prompt for the next remote hostname or IP address.
If you have entered all your XSCF's, just hit return to make explorer finish the xscfextended part.

Remote hostname or IP address: 10.4.145.76
Remote username:

11. When prompted ; Supply the user login on the xscf.

Enter the XSCF explorer login you setup earlier, which is named explorer. Hit return when to use SSH for the data transport, after which explorer will contact the XSCF to start snapshot collecting. When ready, it will prompt you for the next XSCF.

Remote hostname or IP address: 10.4.145.76
Remote username: explorer
Telnet password or blank to use SSH connection:
.
.

May 03 07:00:22 v4u-m8000a-dom00[25803]10 12:12:28 m3000-tvp540-d[1770] xscfextended: Data gathering complete for 10.4.145.76.

Remote hostname or IP address:
May 03 07:01:22 v4u-m8000a-dom00[25803] explorer: data collection complete
May 03 07:01:22 v4u-m8000a-dom00[25803] explorer: removing previous explorers from /opt/SUNWexplo/output
May 03 07:01:22 v4u-m8000a-dom00[25803] explorer: Explorer finished

The result of the Oracle Explorer data collection can be found in
/opt/SUNWexplo/output directory. The packed result is /opt/SUNWexplo/output/explorer.809210d0.v4u-m8000a-dom00-2011.05.03.06.50.tar.gz
You can send this file to Oracle

explorer, xscfextended, OPL
Previously Published As @ 89403

This solution has no attachment