| Home | Current Systems | Former STK Products | EOL Systems | Components | General Info | Search | Feedback |
| Sun System Handbook - ISO 4.1 October 2012 Internal/Partner Edition | ||
|
|
||
 |
||||||||||||
Solution Type Technical Instruction Sure Solution 1380335.1 : How to enable Oracle ASR (Auto Service Request) on Sun x64 rackmount servers and blades?
In this Document
Applies to:Sun Fire X4170 Server - Version: Not Applicable to Not Applicable - Release: N/A to N/ASun Blade 6048 System - Version: Not Applicable to Not Applicable [Release: N/A to N/A] Sun Fire X4275 Server - Version: Not Applicable to Not Applicable [Release: N/A to N/A] Sun Fire X4440 Server - Version: Not Applicable to Not Applicable [Release: N/A to N/A] Sun Fire X4450 Server - Version: Not Applicable to Not Applicable [Release: N/A to N/A] Information in this document applies to any platform. GoalHow to enable Oracle ASR (Auto Service Request) on Sun x64 rackmount servers and blades?SolutionThis document describes the steps to activate and enable ASR and the associated telemetry (ILOM and FMA) on Oracle x64 server and Blade systems.Whether this is a new installation of ASR, or if you already have some familiarity with the ASR software, it is highly recommended that you review the ASR architecture discussed below. The following diagram shows the end-state environment once ASR is installed. ASR Architectural Summary ASR Assets: ASR Assets are qualified systems that are configured to report its hardware telemetry to an ASR Manager. ASR Manager: The ASR Manager is a system that centrally accepts hardware telemetry data sent from a group of ASR Assets. The ASR Manager filters the incoming data and forwards potential fault telemetry to Oracle/ASR Backend systems. Additional features of the ASR Manager include the following: The ASR Manager is always installed first, followed by ASR Assets. You have the option to install more than one instance of an ASR Manager. The reasons to do this may be to support a large amount of ASR Assets and/or for organizational reasons, such as grouping ASR Assets by data center, support group, subnet, or other grouping scheme as desired by the customer. An ASR Asset communicates its telemetry to one ASR Manager only. The ASR Manager system can be installed as an ASR Asset. This way, the ASR Manager system can report its hardware telemetry, as does an ASR Asset. The telemetry data that is sent from the ASR Manager to the Oracle /ASR Backend Systems is encrypted. Oracle/ASR Backend Systems: The ASR backend infrastructure at Oracle collects all telemetry data forwarded to it from the ASR Manager, or ASR Managers if multiple instances are installed. The fault rule technology on these backend systems ascertain the reality of the fault telemetry, and forwards recognized faults to Oracle's Service Request system. From there, the following actions occur: A Service Request, also called a case, is created and assigned to an Oracle Support Engineer. At the same time, an email notification of the Service Request is sent to your support contact on record associated with the system reporting a fault. The Service Request is handled in accordance with the asset's Support or Warranty contract. Oracle Support Interaction: Once an Oracle Support Engineer begins working on the Service Request, the engineer may collect additional information from you to better determine resolution to the hardware issue. Resolution to the issue may involve system configuration or the following possibilities: Order and ship a replacement part with installation instructions to you. These are called Customer Replaceable Units (CRUs). Order and ship a replacement part to the customer site to be installed by an Oracle Field Engineer. These are called Field Replaceable Units (FRUs). The ASR Manager software runs as a plug-in application to the Oracle Automated Service Manager (OASM) software. OASM is an applications environment that manages various plug-in applications used mainly for system-level management. OASM simplifies different Oracle Network Service deployments and provides a secure transport mechanism for telemetry data. In addition to ASR, an example of another OASM plug-in application is Secure File Transport (SFT). ASR Manager Network Connection Requirements The ASR Manager System must have an internet connection – either a direct connection or through a proxy. If you access the internet through a proxy, check with your network administrator to collect information needed to configure the ASR Manager system. You will need to know: Proxy server name Proxy port number Proxy user name Proxy password NTLM Domain Name (if applicable) NTLM Host Name (if applicable) SASM Host Information (if applicable) Check and make note of the ASR Manager IP address: ifconfig -a To test the connection to Oracle, in a browser, go to: https://transport.oracle.com:443 You can also test your connection in a terminal window: telnet transport.oracle.com 443 /usr/sfw/bin/wget https://transport.oracle.com If you receive a “connected” message, the connectivity is successful. ASR Asset Network Connection Requirements Contact your network administrator to confirm or enable the following: Set-up firewall rules to allow bi-directional SNMP/UDP traffic to traverse between ASR Assets and the ASR Manager. Ensure that ASR Assets can send SNMP telemetry data out on port 162 to the ASR Manager. Ensure that the ASR Manager can communicate with ASR Assets, via http, using port 6481. Check and make note of the ASR Asset IP address: ifconfig -a If working with a system that has a service processor, such as Blade systems and X-series systems, obtain the service processor and/or the chassis IP address. These will be required for ASR installation Enable FMA Telemetry Please refer to http://docs.oracle.com/cd/E11857_01/install.111/e18475/configure_manage_asr_assets.htm#BEICBGHC. Follow the procedure below to enable FMA telemetry sources: Make sure you are logged in as root on the system whose telemetry you wish to enable. This could be either an ASR Manager or an ASR Asset system. Go to the directory where you untarred the Asset Bundle file, and then go to the specific ASR Asset Bundle directory. For example: On an ASR Asset: cd /file_copy_location/asrassetbundle On an ASR Manager: cd /opt/SUNWswasr/asrassetbundle Launch the ASR Asset menu. Run ./asrassetmenu.sh to display the menu: Welcome to the ASR asset menu ---------------------------------------------- 1) Check system for ASR qualifications 2) Add a trap-destination to SunMC agent 3) Add a trap-destination to FMA agent 4) Remove a trap-destination from SunMC agent 5) Remove a trap-destination from FMA agent 6) List SunMC agent trap-destinations 7) List FMA agent trap-destinations 8) Test event to verify ASR connectivity 9) Exit Select 3 to enable FMA telemetry. Respond to the script's prompts as follows: When prompted, âPlease enter Hostname or IP address of SASM ASR plugin,â enter the information for the ASR Manager. Whether you are enabling telemetry on the ASR Manager system or on ASR Asset systems, the hostname or IP entered must be for the installed ASR Manager. When prompted, âPlease enter SNMP port of SASM ASR plugin (q to quit) [162],â press [Return] or enter another port if you have changed your port settings for ASR When prompted, âDo you want to set trap-destination [y,n,q],â confirm the displayed information, enter Y, and press [Return]. The ASR Asset Menu then enables the telemetry and displays where the telemetry from this system will be sent (IP or hostname of the ASR Manager). Repeat for all ASR Assets. When complete, enable any other telemetry sources that may reside on the system (for example, ILOM). Refer to "Set Up Telemetry Sources to ASR", if required, and return to these instructions. If you have enabled all telemetry sources on an ASR Asset system, go to "Activate ASR Assets". Command Line Options for Setting Solaris FMA Trap Destinations You can incorporate ASR asset configuration into your automated provisioning process. The asrassetmenu.sh script now includes command line options for setting Solaris FMA trap destinations. To set a Solaris FMA trap destination from the command line: asrassetmenu.sh -solaris <destination_IP_address> <port_number> The asrassetmenu.sh script will exit with an error status value for any of these conditions: <destination_IP_address> not provided <port_number> not provided Trap destination unable to be set Enabling ASR on the ILOM: Please refer to http://docs.oracle.com/cd/E11857_01/install.111/e18475/configure_manage_asr_assets.htm#BEICBGHC Enable ILOM: Command Line Interface Follow these ILOM command line procedures below to enable ILOM telemetry. If you used the GUI method, as described in the previous procedure, you do not need to complete these steps. Open a terminal window and ssh to the IP address of the ILOM network interface: ssh IP_address_of_ILOM_interface Log in as root. Before enabling ILOM telemetry, it is important to understand that ILOM supports up to 15 user-configurable alert rules. It is one of these rules you must set to enable ILOM telemetry. Therefore, you must first choose one of these rules (1 to 15) to set. Before choosing, however, you must determine that the rule you select is not currently being used. You can use the web-based GUI method to determine this quickly, as discussed in "Enable ILOM: GUI Interface". Otherwise, run the following command to determine an available alert rule slot. For Rack Mount Servers and Blades: -> show /SP/alertmgmt/rules/x For the Sun Blade 6000 Chassis: -> show /CMM/alertmgmt/rules/x Important: Substitute x for a rule number (1 to 15). If you see that a rule level is set to anything else but disabled, the alert rule slot is occupied. Rerun the command above and substitute the rule number with the next number in sequence (for example, 2, then 3, etc.) until you find an available alert rule slot. Once an alert rule slot is determined, run one of the following commands. Pay close attention to the following data entry points before running the command: rules/x: For x, enter a number from 1 to 15, depending upon which alert rule slot you have determined is unoccupied using the previous step. destination: Enter the IP address of the ASR Manager. destination_port: The port is assigned to the default for SNMP (162). For ILOM versions 2.0.4.2 or lower, the port cannot be changed from the default. For ILOM versions higher than 2.0.4.2, the destination port can be changed if needed. If you wish to change the port, add the destination_port=xxx parameter into the command following the destination parameter. Note: If you are removing a trap, set the level parameter to disable. For Rack Mount Servers and Blades: -> set /SP/alertmgmt/rules/x type=snmptrap level=minor destination=IP_of_ASR Manager snmp_version=2c community_or_username=public For the Sun Blade 6000 Chassis: -> set /CMM/SP/alertmgmt/rules/x type=snmptrap level=minor destination=IP_of_ASR Manager snmp_version=2c community_or_username=public Generate test alerts. Type one of the following command paths to set the working directory: For a rack-mounted server SP, type: cd /SP/alertmgmt/rules For a blade server SP, type: cd /CH/BLn/SP/alertmgmt/rules For a chassis CMM, type: cd /CMM/alertmgmt/CMM/rules Type the following command to generate a test alert: ->set testalert=true If configured properly, you will receive a test Service Request email. Repeat this process (steps 1 through 6) for each ASR Asset using ILOM telemetry. If you have enabled all telemetry sources on an ASR Asset system, go to "Activate ASR Assets". Activate ASR Assets Follow the procedures below to activate ASR Assets. Once ASR Assets are activated, the configuration of the ASR Assets is complete. All assets to be activated should already have telemetry trap destinations set, as described in "Set Up Telemetry Sources to ASR". If this has not been done, complete that task first before continuing. Proceed to one of the following options, depending upon your circumstance: If you are activating a Sun Blade system, go to "Activate Blade Assets" to activate the system. If you are activating an Exadata system, go to "Activate Exadata Assets" to activate the system. For all other systems, use the procedure below: Open a terminal window and log in as root on the ASR Manager system. Important: Activating ASR Assets is not done on the assets themselves but on the ASR Manager system only. Run the following activate command for each ASR Asset. Be sure to use the IP or hostname of the ASR Asset system. asr activate_asset -i <IP address>or asr activate_asset -h <hostname> Login to My Oracle Support to complete the activation process. See "Enable ASR Assets in My Oracle Support" for details. While you are logged into My Oracle Support, you can view your ASR Assets. In the My Oracle Support Dashboard, click the âMore...â tab. Then select âSettingsâ from the menu. In the âSettingsâ pane on the left of the window, select âAssetsâ (located under the Administrative submenu). A complete list of all ASR Assets is displayed: Note: Auto Activation: If the ASR Manager receives fault telemetry from an asset that has not been previously activated, ASR automatically attempts to activate the asset as if the asr activate_asset command is executed. Activate Blade Assets Use the following procedure to activate a Sun Blade system. Keep in mind that Blade systems also include the chassis within which the Blade systems are installed. Therefore, when activating, the Blade and the chassis must be activated. Chassis telemetry reports power and environmental faults, and blade telemetry reports faults specific to the blade's subsystems. You will need the IP addresses of the Blade chassis and the Blade systems in order to complete this procedure. New ASR Activation on Blade Systems and Chassis Open a terminal window and log in as root on the ASR Manager system. Important: Activating ASR Assets is not done on the assets themselves but on the ASR Manager system only. Run the following commands. If activating an X6270 or X6275 system, see the note further below. asr activate_asset -i Chassis_ILOM_IP asr activate_blade -i Blade_IP -c Chassis_ILOM_IP Repeat the asr activate_blade command for each Blade within the chassis that you desire to have under ASR management. Keep in mind that the Blade systems must be qualified for ASR, as specified in the âSystems Qualified for ASRâ link at: http://oracle.com/asr Note: For the X6270, the asr activate_blade command is issued twice. It is issued once where Blade IP is the IP of the blade's service processor, and once where Blade IP is the IP of the blade's host. The host of an X6270 or X6275 can only be activated if it is running Solaris. It is acceptable to only activate the service processor of the blade in this case. The ILOM on the service processor for an X6270 must be configured. Login to My Oracle Support to complete the activation process. See "Enable ASR Assets in My Oracle Support" for details. While you are logged into My Oracle Support, you can view your ASR Assets. In the My Oracle Support Dashboard, click the âMore...â tab. Then select âSettingsâ from the menu. In the âSettingsâ pane on the left of the window, select âAssetsâ (located under the Administrative submenu). A complete list of all ASR Assets is displayed: Previously ASR-Activated Blade Systems If you have previously activated a Blade chassis, follow these instructions to activate Blade systems. Open a terminal window and log in as root on the ASR Manager system. Important: Activating ASR Assets is not done on the assets themselves but on the ASR Manager system only. Run the following command. If activating an X6270 or X6275 system, see the note further below. asr activate_blade -i Blade_IP -c Chassis_ILOM_IP Repeat the asr activate_blade command for each Blade within the chassis. Keep in mind that the Blade systems must be qualified for ASR, as specified in the âSystems Qualified for ASRâ link at: http://oracle.com/asr Note: For the X6270, the asr activate_blade command is issued twice. It is issued once where Blade IP is the IP of the blade's service processor, and once where Blade IP is the IP of the blade's host. The host of an X6270 or X6275 can only be activated if it is running Solaris. It is acceptable to only activate the service processor of the blade in this case. The ILOM on the service processor for an X6270 must be configured. Login to My Oracle Support to complete the activation process. See "Enable ASR Assets in My Oracle Support" for details. While you are logged into My Oracle Support, you can view your ASR Assets. In the My Oracle Support Dashboard, click the âMore...â tab. Then select âSettingsâ from the menu. In the âSettingsâ pane on the left of the window, select âAssetsâ (located under the Administrative submenu). A complete list of all ASR Assets is displayed: Sun Blade X6275 Configuration A Sun Blade X6275 server has two Service Processors that should be configured and activated for ASR using the steps described above. Also, each X6275 has two Hosts than can be configured as a secondary telemetry source. Sun Blade X6275 Service Processors Activate each Sun Blade X6275 Service Processor with the command: activate_blade -i <SP_IP_address> -c <CMM_IP_address> When activated both X6275 Service Processors located on a single X6275 Blade Server will report ASR events independently. Each activated Service Processor on a single blade will be displayed in ASR as having the same serial number and a different IP address. Sun Blade X6275 Hosts To use this telemetry each Host will need to be activated using the command: activate_blade -i <Host_IP_address> -c <CMM_IP_address> When activated both X6275 Hosts located on a single X6275 Blade Server will report ASR events independently. Each activated Host on a single blade will be displayed in ASR as having the same serial number and a different IP address. Activate Exadata Assets To activate Exadata assets, run the following command: activate_exadata [-i exadataip -h exadatahostname -l ilomip] [-?]or activate_exadata [-i exadataip -h exadatahostname -n ilomhostname] [-?] The parameters are: -i exadataip - The IP address of the database server host or storage cell. -h exadatahostname - The hostname of the database server host or storage cell. -l ilomip - The IP address of the ILOM corresponding to the database server host or storage cell. -n ilomhostname - The hostname of the ILOM corresponding to the database server host or storage cell. -? - Help (optional), displays help information. Note: For details on enabling Oracle Auto Service Request on the Oracle Exadata servers, please refer to Oracle Exadata documentation. Activating Exalogic Assets To activate ASR on Exalogic OpenStorage Assets, use the embedded system management GUI. This is the same method for activating ASR on a standalone Sun Unified Storage 7xxx product. To activate ASR on each Exalogic X4170 M2 server, use the asr activate_asset command from the ASR Manager. See Enable ILOM Telemetry to configure SNMP trap destinations on each of the ILOMs. The recommended way to connect the Exadata and components to the ASR server is via the management network. There should be etwork connectivity between the management rail and ASR manager. All ILOMS and OEL on the DB nodes and storage cells are configured to send SNMP traps to the manager. Connecting Exadata components thru' the management rail makes the most sense, especially for the storage cells since they have no public network access. Running ASR on third-party hardware ASR Manager server no longer requires Oracle hardware. It can run on any vendor's x86 server or SPARC, even in a virtual machine. The OS support is still limited to Oracle Linux, Solaris x86 or Solaris SPARC. The ASR assets (those servers monitored by ASR) are of course Oracle/Sun hardware only. Please see http://www.oracle.com/technetwork/server-storage/asr/overview/hardware-recommendations-330108.html Products qualified for ASR http://wikis.sun.com/display/ASRSO/Sun+Products+Qualified+for+ASR ASR Fault Coverage Information http://wikis.sun.com/display/asrfaults/ASR+Fault+Coverage+Information Example ASR Diagnoses: Example of an ASR alarm without ILOM fdd diagnosis. Note FRU PN is not identified. ASR: Memory Correctable ECC Fault ----------------------------------- Hostname: slcc03db05-c Product Type: SUN FIRE X4170 M2 SERVER Summary:ASR: Memory Correctable ECC Fault Description:Refer to - http://www.sun.com/msg/SPX86-8001-H4 Note: This is a component of an Exadata Device. sunHwTrapSystemIdentifier = Exadata Database Machine X2-2 AK00004645 sunHwTrapChassisId = 1105FMM0RJ sunHwTrapProductName = SUN FIRE X4170 M2 SERVER sunHwTrapSuspectComponentName = /SYS/MB/P0/D4 sunHwTrapFaultClass = fault.memory.intel.dimm_ce sunHwTrapFaultCertainty = 100 sunHwTrapFaultMessageID = http://www.sun.com/msg/SPX86-8001-H4 sunHwTrapFaultUUID = fabca3cc-9d24-e160-e848-ecb602d8f675 sunHwTrapAssocObjectId = .1.3.6.1.2.1.47.1.1.1.1.2.22 FEMS rule matched:ILOM FDD - Xxx7x - SPX86-8001-H4 Refer to - http://www.sun.com/msg/SPX86-8001-H4 Extra information:- Alerts received for this system in last 2 months (limit 10): None --------- Example of an ASR alarm with ILOM fdd diagnosis. Note FRU PN is clearly identified. =ASR Alarm= Automatic Service Request (ASR) Alarm Generated: 2011-11-01 13:01:44 Severity : 3 Device : 1114FMM044 Eventcode: SPX86-8001-H4 Event num: SPX86-8001-H4 ASR: Memory Correctable ECC Fault ----------------------------------- Hostname: windcel03c-ilom Product Type: SUN FIRE X4270 M2 SERVER Summary:ASR: Memory Correctable ECC Fault Description:http://www.sun.com/msg/SPX86-8001-H4 TrapEventTime = Tue Nov 1 20:01:44 2011 FaultMessageID = SPX86-8001-H4 TrapFaultUUID = 9e258151-1ec9-469a-eb2a-82e511db8e15 TrapKaUrl = http://www.sun.com/msg/SPX86-8001-H4 TrapSeverity = 0 TrapProductManufacturer = ORACLE CORPORATION ProductName = SUN FIRE X4270 M2 SERVER ProductSn = 1114FMM044 DiagEntity = fdd(1) SystemIdentifier = Exadata Database Machine X2-2 AK00013981 Hostname = windcel03c-ilom SuspectCount = 1 Event Suspect 1 Information SuspectFruFaultCertainty = 100 SuspectFruFaultClass = fault.memory.intel.dimm_ce SuspectFruName = 4GB DDR3 SDRAM 666 SuspectFruLocation = /SYS/MB/P0/D5 SuspectFruChassisId = 1114FMM044 SuspectFruManufacturer = HYNIX SEMICONDUCTOR INC. SuspectFruPn = HMT351R7BFR4A-H9 SuspectFruSn = 12C8BBF5 SuspectFruRevision = SuspectFruStatus = faulted(3) NOTE: Please include the asset's Distribution Email list, [email protected] in all customer communications regarding this SR FEMS rule matched:ILOM_Diagnosed - SPX86-8001-H4 - Xxx7x NOTES Using ILOM as a Telemetry Source:
Attachments This solution has no attachment |
||||||||||||
|
||||||||||||