Sun SPARC Enterprise Mx000 Servers: After UPS failure reports domain shuts down and does not power on

Asset ID:	1-72-1353452.1
Update Date:	2011-09-01
Keywords:

Solution Type Problem Resolution Sure

Solution 1353452.1 : Sun SPARC Enterprise Mx000 Servers: After UPS failure reports domain shuts down and does not power on

Applies to:

Sun SPARC Enterprise M5000 Server - Version: Not Applicable and later   [Release: N/A and later ]
Sun SPARC Enterprise M4000 Server - Version: Not Applicable and later    [Release: N/A and later]
Sun SPARC Enterprise M3000 Server - Version: Not Applicable and later    [Release: N/A and later]
Information in this document applies to any platform.

Symptoms

XSCF> poweron -a
DomainIDs to power on:00
Continue? [y|n] :y
Poweron canceled because of power failure.

Changes

$ grep "^Aug 2[56]" @tmp@cli@bin@showlogs_event_-v.out
Aug 25 05:52:44 UTC 2011 UPS power recovery (/UPS#0)
Aug 25 05:52:47 UTC 2011 UPS power failure (/UPS#0)
Aug 25 05:52:57 UTC 2011 Initiate shutdown (power failure)
Aug 25 05:54:48 UTC 2011 DomainID 0 state change (shutdown started, detail#2)
Aug 25 05:54:49 UTC 2011 Domain issued power-off request to RCI target (DomainID 0)
Aug 25 05:54:52 UTC 2011 All domains shutdown started
Aug 25 05:55:01 UTC 2011 DomainID 0 state change (Powered off, detail#2)
Aug 25 05:55:22 UTC 2011 UPS power recovery (/UPS#0)
Aug 25 05:55:24 UTC 2011 UPS power failure (/UPS#0)
Aug 25 05:55:26 UTC 2011 UPS power recovery (/UPS#0)
Aug 25 05:55:28 UTC 2011 UPS power failure (/UPS#0)
Aug 25 05:55:33 UTC 2011 UPS power recovery (/UPS#0)
Aug 25 05:55:34 UTC 2011 UPS power failure (/UPS#0)
Aug 25 05:55:53 UTC 2011 System powered off
Aug 25 11:52:03 UTC 2011 Power recovery (/PSU#0)
Aug 25 11:52:33 UTC 2011 Power switch is pressed (long)
Aug 25 11:54:56 UTC 2011 Power failure (/PSU#0)
Aug 25 11:59:17 UTC 2011 Panel mode switch is switched to locked position
Aug 25 11:59:37 UTC 2011 XSCF ready
Aug 25 11:59:57 UTC 2011 XSCFU was stopped by power failure. Power failure date is 2011/08/25 11:55:00
Aug 25 12:02:00 UTC 2011 UPS power recovery (/UPS#0)
Aug 25 12:02:01 UTC 2011 System powered on
Aug 25 12:02:11 UTC 2011 UPS power failure (/UPS#0)
Aug 25 12:02:17 UTC 2011 UPS power recovery (/UPS#0)
Aug 25 12:02:17 UTC 2011 UPS power failure (/UPS#0)
Aug 25 12:03:26 UTC 2011 UPS power recovery (/UPS#0)
Aug 25 12:03:27 UTC 2011 UPS power failure (/UPS#0)
Aug 25 12:03:27 UTC 2011 UPS power recovery (/UPS#0)
Aug 25 12:03:31 UTC 2011 UPS power failure (/UPS#0)
Aug 25 12:03:32 UTC 2011 UPS power recovery (/UPS#0)
Aug 25 12:03:32 UTC 2011 System powered off
Aug 25 12:03:43 UTC 2011 UPS power failure (/UPS#0)
Aug 25 12:04:34 UTC 2011 UPS power recovery (/UPS#0)
Aug 25 12:04:36 UTC 2011 System powered on
Aug 25 12:05:13 UTC 2011 DomainID 0 state change (initialize phase started, detail#0)
Aug 25 12:05:26 UTC 2011 DomainID 0: Reset released
Aug 25 12:05:37 UTC 2011 Current domains' phase (DomainID 0 domain phase: CPU Check)
...
Aug 25 12:07:03 UTC 2011 DomainID 0 state change (boot process started, detail#9)
Aug 25 12:07:27 UTC 2011 UPS power failure (/UPS#0)
Aug 25 12:07:28 UTC 2011 UPS power recovery (/UPS#0)
Aug 25 12:07:28 UTC 2011 UPS power failure (/UPS#0)
Aug 25 12:07:32 UTC 2011 UPS power recovery (/UPS#0)
Aug 25 12:07:35 UTC 2011 UPS power failure (/UPS#0)
Aug 25 12:07:46 UTC 2011 Initiate shutdown (power failure)
Aug 25 12:08:06 UTC 2011 DomainID 0 state change (system running, detail#10)
Aug 25 12:10:37 UTC 2011 UPS power recovery (/UPS#0)
Aug 25 12:10:40 UTC 2011 UPS power failure (/UPS#0)
Aug 25 12:10:41 UTC 2011 UPS power recovery (/UPS#0)
Aug 25 12:10:46 UTC 2011 UPS power failure (/UPS#0)
Aug 25 12:10:56 UTC 2011 Initiate shutdown (power failure)
Aug 25 12:11:09 UTC 2011 UPS power recovery (/UPS#0)
Aug 25 12:11:10 UTC 2011 UPS power failure (/UPS#0)
Aug 25 12:11:10 UTC 2011 UPS power recovery (/UPS#0)
Aug 25 12:11:12 UTC 2011 UPS power failure (/UPS#0)
Aug 25 12:11:22 UTC 2011 Initiate shutdown (power failure)
Aug 25 12:11:32 UTC 2011 UPS power recovery (/UPS#0)
Aug 25 12:11:32 UTC 2011 UPS power failure (/UPS#0)
Aug 25 12:11:43 UTC 2011 Initiate shutdown (power failure)
Aug 25 13:01:02 UTC 2011 Power switch is pressed (long)
Aug 25 13:01:24 UTC 2011 Power switch is pressed (short)
Aug 25 13:06:25 UTC 2011 XSCF ready
Aug 25 13:06:46 UTC 2011 XSCFU was stopped by power failure. Power failure date is 2011/08/25 13:02:05
Aug 26 09:30:32 UTC 2011 Power switch is pressed (long)
Aug 26 09:30:39 UTC 2011 Power switch is pressed (long)
Aug 26 09:49:39 UTC 2011 maintenance event (FRU is chosen to be replaced: /PSU#0)
Aug 26 09:51:27 UTC 2011 Unit configuration change (remove) /PSU#0
Aug 26 09:51:55 UTC 2011 Unit configuration change (add) /PSU#0
Aug 26 09:53:26 UTC 2011 maintenance event (FRU test normally completed: /PSU#0)
Aug 26 09:53:42 UTC 2011 maintenance event (FRU replacement procedure normally completed: /PSU#0)
Aug 26 09:54:51 UTC 2011 maintenance event (FRU is chosen to be replaced: /PSU#1)
Aug 26 09:56:03 UTC 2011 Unit configuration change (remove) /PSU#1
Aug 26 09:56:22 UTC 2011 Unit configuration change (add) /PSU#1
Aug 26 09:58:36 UTC 2011 maintenance event (FRU test normally completed: /PSU#1)
Aug 26 09:58:53 UTC 2011 maintenance event (FRU replacement procedure normally completed: /PSU#1)

Cause

A non working UPS will bring down the domains and stop them from booting afterwards.
The UPS is connected through a UPC port of the XSCFU.
If no UPS is used, please check if someone accidentally connected other equipment to that port and if so, remove it ASAP.
Otherwise we may have encountered a fault on the XSCFU's UPC port.

The source code browser suggests that there is only one situation when that error message is reported.

-> cli_cmn_msg.h
357 #define CLI_MSG_POWERON_RETURN_FAILURE \
358 C_("Poweron canceled because of power failure.")
->poweron.c
404 } else if (options.domain[i] == TRUE &&
405 response.result[i]
406 == CLI_ERR_POWER_FAILURE) {
407 CLI_TRACE_ERR(
408 CLI_TRC_DOMAIN_POWERON(0x0063),
409 CLI_TRC_DATA(response.result[i]));
410 lcmn_close_socket(process_key);
411 cli_exit(&ah, CLI_EXIT_ERR_API,
412 CLI_ERR_POWER_FAILURE,
413 CLI_MSG_POWERON_RETURN_FAILURE);
-> poweron.c
58 #define CLI_ERR_POWER_FAILURE 133
...
83 * - 133 : Failure of POWERON because of power failure of UPS.

Solution

Fix the UPS.
If there is no UPS, remove any cables from the UPC ports.
If there are (and were) no cables on any UPC port schedule a repair action to replace the XSCFU.

Attachments

This solution has no attachment