Document Audience:INTERNAL
Document ID:A0193-1
Title:Sun Fire 15K systems with hsPCI boards having Schizo 2.2 ASICs can suffer a panic due to a timing race condition.
Copyright Notice:Copyright © 2007 Sun Microsystems, Inc. All Rights Reserved
Update Date:Fri Jan 03 00:00:00 MST 2003

----------------------------------------------------------------------------
            - Sun Proprietary/Confidential: Internal Use Only -
----------------------------------------------------------------------------

                             FIELD CHANGE ORDER
            (For Authorized Distribution by Enterprise Services)
            
FCO #: A0193-1
Status: inactive
Synopsis: Sun Fire 15K systems with hsPCI boards having Schizo 2.2 ASICs can suffer a panic due to a timing race condition.
Date: Jan/03/2003
SunAlert: Y
Top FIN/FCO Report: Yes
Products Reference: F15K hsPCI boards with Schizo 2.2 ASICs
Product Category: Server / System Component
Product Affected: 
Mkt_ID     Platform     Model    Description          Serial Number
------	   --------     -----    -----------	      -------------
-          F15K           -      Sun Fire 15K              _

X-Options Affected
--------- -------
Mkt_ID     Platform      Model    Description          Serial Number
------	   --------      -----    -----------	      -------------
X4575A     F15K            -      hsPCI Assembly            -




Mkt_ID     Platform     Model    Description          Serial Number
------	   --------     -----    -----------	      -------------
-          F15K           -      Sun Fire 15K              _

X-Options Affected
--------- -------
Mkt_ID     Platform      Model    Description          Serial Number
------	   --------      -----    -----------	      -------------
X4575A     F15K            -      hsPCI Assembly            -

AFFECTED PARTS:

Part Number             Description              Model
-----------             -----------              -----
501-5397-08(Or Less)    hsPCI I/O Board            -


REFERENCES :
  ESC: 535301      
  ESC: 535705      
  ESC: 536269      
  ESC: 536227      
  ESC: 535927      
  ESC: 536469      
  ECO: WO_23260 
  ECO: WO_23112      
  ECO: WO_23410    
  PatchID: 112665-01   Nexus Driver
  Manual:806-3511-10   Sun Fire 15K HW Installation and De-Installation Guide
  LEAP: 1963       
  FIN: I0820-1    
  SunAlert: 44582      

PROBLEM DESCRIPTION:

Change History 
--------------
A0193-1

 Date Modified: Jan/03/03
 Updates: PLANNED IMPLEMENTION COMPLETION DATE
Parts Affected: 
Part Number             Description              Model
-----------             -----------              -----
501-5397-08(Or Less)    hsPCI I/O Board            -
References: 
ESC: 535301      
  ESC: 535705      
  ESC: 536269      
  ESC: 536227      
  ESC: 535927      
  ESC: 536469      
  ECO: WO_23260 
  ECO: WO_23112      
  ECO: WO_23410    
  PatchID: 112665-01   Nexus Driver
  Manual:806-3511-10   Sun Fire 15K HW Installation and De-Installation Guide
  LEAP: 1963       
  FIN: I0820-1    
  SunAlert: 44582
Issue Description: 
Under certain conditions Sun Fire 15K servers, with hsPCI I/O boards that
contain Schizo 2.2 ASICs, may experience a domain panic.  Only Sun Fire
15K servers shipped prior to April 1, 2002 are affected.  Sun Fire 12K
servers are not impacted as none where shipped with affected hsPCI I/O boards.

No silent data corruption occurs has a result of this issue.

Domain panics may occur with the following Sun Fire 15K configurations:

  Hardware Components:

     hsPCI I/O Board revision 2.2 (501-5397-08 or lower)
     JNI 32-bit PCI-to-Fibre Channel HBA  (FCI-1063-x)
     SunSwift PCI SCSI (Fresh Choice) adapters (X1032A)

  Nexus Driver:

     pcisch (PCI Bus nexus driver 1.199)

     NOTE: Version 1.199 of the Nexus Driver is the default version shipped
           with Solaris 8.  This is the version installed unless Patch
           112665-01 has been applied.

  Domain Configuration:

     20+ CPUs utilizing ISP (SCSI HBA Driver)
     or JNI FCI-1063-x HBA are more susceptible.

This failure has only been observed with ISP and/or JNI drivers.
However, not all configurations with the ISP and/or the JNI driver are affected.
The failure is configuration specific.  When the failure manifests itself as a
panic, these drivers are in the panic string, which helps to identify the
failure.

When the panic has been seen with the ISP driver, the panic string is:

   "isp_scsi_impl_pktfree: freeing free packet"

For the JNI driver used with the JNI FCI-1063-x HBA, the system will log
"INB_SCSI_COMPLETE interrupt with INVALID tag" errors before the domain
panics with a "panic assertion failed:" These error messages and panic
strings correlate with domain configurations utilizing the JNI FCI-1063-x
Host-Bus-Adapter.

Domain configurations utilizing an ISP (SCSI Host Bus Adapter Driver)
configuration generate the "isp_scsi_impl_pktfree: freeing free packet"
panic string.

In the JNI configuration the following errors were observed;

  "INB_SCSI_COMPLETE interrupt with INVALID tag"

before the domain crashed with a "panic assertion failed:"

With the hsPCI configuration the domain will crash with an
"isp_scsi_impl_pktfree: freeing free packet" panic string.

Root cause of this problem is due to a transaction ordering issue within the
I/O controller.  The I/O controller does not follow certain ordering rules
and may have data remaining from a previous read/write while the current
transaction is being processed.

Corrective action was made available in manufacturing via ECO# WO_23260
by dash rolling FRU part number 501-5397 to -09 on March 07, 2002.
Corrective Action was made available in Enterprise Services via
LEAP# 1963 on April 17, 2002.

Sun Legal approved Customer Letter can be located at;

  http://sdpsweb.EBay/FIN_FCO/FCO/FCO_A0192-1_Dir/CustomerLetter.ps

CUSTOMER LIST: Reference the following URL for a list of affected customer
               shipments;

http://sdpsweb.ebay/FIN_FCO/FCO/FCO_A0193-1_Dir/F15Kcust.sdc

Note: To view document click on the above URL, then save to your local
      disk using your Netscape 'file' button and select 'save as', then
      open file locally using StarOffice.
Parts Affected: 
changed from October 31, 2002
                                         to February 28, 2003
--------------
 
Under certain conditions Sun Fire 15K servers, with hsPCI I/O boards that
contain Schizo 2.2 ASICs, may experience a domain panic.  Only Sun Fire
15K servers shipped prior to April 1, 2002 are affected.  Sun Fire 12K
servers are not impacted as none where shipped with affected hsPCI I/O boards.

No silent data corruption occurs has a result of this issue.

Domain panics may occur with the following Sun Fire 15K configurations:

  Hardware Components:

     hsPCI I/O Board revision 2.2 (501-5397-08 or lower)
     JNI 32-bit PCI-to-Fibre Channel HBA  (FCI-1063-x)
     SunSwift PCI SCSI (Fresh Choice) adapters (X1032A)

  Nexus Driver:

     pcisch (PCI Bus nexus driver 1.199)

     NOTE: Version 1.199 of the Nexus Driver is the default version shipped
           with Solaris 8.  This is the version installed unless Patch
           112665-01 has been applied.

  Domain Configuration:

     20+ CPUs utilizing ISP (SCSI HBA Driver)
     or JNI FCI-1063-x HBA are more susceptible.

This failure has only been observed with ISP and/or JNI drivers.
However, not all configurations with the ISP and/or the JNI driver are affected.
The failure is configuration specific.  When the failure manifests itself as a
panic, these drivers are in the panic string, which helps to identify the
failure.

When the panic has been seen with the ISP driver, the panic string is:

   "isp_scsi_impl_pktfree: freeing free packet"

For the JNI driver used with the JNI FCI-1063-x HBA, the system will log
"INB_SCSI_COMPLETE interrupt with INVALID tag" errors before the domain
panics with a "panic assertion failed:" These error messages and panic
strings correlate with domain configurations utilizing the JNI FCI-1063-x
Host-Bus-Adapter.

Domain configurations utilizing an ISP (SCSI Host Bus Adapter Driver)
configuration generate the "isp_scsi_impl_pktfree: freeing free packet"
panic string.

In the JNI configuration the following errors were observed;

  "INB_SCSI_COMPLETE interrupt with INVALID tag"

before the domain crashed with a "panic assertion failed:"

With the hsPCI configuration the domain will crash with an
"isp_scsi_impl_pktfree: freeing free packet" panic string.

Root cause of this problem is due to a transaction ordering issue within the
I/O controller.  The I/O controller does not follow certain ordering rules
and may have data remaining from a previous read/write while the current
transaction is being processed.

Corrective action was made available in manufacturing via ECO# WO_23260
by dash rolling FRU part number 501-5397 to -09 on March 07, 2002.
Corrective Action was made available in Enterprise Services via
LEAP# 1963 on April 17, 2002.

Sun Legal approved Customer Letter can be located at;

  http://sdpsweb.EBay/FIN_FCO/FCO/FCO_A0192-1_Dir/CustomerLetter.ps

CUSTOMER LIST: Reference the following URL for a list of affected customer
               shipments;

http://sdpsweb.ebay/FIN_FCO/FCO/FCO_A0193-1_Dir/F15Kcust.sdc

Note: To view document click on the above URL, then save to your local
      disk using your Netscape 'file' button and select 'save as', then
      open file locally using StarOffice.
      

 February 28, 2003

IMPLEMENTATION:

 ---
| X |   MANDATORY (Fully Pro-Active)
 ---

 ---
|   |   CONTROLLED PRO-ACTIVE (per Sun Geo Plan)
 ---

 ---
|   |   UPON FAILURE
 ---

REPLACEMENT TIME ESTIMATE: 2 hours

SPECIAL CONSIDERATION:

The below link to a Sun Alert has a "Restricted" Distribution.  Please print
and use this Sun Alert in addition to, or in place of, the customer letter.
Communicate to your *affected* customers only.  Typically, Sun Alerts have a
wider distribution on Contract and Free SunSolve, but the Sun Alert program
has been enhanced to include what is known as "Targeted Sun Alerts" for
affected customers only.

  http://sdpsweb.EBay/FIN_FCO/FCO/FCO_A0193-1_Dir/44582_public.html

Note: If you have questions please contact the Sun Alert Program Office.
      Ref;  http://sunalert.ebay/progorg.html

CORRECTIVE ACTION :

All swap activities should be directed to the following Enterprise Services
timezone representatives to ensure proper prioritization with the Global
Prioritization Committee (GPC);

  EMEA: Richard Porter
  AMER: Gary Replogle
  APAC: Kam-Weng Goh

The following is a list of GPC Global Sales Organization timezone 
representatives;

  AMER (US):   Jeff Barteld
  AMER (INTL): Kerry Roller
  EMEA:        Jon Ireland
  APAC:        Peter Chadford


Proactivly Replace Schizo 2.2 ASIC based hsPCI assemblies, 501-5397-08
(or below) with Schizo 2.3 ASIC based hsPCI assemblies, 501-5397-09
(or above).

Additionally, upgrade of Nexus Driver with point patch 112665-01 is required.

Until this point patch becomes available on SunSolve it can temporary be
located at the below URL;

   http://sdpsweb.ebay/FIN_FCO/FCO/FCO_A0193-1_Dir/112665-01.tar

Tag all returned boards with "FCO A0193-1" and return via Overnight Freight.

COMMENTS:

BILLING TYPE:

 Warranty: Sun will provide parts at no charge under Warranty
           Service. On-Site Labor Rates are based on how the
           system was initially installed.

 Contract: Sun will provide parts at no charge. On-Site Labor Rates
           are based on the type of service contract.

 Non Contract: Sun will provide parts at no charge. Installation by
               Sun is available based on the On-Site Labor Rates
               defined in the Price List.

--------------------------------------------------------------------------
Implementation Footnote:
________________________

i)   In case of Mandatory FCOs, Sun Services will attempt to contact
      all known customers to recommend the part upgrade.

ii)  For controlled proactive swap FCOs, Sun Services mission critical
     support teams will initiate proactive swap efforts for their respective
     accounts, as required.

iii) For Replace upon Failure FCOs, Sun Services partners will implement

     the necessary corrective actions as and when they are required.

--------------------------------------------------------------------------

All released FINs and FCOs can be accessed using your favorite network
browser as follows:

SunWeb Access:
______________

* Access the top level URL of http://sdpsweb.Central/FIN_FCO/

* From there, select the appropriate link to query or browse the FIN and
  FCO Homepage collections.

SunSolve Online Access:
_______________________

* Access the SunSolve Online URL at http://sunsolve.Central/

* From there, select the appropriate link to browse the FIN or FCO index.

Internet Access:
_______________

* Access the top level URL of  https://spe.sun.com

--------------------------------------------------------------------------
General:
________

Send questions or comments to [email protected]

---------------------------------------------------------------------------
Implementation: 
---
| X |   MANDATORY (Fully Pro-Active)
 ---

 ---
|   |   CONTROLLED PRO-ACTIVE (per Sun Geo Plan)
 ---

 ---
|   |   UPON FAILURE
 ---
Replacement Time Estimate: 
2 hours
Special Considerations: 
The below link to a Sun Alert has a "Restricted" Distribution.  Please print
and use this Sun Alert in addition to, or in place of, the customer letter.
Communicate to your *affected* customers only.  Typically, Sun Alerts have a
wider distribution on Contract and Free SunSolve, but the Sun Alert program
has been enhanced to include what is known as "Targeted Sun Alerts" for
affected customers only.

  http://sdpsweb.EBay/FIN_FCO/FCO/FCO_A0193-1_Dir/44582_public.html

Note: If you have questions please contact the Sun Alert Program Office.
      Ref;  http://sunalert.ebay/progorg.html
Corrective Action: 
All swap activities should be directed to the following Enterprise Services
timezone representatives to ensure proper prioritization with the Global
Prioritization Committee (GPC);

  EMEA: Richard Porter
  AMER: Gary Replogle
  APAC: Kam-Weng Goh

The following is a list of GPC Global Sales Organization timezone 
representatives;

  AMER (US):   Jeff Barteld
  AMER (INTL): Kerry Roller
  EMEA:        Jon Ireland
  APAC:        Peter Chadford


Proactivly Replace Schizo 2.2 ASIC based hsPCI assemblies, 501-5397-08 (or 
below)
with Schizo 2.3 ASIC based hsPCI assemblies, 501-5397-09 (or above).

Additionally, upgrade of Nexus Driver with point patch 112665-01 is required.

Until this point patch becomes available on SunSolve it can temporary be
located at the below URL;

   http://sdpsweb.ebay/FIN_FCO/FCO/FCO_A0193-1_Dir/112665-01.tar

Tag all returned boards with "FCO A0193-1" and return via Overnight Freight.
Comments: 
Billing Type: 
Warranty: Sun will provide parts at no charge under Warranty
           Service. On-Site Labor Rates are based on how the
           system was initially installed.

 Contract: Sun will provide parts at no charge. On-Site Labor Rates
           are based on the type of service contract.

 Non Contract: Sun will provide parts at no charge. Installation by
               Sun is available based on the On-Site Labor Rates
               defined in the Price List.

--------------------------------------------------------------------------
Implementation Footnote: 
________________________

i)   In case of Mandatory FCOs, Enterprise Services will attempt to contact
      all known customers to recommend the part upgrade.

ii)  For controlled proactive swap FCOs, Enterprise Services mission critical
     support teams will initiate proactive swap efforts for their respective
     accounts, as required.

iii) For Replace upon Failure FCOs, Enterprise Services partners will implement

     the necessary corrective actions as and when they are required.

--------------------------------------------------------------------------

All released FINs and FCOs can be accessed using your favorite network
browser as follows:

SunWeb Access:
______________

* Access the top level URL of http://sdpsweb.EBay/FIN_FCO/

* From there, select the appropriate link to query or browse the FIN and
  FCO Homepage collections.

SunSolve Online Access:
_______________________

* Access the SunSolve Online URL at http://sunsolve.Central/

* From there, select the appropriate link to browse the FIN or FCO index.

Internet Access:
_______________

* Access the top level URL of https://infoserver.Sun.COM

--------------------------------------------------------------------------
General:
________

Send questions or comments to [email protected]

---------------------------------------------------------------------------

From - Fri Jan  3 09:13:30 2003
Return-path: 
Received: from centralmail1brm.Central.Sun.COM ([129.147.58.122])
 by edgemail1.Central.Sun.COM
 (iPlanet Messaging Server 5.2 HotFix 1.08 (built Dec  6 2002))
 with ESMTP id <[email protected]> for
 hd4723@ims-ms-daemon; Fri, 03 Jan 2003 09:13:06 -0700 (MST)
Received: from sunmail3.sfbay.sun.com
 (sunmail3.SFBay.Sun.COM [129.149.247.180])	by centralmail1brm.Central.Sun.COM
 (8.12.2+Sun/8.12.2/ENSMAIL,v2.2) with ESMTP id h03GD4Oq020802; Fri,
 03 Jan 2003 09:13:04 -0700 (MST)
Received: from bast.Central.Sun.COM (bast.Central.Sun.COM [129.147.4.36])
	by sunmail3.sfbay.sun.com (8.11.6+Sun/8.11.6/ENSMAIL,v2.2)
 with ESMTP id h03GCvB05180	for ; Fri,
 03 Jan 2003 08:12:58 -0800 (PST)
Received: from peacemaker (peacemaker [129.147.20.82])
	by bast.Central.Sun.COM (8.10.2+Sun/8.10.2/ENSMAIL,v2.2)
 with SMTP id h03GCqW15670; Fri, 03 Jan 2003 09:12:52 -0700 (MST)
Date: Fri, 03 Jan 2003 09:12:52 -0700 (MST)
From: Joe Davis 
Subject: FCO:RELEASE:Approved FCO A0193-1 (Sun Fire 15K systems with hsPCI
 boards having Schizo 2.2 ASICs can suffer a panic due to a timing race
 condition)
To: [email protected]
Cc: [email protected], [email protected]
Reply-to: Joe Davis 
Message-id: <[email protected]>
MIME-version: 1.0
X-Mailer: dtmail 1.3.0 @(#)CDE Version 1.4.6_06 SunOS 5.8 sun4u sparc
Content-type: TEXT/plain; charset=us-ascii
Content-transfer-encoding: 7BIT
Content-MD5: XzlBn/D3JLL1loOj7mM+yQ==
Original-recipient: rfc822;[email protected]
Status: RO
X-Status: $$$$
X-UID: 0000000002

----------------------------------------------------------------------------
            - Sun Proprietary/Confidential: Internal Use Only -
----------------------------------------------------------------------------

                             FIELD CHANGE ORDER
            (For Authorized Distribution by Enterprise Services)
Statusinactive