Sun Microsystems, Inc.  Sun System Handbook - ISO 4.1 October 2012 Internal/Partner Edition
   Home | Current Systems | Former STK Products | EOL Systems | Components | General Info | Search | Feedback

Asset ID: 1-72-1495798.1
Update Date:2012-10-09
Keywords:

Solution Type  Problem Resolution Sure

Solution  1495798.1 :   Sun Storage 7000 Unified Storage System: In the 2011.04.24.1.0 Appliance Release, the logs do not roll automatically  


Related Items
  • Sun Storage 7410 Unified Storage System
  •  
  • Sun Storage 7310 Unified Storage System
  •  
  • Sun ZFS Storage 7120
  •  
  • Sun Storage 7110 Unified Storage System
  •  
  • Sun ZFS Storage 7320
  •  
  • Sun ZFS Storage 7420
  •  
  • Sun Storage 7210 Unified Storage System
  •  
Related Categories
  • PLA-Support>Sun Systems>DISK>NAS>SN-DK: 7xxx NAS
  •  




In this Document
Symptoms
Changes
Cause
Solution
References


Created from <SR 3-6223868873>

Applies to:

Sun Storage 7410 Unified Storage System - Version Not Applicable to Not Applicable [Release N/A]
Sun Storage 7310 Unified Storage System - Version Not Applicable to Not Applicable [Release N/A]
Sun Storage 7210 Unified Storage System - Version Not Applicable to Not Applicable [Release N/A]
Sun Storage 7110 Unified Storage System - Version Not Applicable to Not Applicable [Release N/A]
Sun ZFS Storage 7320 - Version Not Applicable to Not Applicable [Release N/A]
7000 Appliance OS (Fishworks)

Symptoms

There are several symptoms associated with this issue which are all caused by certain log files growing to a huge size - due to a bug, the logs are not rolled automatically in 2011.04.24.1.0.

Symptoms may include:

  • Administrative interface (BUI and CLI) taking a long time to start up.
  • Administrative interface may even stop working, take a long time to respond to commands, or hang.
  • When attempting to view log files, the BUI will hang or take a very long time to show the output of the log file.
  • The system will be running 2011.04.24.1.0, the issue is fixed in all later versions and does not affect earlier versions.

There will be several other symptoms visible if looking at a bundle or on a live system at the OS shell:

  • The akd process could be taking up a lot of memory - close to it's 3GB limit. On a live system check with
# prstat 1 1 | sed -ne 1p -e /akd/p

 or if looking at a bundle look at the size of the akd core in the akd directory of the bundle.

  • The log files, especially akd.ak and rm.ak will be huge as they will not have been rolled since upgrading to 2011.04.24.1.0. These log files will be in /var/ak/logs, or in the logs directory of a bundle.
  • On a live system there will be no /var/logadm directory.

To discuss this information further with Oracle experts and industry peers, we encourage you to review, join or start a discussion in the My Oracle Support Community - 7000 Series ZFS Appliances

Changes

The system has been upgraded to appliance software version 2011.04.24.1.0 (aka 2011.1.1.0)

Cause

A known problem in the 2011.04.24.1.0 software has caused the logs not to roll. This can mean that huge log files build up.

This in turn causes the akd process, that is responsible for managing the Administration Interface, to use up lots of memory, as it has to sanity check the logs when it starts up.

If the log files get too big then the akd process can reach the 32 bit limit of around 3.4GB of memory.

 

See CR 7127845  (2011.1 breaks logadm - logadm: Error: open /var/logadm/timestamps: No such file or directory)  for full details.

 

Solution

The full solution would be to upgrade to a version of appliance software later than 2011.04.24.1.1.

A workaround is available that will provide relief if a maintenance window for an upgrade cannot be arranged in the short term.

Please contact Oracle Support to raise a service request if you need this workaround. The workaround will need to be carried out via a remote support session, such as via Shared Shell or Webex.

 

The workaround is to manually create the /var/logadm directory and then manually roll the log files.

Open shared shell session to customer system.

S7000:> confirm shell
# mkdir /var/logadm

To roll the log files manually please see the procedure documented in the following wiki article:

https://stbeehive.oracle.com/teamcollab/wiki/AmberRoadSupport:Solaris+shell+procedure+to+'clean+up'+(and+rotate)+log+files

If unable to access the ZFS Storage Appliance Support wiki link above, please contact a member of the NAS Support team for further advice.

References

@https://stbeehive.oracle.com/teamcollab/wiki/AmberRoadSupport:Solaris+shell+procedure+to+'clean+up'+(and+rotate)+log+files

Attachments
This solution has no attachment
  Copyright © 2012 Sun Microsystems, Inc.  All rights reserved.
 Feedback