Sun Storage 7000 Unified Storage System: Creation/deletion of large amount of VDI LUNs can cause BUI/CLI hang

Asset ID:	1-72-1408593.1
Update Date:	2012-06-29
Keywords:

Solution Type Problem Resolution Sure

Solution 1408593.1 : Sun Storage 7000 Unified Storage System: Creation/deletion of large amount of VDI LUNs can cause BUI/CLI hang

Applies to:

Sun ZFS Storage 7120 - Version Not Applicable to Not Applicable [Release N/A]
Sun ZFS Storage 7420 - Version Not Applicable to Not Applicable [Release N/A]
Sun Storage 7410 Unified Storage System - Version Not Applicable to Not Applicable [Release N/A]
Sun Storage 7110 Unified Storage System - Version Not Applicable to Not Applicable [Release N/A]
Sun Storage 7210 Unified Storage System - Version Not Applicable to Not Applicable [Release N/A]
7000 Appliance OS (Fishworks)

Symptoms

To discuss this information further with Oracle experts and industry peers, we encourage you to review, join or start a discussion in the My Oracle Support Community - 7000 Series ZFS Appliances

Symptoms observable by the customer:

VDI client systems report 'unresponsive storage'
VDI marks storage as disabled and cloning fails
Cannot login to BUI/CLI

Symptoms observable by the Oracle Support engineer:

prstat shows akd as taking up ~3.5 GB of memory:

PID USERNAME  SIZE   RSS STATE  PRI NICE      TIME  CPU PROCESS/NLWP

  2515 daemon   5972K 5172K sleep   60  -20  26:46:25 0.3% nfsd/78

   951 root        0K    0K sleep   99  -20  10:39:50 0.2% zpool-pool-0/158

  1285 root       41M   21M sleep   59    0   0:01:05 0.1% httpd/1

  2455 root     3540M 3540M sleep  100    -   1:48:29 0.0% akd/175  <<<  4302 root     4240K 2848K cpu4    59    0   0:00:00 0.0% prstat/1

  1360 root       34M   27M sleep   59    0   0:03:23 0.0% fmd/33

  1796 root     9136K 4588K sleep   59    0   0:00:00 0.0% syseventd/18

  4292 webservd   40M 7732K sleep   59    0   0:00:00 0.0% httpd/1

Many stash objects under /var/ak/stash/com/sun/ak/xmlrpc/capacity

Cause

This is a known issue due to VDI 'rolling over' unused shares each hour, and the system doesn't clean up deleted shares.

Known issue - <SunBug 7085319> - AKD should abandon nas_capacity_share objects for deleted shares

Solution

Recommended action for the customer:

For the fix for this issue, the Appliance Firmware Release should be upgraded to the 2011.1.1.0 release or later (2011.1.2.1 or later if running 7x10 system with SAS-1 Disk Trays).

You will need to engage Oracle Support, by opening a Service Request, so that Oracle Support Services can provide confirmation of this issue and then carry out the appropriate activities to resolve the issue.

Recommended actions for the Oracle Support engineer:

Remove the capacity objects from the appliance stash - see the following wiki document:

https://stbeehive.oracle.com/teamcollab/wiki/AmberRoadSupport:Solaris+shell+procedure+to+remove+capacity+objects

If you cannot access this document engage NAS Storage-TSC for assistance

Back to <Document 1401282.1> Sun Storage 7000 Unified Storage System: How to Troubleshoot Unresponsive Administrative Interface (BUI/CLI Hang).

References

@ <BUG:7085319> - AKD SHOULD ABANDON NAS_CAPACITY_SHARE OBJECTS FOR DELETED SHARES
@Support wiki - remove stash capacity objects: https://stbeehive.oracle.com/teamcollab/wiki/AmberRoadSupport:Solaris+shell+procedure+to+remove+capacity+objects
<NOTE:1401282.1> - Sun Storage 7000 Unified Storage System: How to Troubleshoot Unresponsive Administrative Interface (BUI/CLI hang)

Attachments

This solution has no attachment