Sun Storage 7000 Unified Storage System: Low throughput during ndmp tape cloning

Asset ID:	1-72-1363330.1
Update Date:	2012-06-19
Keywords:

Solution Type Problem Resolution Sure

Solution 1363330.1 : Sun Storage 7000 Unified Storage System: Low throughput during ndmp tape cloning

Applies to:

Sun Storage 7110 Unified Storage System - Version Not Applicable to Not Applicable [Release N/A]
Sun ZFS Storage 7320 - Version Not Applicable to Not Applicable [Release N/A]
Sun ZFS Storage 7420 - Version Not Applicable to Not Applicable [Release N/A]
Sun ZFS Storage 7120 - Version Not Applicable to Not Applicable [Release N/A]
Sun Storage 7210 Unified Storage System - Version Not Applicable to Not Applicable [Release N/A]
7000 Appliance OS (Fishworks)
Symptoms

Slow ndmp tape cloning in ZFS Unified Storage appliance.

NDMP tape cloning to a locally attached tape drives can only achieve around 6MB/sec
Normal NDMP backup can achieved much higher around 75MB/sec range.

To discuss this information further with Oracle experts and industry peers, we encourage you to review, join or start a discussion in the My Oracle Support Community - 7000 Series ZFS Appliances

Cause

NDMP Tape Cloning Performance problems exist in pre 2011.x appliance NDMP code. Performance improvements to use larger mover record size, removing timeout delay and adding multi-thread capability are being planned.

The NDMP Tape Cloning performance can be checked in a similar manner to the following.
In the following example, the Data Management Application is running Networker 7.6.1 with Ultrium-5 tape devices connected directly to ZFS Unified Storage appliance.
DMA can be any certified platform for ZFS Unified Storage appliance.

Networker nsrndmp_clone run in debug mode:

# nsrndmp_clone -vvvv -b'Monthly NDMP Clone' -S 2984830673

...

77398:nsrndmp_clone: NDMP Cloning started

...

67140:nsrndmp_clone: Open the source tape device : /dev/rmt/0n

69700:nsrndmp_clone: record size is set to 60 KB

67141:nsrndmp_clone: Open the destination tape device : /dev/rmt/1n

69700:nsrndmp_clone: record size is set to 60 KB

/rmt/0n (NDMP)(J) LTO Ultrium-5 E05144L5 2:reading, data

/rmt/1n (NDMP)(J) LTO Ultrium-5 E05181L5 1:writing at 5820 KB/s, 408 MB

                                                                    ** less than 6MB/sec **

On the Sun Unified Storage BUI, use analytics to confirm - Data Movement: NDMP bytes transferred to/from tape.
Alternatively, ndmpstat shows the number of read/write bytes transferred of about 5670 KBytes

aksh> confirm shell ndmpstat 2 5



wthr ops file disk tape bytes perf prcnt

r w bk rs rd wr rd wr rd wr rd wr bk rs dsk tpe idl

0 0 0  0  0  0  0  0  0  0  0  0  0  0  0   0   100

2 0 0 0 0 0 0 0 11340 11340 5670 5670 2 2 0 100 0

2 0 0 0 0 0 0 0 11340 11340 5670 5670 2 2 0 100 0

2 0 0 0 0 0 0 0 11340 11340 5670 5670 2 2 0 100 0

2 0 0 0 0 0 0 0 11400 11400 5700 5700 2 2 0 100 0

Solution

At the time of writing the current firmware version is 2010.Q3.4.1. Some improvements have been made in 2011.1.1.0 code already, further improvements are planned. If further assistance is required please raise a Service Request with Oracle Support.

Request an Interim Diagnostic Relief (IDR) to include the fix for :
CR 7048384 small mover record size
CR 7032986 NDMP three-way restore takes too long (include fix for 7033953)

Back to <Document 1387930.1> Sun Storage 7000 Unified Storage System: How to Troubleshoot NDMP Issues.

References

<BUG:7048384> - NDMPD MOVER RECORD SIZE OF 512K IS LIMITING THE USE OF LARGER TAPE BLOCK.
<BUG:7087650> - 2010Q3.1 7410 TAPE CLONING UNDER NDMP IS SLOW UNDER 6MB/SEC
@ <BUG:7048384> - AGILE00193486: WHILE CREATING MFR PARTS IN IM, ERROR MSG SAYING THAT NEWLY
<BUG:7032986> - NDMP THREE-WAY RESTORE TAKES TOO LONG
<NOTE:1387930.1> - Sun Storage 7000 Unified Storage System: How to Troubleshoot NDMP Issues

Attachments

This solution has no attachment