VTL - Failure to expand or shrink virtual tape

Asset ID:	1-72-1337682.1
Update Date:	2011-07-29
Keywords:

Solution Type Problem Resolution Sure

Solution 1337682.1 : VTL - Failure to expand or shrink virtual tape

Applies to:

Sun StorageTek VTL Plus Storage Appliance - Version: 1.0 - Build 1323 to 2.0 - Build 1590d - Release: 1.0 to 2.0
Information in this document applies to any platform.

Symptoms

Getting errors when expanding a virtual tape, resulting in a failed expansion.
Getting errors when shrinking a virtual tape, resulting in a failed shrinking of a tape.

Here are sample messages which suggest this problem:

1. VTape Expansion failure due to space
Apr 27 17:00:00 TLE_INFO: TAPE CACHING: System needs 6144MB to expand tape VID 1966
Apr 27 17:00:00 TLE_INFO: TAPE CACHING: No more tapes can be reclaimed
Apr 27 17:00:00 Not enough space to expand virtual tape||VIRTUAL TAPE 444

2a. VTape Expand Failure
May 26 02:40:10 xxxxxxvtl05 genunix: [ID 459543 kern.notice] <6>[fffffe800240ec80] TLE_INFO: Expand Virtual Tape 42 failed, Error -2147417599, Retrying without wait

2b. VTape Shrink Failure
May 26 09:20:39 xxxxxxvtl06 ipstorcomm [mgtpipe_exec.c:pipe_thread:921][1111]: [ID 604262 daemon.notice] Rcv'd mgtpipe cmd: 'tapelocation 12116 [VM0270] 1 10003 270 -1'
May 26 09:21:32 xxxxxxvtl06 genunix: [ID 459543 kern.notice] <6>[fffffe80040c7c80] TLE_INFO: Shrink Virtual Tape 13171 failed, Error -2147417599, Retrying without wait
May 26 09:21:32 xxxxxxvtl06 ipstorcomm [mgtpipe_exec.c:pipe_thread:921][1111]: [ID 408748 daemon.notice] Rcv'd mgtpipe cmd: 'tle_shrink_tape 7615 13171 161'
May 26 09:24:32 xxxxxxvtl06 kfsnbase: [ID 188213 kern.notice] IOCORE1 Command (7618): '[pgm: 8, func 2, cmd_size 8]' has timed out due to no response from execution
May 26 09:24:32 xxxxxxvtl06 genunix: [ID 459543 kern.notice] <6>[fffffe8000df4c80] TLE_ERROR: TapeUpCall-CallBackup: Failed FSN EC -2
May 26 09:24:32 xxxxxxvtl06 genunix: [ID 459543 kern.notice] <6>[fffffe80040c7c80] TLE_ERROR: ShrinkTape [VM1325], 13171: Failed EC -2

Here are subsequent messages that might indicate this problem:

May 26 21:37:57 xxxxxxvtl06 kfsnbase: [ID 188213 kern.notice] IOCORE1 vdev_exec_dispatch, logical block address out of range, startblk: 10491455, nblk: 512, svdp->size: 10485760
May 26 21:37:57 xxxxxxvtl06 genunix: [ID 459543 kern.notice] <6>[fffffe8000f5dc80] TLE_ERROR: ASYNC WRITE FAILED to drive 10028, Error 1541
May 26 21:37:57 xxxxxxvtl06 sanidmgr[648]: [ID 702911 daemon.notice] IPSTOR||0||E||0x00009c82||Vtape write failed||10028||1541
May 26 21:39:58 xxxxxxvtl06 ipstorcomm [mgtpipe_exec.c:pipe_thread:921][1114]: [ID 615144 daemon.notice] Rcv/'d mgtpipe cmd: /'tle_vdevop 8862 13178 0/'

Cause

There are several reasons that you would be unable to expand a tape.

1. If you are out of space, you will be unable to expand a tape.

2. VTL 1323 has a known issue where it is unable to expand, shrink or possibly even write to a virtual tape if there are excessive numbers of expands going on with a sufficently large configuration.

During the expand or shrink of a tape, the ipstor.conf file is locked, parsed, updated, rewritten and unlocked. If there are enough operations queued up, future requests will fail. This is a known issue of 1323.

The async writing failure was caused by tape shrinking or expansion timeout. It is because all the tape information are stored in ipstor.conf file. Every time the VTL try to do a tape expansion or shrinking operation, the file needs to be updated to the disk partitions. Due to the size of the ipstor.conf and many tape expansion/shrinking request, it eventually timeout on some of the tape shrinking request.

Solution

The solution to this problem is to upgrade from VTL+ 1.0 Build1323 to VTL+ 2.0 Build1656 code. Please contact Oracle VTL support for upgrade options.

A workaround to this problem is to limit the amount of tape capacity on demand (TCoD) by either eliminating it, or increasing the TCoD increment size of virtual tapes, thus reducing the number of times it has to rewrite the configuration file. Or relabeling the vtape(s) in question.

Also, subsequent retries to expand/shrink may be successful due to backup expiring data and reclaiming (relabeling) vtape, thus shrinking the size of the ipstor.conf file:

May 26 02:40:10 xxxxxxvtl05 genunix: [ID 459543 kern.notice] <6>[fffffe800240ec80] TLE_INFO: Expand Virtual Tape 42 failed, Error -2147417599, Retrying without wait
...
May 26 02:49:32 xxxxxxvtl05 genunix: [ID 459543 kern.notice] <6>[fffffe800240ec80] TLE_INFO: Tape 42 Expanded successfully, new size 20401094656 bytes

Attachments

This solution has no attachment