Sun Storage 7000 Unified Storage System: How to add L2ARC cache SSDs (Readzillas) to a pool

Asset ID:	1-71-1333120.1
Update Date:	2012-02-18
Keywords:

Solution Type Technical Instruction Sure

Solution 1333120.1 : Sun Storage 7000 Unified Storage System: How to add L2ARC cache SSDs (Readzillas) to a pool

Applies to:

Sun ZFS Storage 7420 - Version: Not Applicable to Not Applicable - Release: N/A to N/A
Sun Storage 7720 Unified Storage System - Version: Not Applicable to Not Applicable   [Release: N/A to N/A]
Sun ZFS Storage 7320 - Version: Not Applicable to Not Applicable   [Release: N/A to N/A]
Sun Storage 7410 Unified Storage System - Version: Not Applicable to Not Applicable   [Release: N/A to N/A]
Sun Storage 7310 Unified Storage System - Version: Not Applicable to Not Applicable   [Release: N/A to N/A]
Information in this document applies to any platform.

Goal

Purpose

An introduction to the L2ARC read cache Solid State Drive devices known as Readzillas in the ZFS Storage Appliance 7000 series, and procedures for adding them to pools.

Scope

Intended for all storage administrators who wish to configure their ZFS Storage Appliance 7000 series systems with Readzillas to improve read performance for certain I/O workloads.

To discuss this information further with Oracle experts and industry peers, we encourage you to review, join or start a discussion in the My Oracle Support Community - 7000 Series ZFS Appliances

Solution

What is a Readzilla

Readzillas are low latency Solid State Drives which can be used to provide an additional layer of read caching between the ARC in main memory and the disks for a particular zpool. This intermediate layer of read cache is called the L2ARC. Using Readzillas will provide the greatest performance benefit for read-heavy workloads where the working set is fairly static and is too large to be cached into the main ARC. Because they are low latency devices reading data cached in the L2ARC will be much faster than reading data directly from disk. Readzillas can only be used in the 7310, 7410, 7320, 7420 and 7720 ZFS Storage Appliance systems.

Configuration in a zpool

Readzillas are associated with a particular zpool as cache devices, either when the pool is first created or they can also be added easily at a later date. The readzilla devices are inserted into the drive slots in the NAS head, so in the case of a cluster, they can only be associated with a zpool controlled by that particular head. Once they have been associated with a zpool then that zpool can only use the readzillas when it is being mastered by the head that contains those readzillas. If a takeover occurs and the zpool switches heads then it cannot use the readzillas in the original head. It is possible to add readzillas on the second cluster head to the same zpool so that a zpool will have access to read cache whatever head it is on. However those readzillas will then need to "warm up" again after a takeover by having the most recent and most frequently accessed data read into them which may take several hours. This configuration will also mean that one head will always have readzillas that are not being used which may be seen as an uneconomical solution. Since it nay take several hours for read cache to reach optimal levels it may be better to simply fix whatever caused the takeover in the first place and failback the pool to use the readzillas on the original head again. Readzillas should never be removed from one head and transferred to the partner in an attempt to provide cache devices for a pool which has been taken over. This will not work.

When readzillas are added to a pool they are simply added as a pool of cache - they cannot be added as a mirror or raidz configuration. If there is an I/O failure to a readzilla then that read is tried again directly from disk. If one readzilla is lost the data in any other readzilla that was part of the zpool cache is still available. The cached data in the failed readzilla is lost but this is not real data loss since any failed I/O to the readzilla is simply retried from disk.

Adding Readzillas

To add a readzilla:

Physically insert the readzilla in an appropriate slot. Once the drives have been physically installed you should wait for a couple of minutes for the software to pick up and recognise the new devices.

Then add the new readzilla to pool-0 using either the CLI or BUI methods detailed below:

CLI

nas-1a:> configuration storage

Ensure that you are in the correct pool, in the example below there was only one pool - pool0 which is now shown as the selected pool in brackets.
nas-1a:configuration storage (pool-0)>
nas-1a:configuration storage (pool-0)> add

This should now give you the prompt
nas-1a:configuration storage (pool-0) verify>

You can verify that you can see the readzilla cache device now by using "show"
nas-1a:configuration storage (pool-0) verify> show

This should show a summary of the devices seen - your readzilla(s) should be visible under the CACHE heading.
If this all looks OK simply type "done" to acknowledge the "verify".
nas-1a:configuration storage (pool-0) verify> done

Now type " done" again to finish the operation and add the readzilla cache device(s).

BUI

Click through to the "Configuration -> Storage" screen.

Under "Available Pools" select the correct pool and then click on the "ADD" button.

This will take you to the next screen which should show the readzilla(s) under "Cache Devices"

Provided this looks correct click on "COMMIT" to add the readzilla(s)

See the full example below:

dwstor1:> configuration storage
dwstor1:configuration storage (Oracle)> ls
Pools:

      POOL             OWNER         DATA PROFILE  LOG PROFILE   STATUS
pool = Oracle           dwstor1       mirror        log_mirror    online
      Oracle1          dwstor2       -             -             exported

Properties:
                         pool = Oracle
                       status = online
                        owner = dwstor1
                      profile = mirror
                  log_profile = log_mirror
                cache_profile = cache_stripe
                        scrub = resilver completed after 0h0m with 0 errors
dwstor1:configuration storage (Oracle)> add

Verify and add devices. Devices may be added on a per-device basis, however
SATA devices in SAS-1 enclosures may be added in half- or whole-chassis units
only. While affected devices may be added, they will not be available for use
without reconfiguring the pool; for best results, defer configuring storage
until any problems can be repaired. Mixing devices of differing speeds within a
storage pool is strongly discouraged.

Subcommands that are valid in this context:

  help [topic]         => Get context-sensitive help. If [topic] is specified,
                          it must be one of "builtins", "commands", "general",
                          "help", "script" or "properties".

  show                 => Show information pertinent to the current context

  abort                => Abort this task (potentially resulting in a
                          misconfigured system)

  done                 => Finish operating on "verify"

  get [prop]           => Get value for property [prop]. ("help properties"
                          for valid properties.) If [prop] is not specified,
                          returns values for all properties.

  set [prop]           => Set property [prop] to [value]. ("help properties"
                          for valid properties.) For properties taking list
                          values, [value] should be a comma-separated list of
                          values.

dwstor1:configuration storage (Oracle) verify> show

       ID     TATUS   ALLOCATION   DATA          LOG           CACHE       RPM
       --    -------   ----------   -----------   -----------   ----------- -----
        0         ok       custom   0             0             1/1   93.2G
                                    -----------   -----------   -----------
                                    0             0             1     93.2G

dwstor1:configuration storage (Oracle) verify> done

Choose Storage Profile. Configure available storage into a pool by defining its
underlying redundancy profile. Carefully read the profile descriptions to
understand how each balances the inherent trade-offs between availability,
performance, and capacity, and select the profile that best fits your workload.
If available, NSPF indicates no single point of failure, which affords certain
profiles the ability for a pool to survive through loss of a single disk shelf.

Subcommands that are valid in this context:

  help [topic]         => Get context-sensitive help. If [topic] is specified,
                          it must be one of "builtins", "commands", "general",
                          "help", "script" or "properties".

  show                 => Show information pertinent to the current context

  abort                => Abort this task (potentially resulting in a
                          misconfigured system)

  done                 => Finish operating on "config"

  get [prop]           => Get value for property [prop]. ("help properties"
                          for valid properties.) If [prop] is not specified,
                          returns values for all properties.

  set [prop]           => Set property [prop] to [value]. ("help properties"
                          for valid properties.) For properties taking list
                          values, [value] should be a comma-separated list of
                          values.

dwstor1:configuration storage (Oracle) config> done
dwstor1:configuration storage (Oracle)> cd /

After the readzilla has been added the pool looked like this - note the extra readzilla c7t0d0 in the cache section:

dwstor1:> confirm shell zpool status
 pool: Oracle
state: ONLINE
status: One or more devices has experienced an unrecoverable error.  An
       attempt was made to correct the error.  Applications are unaffected.
action: Determine if the device needs to be replaced, and clear the errors
       using 'zpool clear' or replace the device with 'zpool replace'.
  see: http://www.sun.com/msg/ZFS-8000-9P
scan: resilvered 47.5K in 0h0m with 0 errors on Fri Feb 17 14:23:55 2012
config:

       NAME                                           STATE     READ WRITE CKSUM
       Oracle                                         ONLINE       0     0     0
         mirror-0                                     ONLINE       0     0     0
           c5t5000C50019E42A14d0                      ONLINE       0     0     0
           c5t5000C50019FBE503d0                      ONLINE       0     0     0
         mirror-1                                     ONLINE       0     0     0
           c5t5000C50019FBF8FFd0                      ONLINE       0     0     0
           c5t5000C50019FBF125d0                      ONLINE       0     0     0
         mirror-2                                     ONLINE       0     0     0
           c5t5000C50019FC4CD4d0                      ONLINE       0     0     0
           c5t5000C50019FC5D41d0                      ONLINE       0     0     5
         mirror-3                                     ONLINE       0     0     0
           c5t5000C50019FC509Bd0                      ONLINE       0     0     0
           c5t5000C50019FCE67Dd0                      ONLINE       0     0     0
       logs
         mirror-4                                     ONLINE       0     0     0
           c5tATASTECZEUSIOPS018GBYTESSTM0000D8A0Ad0  ONLINE       0     0     0
           c5tATASTECZEUSIOPS018GBYTESSTM0000F5FD9d0  ONLINE       0     0     0
       cache
         c0t0d0                                       ONLINE       0     0     0
         c0t1d0                                       ONLINE       0     0     0
         c7t0d0                                       ONLINE       0     0     0
       spares
         c5t5000C50019FDA86Cd0                        AVAIL
         c5t5000C50019FD3114d0                        AVAIL

errors: No known data errors

 pool: system
state: ONLINE
scan: resilvered 4.80M in 0h0m with 0 errors on Fri Apr 15 11:21:41 2011
config:

       NAME          STATE     READ WRITE CKSUM
       system        ONLINE       0     0     0
         mirror-0    ONLINE       0     0     0
           c1t1d0s0  ONLINE       0     0     0
           c1t0d0s0  ONLINE       0     0     0

errors: No known data errors

Removing Readzillas

There is no way for a customer to remove readzillas via the BUI or CLI. This requires a support call to be raised so an appropriately trained Oracle support analyst can carry out the procedure.

See https://stbeehive.oracle.com/teamcollab/wiki/AmberRoadSupport:Introduction+to+Readzillas for further details on removing Readzillas from a pool.

References

https://stbeehive.oracle.com/teamcollab/wiki/AmberRoadSupport:Introduction+to+Readzillas

Attachments

This solution has no attachment