Sun Storage 7310 Unified Storage System
 
Sun Storage 7410 Unified Storage System
 
Sun ZFS Storage 7120
 
Sun Storage 7110 Unified Storage System
 
Sun ZFS Storage 7320
 
Sun ZFS Storage 7420
 
Sun Storage 7210 Unified Storage System
 

PLA-Support>Sun Systems>DISK>NAS>SN-DK: 7xxx NAS
 
.Old GCS Categories>Sun Microsystems>Storage - Disk>Unified Storage
 

In this Document

Symptoms

Cause

Solution

Generic information

VmWare

Solaris Client

Oracle database

References

Applies to:

Symptoms

In some circumstances, some NFS latencies might be discernible on NFS client side. This document provides some typical scenarios where NFS can be tuned to get the best performance.

Cause

Most of the tuning is to be done from client side. The next section introduces some best practices for VmWare as well as Oracle Database. Some of the information can be applied for any NFS client type.

Solution

Generic information

• Number of NFS concurrent connections:
  The only performance issue associated with NFS connections is having too few (a single one) from a single client trying to push 10Gbe Ethernet. Otherwise, do not worry about connections. In the implementation, we are "limited" to about 4 billion connections per system, at least in theory. In practice you're more likely to hit performance limits long before you get to that many - eg if each client produces 1 op/sec, you won't come anywhere close to 1 billion ops/sec. Additionally, you'll probably run into some physical memory limitations before you get very far into the millions of clients.

• Maximum Number of server threads:
  This should at least cover the number of concurrent NFS clients that is anticipated. Allowed range is 20 to 1000. The default value is 500, which is just a trade-off. This can be increased if you want to allocate more CPUs cycles to NFS protocol.

VmWare

• Filesystem block size:
  This is an important topic for best performance. For vSphere 4.x and recommended recordsize, see vmware-refarch-354079.pdf

• Number of guests per NFS share. There no performance improvements on increasing the number of NFS shares on the same pool. For snapshot/replication policies, it is logically easier to group some guests in some specific shares.

Solaris Client

• Parallelism:

To maximize parallelism between an NFS client and the ZFS storage appliance, the following entry can be added in /etc/system on client.

set rpcmod:clnt_max_conns = 8

As of S10u8, the default is 1, which allows for a single connection from the client to each server. This results in a single thread bottleneck for communication with each server. Increasing this value on the client increases the number of connections to the NAS head.

• Attribute caching on client side:

See <Document 1164673.1> : NFS Performance Decline Introduced by Mount Option "actimeo=0". 

With actimeo=0, you disable attribute caching in the client, which can result in getattr storms during heavy file open/close activity.  With noac, you also disable client side write behind caching but still allow read caching (assuming forcedirectio is not also enabled). In Solaris, that turns all of your writes into synchronous 8KB writes to the server, which can have a dramatic performance impact. You should really never use actimeo=0 or noac unless explicitly required by an application. The only common application that should be using either of these is Oracle RAC. If you are given the choice of actimeo=0 or noac, choose actimeo=0.

Oracle database

There is a lot of conflicting information on this subject from a lot of different source.  The best reference of describing what mount options to use is <Document 359515.1> : Mount Options for Oracle files when used with NAS devices

• wsize and rsize:
  In <Document 359515.1>, rsize and wsize are set to 32kB, which is conservative for a lot of NAS heads and network infrastructures. The good news is that 32kB always works, but the bad news is that another choice might work better. In the case of the ZFSSA, use a 128kB rsize and a 1024kBwsize for optimal performance on Q3.4 code and earlier. When we get the 2011Q1 code (next major release) we can use 1024kB for both rsize and wsize.

The 128kB rsize is necessary due to a CPU xcall problem related to using kmem_alloc when we do an I/O larger than 128kB, and this is fixed in 2011Q1.
An rsize greater than 128kB will bottleneck the ZFS appliance at about 400 MBPS due to CPU xcalls (fixed in 2011Q1).

In Short, use 128KB for rsize and 1024KB for wsize.

 

Back to <Document 1331769.1> Sun Storage 7000 Unified Storage System: How to Troubleshoot Performance Issues.

References

This solution has no attachment