OKM/KMS - All agents take a long time to retrieve keys from KMA(s)

Asset ID:	1-72-1476270.1
Update Date:	2012-07-23
Keywords:

Solution Type Problem Resolution Sure

Solution 1476270.1 : OKM/KMS - All agents take a long time to retrieve keys from KMA(s)

Applies to:

Sun StorageTek SL8500 Modular Library System - Version All Versions and later
Information in this document applies to any platform.

Symptoms

Reported tape job failure due to timeout using OKM solutions, This was because encryption keys were intermittently provided after 3-5 minutes from agent key request. VOP log shows below (omitted messages) and took for 3 minutes with failover.

4:34 PM May 10, 2012 - Cartridge loaded is encrypted

4:34 PM May 10, 2012 - KMS2.0:my_tcp_connect : Socket endpoint: https://10.3.128.193:3334 host: 10.3.128.193 port: 3334

4:35 PM May 10, 2012 - KMS2.0:my_tcp_connect : select err=236 [Operation now in progress]

4:36 PM May 10, 2012 - KMS2.0:msg#=520,op=AGENT_LOADBALANCER_FAILOVER

4:36 PM May 10, 2012 - msg=Failing over to retry this addr,eid=,addr=10.3.128.194

4:36 PM May 10, 2012 - KMS2.0:msg#=796,op=AUDIT_CLIENT_AGENT_RETRIEVE_DATA_UNIT_SOAP_ERROR

4:36 PM May 10, 2012 - msg= SoapFaultCode=SOAP-ENV:Client SoapFaultString=Timeout

4:36 PM May 10, 2012 - KMS2.0:my_tcp_connect : Socket endpoint: https://10.3.128.194:3334 host: 10.3.128.194 port: 3334

4:36 PM May 10, 2012 - KMS2.0:my_tcp_connect : select err=236 [Operation now in progress]

4:37 PM May 10, 2012 - KMS2.0:msg#=520,op=AGENT_LOADBALANCER_FAILOVER

4:37 PM May 10, 2012 - msg=Failing over to retry this addr,eid=,addr=10.3.128.194

4:37 PM May 10, 2012 - KMS2.0:msg#=793,op=AUDIT_CLIENT_AGENT_RETRIEVE_DATA_UNIT_KEYS_SOAP_ERROR

4:37 PM May 10, 2012 - msg= SoapFaultCode=SOAP-ENV:Client SoapFaultString=Timeout 4:37 PM May 10, 2012 - Loading 1 key(s) into encryption hardware

4:37 PM May 10, 2012 - Tape Cartridge is LOADED

Cause

Cabled incorrectly.

Solution

Both KMA dump dladm.txt show aggregation ports (nge0) were up as below.

nge1 0:26:9e:2:74:5 1000 Mbps full up attached

nge0 0:26:9e:2:74:4 1000 Mbps full up attached

nge1 0:26:9e:2:76:b9 1000 Mbps full up attached

nge0 0:26:9e:2:76:b8 1000 Mbps full up attached

Interviewed actual network cable connection to the customer and reported both KMAs LAN3 (nge0) ports were connected with each other. This is NOT recommended configuration.
Everything was fine after removing this cable. Both dladm.txt show normal state.

nge1 0:26:9e:2:76:b9 1000 Mbps full up attached

nge0 0:26:9e:2:76:b8 0 Mbps unknown down standby

nge1 0:26:9e:2:76:b9 1000 Mbps full up attached

nge0 0:26:9e:2:76:b8 0 Mbps unknown down standby

Solutions : Removed LAN3 (nge0) cable

Provided by Naoshi Iida

Attachments

This solution has no attachment