Sun Fire B10n Content Load Balancing Blade
 
Sun Fire B10p SSL Proxy Blade Server
 
Sun Fire B1600 Blade System Chassis
 
Sun Fire B100s Blade Server
 

GCS>Sun Microsystems>Servers>Blade Servers
 

1) B10n
2) B10p
3) Router (a ServerBlade can also act as a router)
4) B1600 switch
5) Server Blades
6) Clients (a ServerBlade can also act as client)

1) B10n Setup

Some limits:
- Maximum number of SSL blades that can be added per service - 16
- Maximum number of SSL blade entries that can be created on a B10n - 128

Configuring the network/vlan:
-----------------------------
1.1 config ip interface 0 192.4.142.79 mask 255.255.255.0

       - Set the IP address on interface 0 to 192.4.142.79 with a subnet
mask of 255.255.255.0

1.2 config data vlan 10

       - Set the data/client VLAN to 10.

1.3 config enable vlan data

       - Enable the data/client VLAN.

1.4 config management vlan 30

       - Set the management VLAN to 30

1.5 config enable vlan management

       - Enable the management VLAN.

Configuring B10p's information:
-------------------------------
1.6 config ssl name ssl1 192.4.142.58

       - Create an SSL blade entry on B10n with the name "ssl1" and one
interface specified at 192.4.142.58. NOTE: The interface IP address
should correspond to the one configured on the SSL blade with the
"set management" command.

1.7 config ssl port-pair ssl1 secureport 443 clearport 880

       - Add a port pair to the entry with the secureport specified at 443 and
the clearport specified at 880. NOTE: This should correspond to the
same values specified on B10p blade with the "set portpair"
command (see section 2 for B10p setup). 

Verifying B10p configuration on B10n:
-------------------------------------
1.8 show ssl

       - Displays basic information about all the SSL blades configured on
B10n

1.9 show ssl ssl1

       - Displays detailed information about the SSL blade entry "ssl1".

Configuring a Layer 7 SSL service on B10n:
------------------------------------------
1.10 config service name svc1 vip 192.50.50.2:443:tcp ssl 880 interface 0
lb-layer 7 l7-proto http

       - Create an SSL service on B10n that is load balanced on layer 7 for
the HTTP protocol. The service "svc1" is bound to interface 0 and
is offered at the VIP 192.50.50.2, port 443 and TCP protocol. The
port specified after the ssl keyword, i.e., 880, is the decrypted
port.

NOTE: The VIP specified for the service, i.e., 192.50.50.2 in this example should be configured as the server address in the "create service" command on
all the SSL blades added to the service. The service port (443 in this
example) should correspond to the secure port of the port pair associated to
the service on B10p and the decrypted port (880 in this example) should
correspond to the clear port of the port pair on B10p.

add netmask to VIP: config vip-netmask {ip addr/hostname} mask netmask

1.11 config service lb-group default svc1 server 192.4.142.71:0:tcp:2:1
192.4.142.80:0:tcp:3:1 192.4.142.74:0:tcp:4:1 192.4.142.75:0:tcp:5:1
192.4.142.77:0:tcp:6:1 scheme wt-round-robin

       - Configure the default load balancing group of the service with 5
servers 192.4.142.71, 192.4.142.80, 192.4.142.74, 192.4.142.75 and
192.4.142.77 and the LB scheme specified as weighted round robin.

       - B10n Management IP address and Server Management IP address should
be on the same subnet.

1.12 config service ssl svc1 ssl ssl1:active

       - Add the SSL blade entry "ssl1" to the service in an active mode.

NOTE: An SSL service cannot be enabled until one or more SSL entries are
added to it using the above command.

1.13 config service vlan svc1 vlan 50

       - Set the service VLAN to 50. B10n will tag all traffic from this
service, destined to the backend servers with the VLAN ID 50 when
VLAN is enabled on the service.

1.14 config enable service vlan svc1

       - Enable VLAN tagging for the service.

1.15 config enable service name svc1

       - Enable the service "svc1" on B10n.

1.16 commit force

       - Save the configuration changes

Checking the service config on B10n:
------------------------------------
16. show service svc1

2) B10p Setup:

2.1 create key

Enter key name: key1
Enter key strength (1024): 1024
Key key1 generated.

       - This creates the key "key1" on B10p. Use "show key" to display all
the keys configured on the B10p board.

2.2 create certificate

Enter key name: key1
Enter country (US): US
Enter state or province (CA): CA
Enter locality (Company Town): Newark
Enter common name (www.companyname.com): www1.sun.com
Enter organization (Company Name): Sun Microsystems
Enter organization unit (Company Unit): PTS
Enter email address ([email protected]): [email protected]
Certificate key1 generated.

       - This creates a certificate using the key "key1". Use "show key" to
display the certificate along with the key.

2.3 set routed

Enter port number (1..2) (1): 1
Enter router inbound IP address (x.x.x.x): 192.4.142.79
Enter primary router outbound IP address (x.x.x.x): 192.100.100.254
Enter secondary router outbound IP address (x.x.x.x): 0.0.0.0

       - This sets the parameters on port 1 for operation of B10p in the
routed mode.

NOTE: The router inbound IP address corresponds to the management IP address
configured on B10n with the "config ip" command.

2.4 set inband

Enter port number (1..2) (1): 1
Enter inband (data) IP Address (x.x.x.x): 192.100.100.205
Enter inband (data) netmask (x.x.x.x): 255.255.255.0

       - This sets the inband (data) IP address on port 1 to 192.100.100.205
with a subnet mask of 255.255.255.0.

NOTE: This address has to be on the same subnet as the outbound router IP
address as configured by the "set routed" command.

2.5 set management

Enter port number (1..2) (1): 1
Enter inband (admin) IP Address (x.x.x.x): 192.4.142.58
Enter inband (admin) netmask (x.x.x.x): 255.255.255.0
Enter inband (admin) gateway (x.x.x.x): 0.0.0.0

       - This sets the management parameters on port 1. The management IP is
set to 192.4.142.58 with a subnet mask of 255.255.255.0.

NOTE: This is the IP used for health checks towards the inbound router, i.e.,
B10n and also the one configured on B10n, for B10n to perform health checks on
B10p.

2.6 set vlan client 10

       - This sets the client VLAN as 10.

NOTE: This is the VLAN on which all SSL encrypted traffic (to be load
balanced) from the client is sent. The value should also correspond to that
set on B10n with the "config data vlan" command.

2.7 set vlan management

Enter port number (1..2) (1): 1
Enter management vlan tag (admin) (0..4095): 30

       - This sets the management VLAN on port 1 to 30

NOTE:
- This is the VLAN on which all the management traffic from B10p is
sent (i.e., for ftp, export, health checks towards the inbound router etc.).
The value should also correspond to that set on B10n with the
"config management vlan" command.
- B10p management IP address and B10n management IP address should be on the
same subnet.

2.8. set vlan inband

Enter port number (1..2) (1): 1
Enter management vlan tag (0..4095): 10

       - This sets the inband (data) VLAN on port 1 to 10.

NOTE: This is the VLAN on which all health check traffic towards the outbound
router is sent out. Its value should correspond to that used in the "set vlan client" command on B10p.

2.9 set vlan filter enable

       - This enables the VLAN filtering on B10p. This means that B10p
will not process any incoming traffic on the client VLAN (10 in
this example). This filtering is a security measure on B10p.

NOTE: For B10n load balancing with B10p, the VLAN filter has to be "enabled".

2.10 set portpair

Enter portpair number (1..4) (1): 1
Enter secure port (https) (443): 443
Enter clear port (http) (880): 880

       - This configures port pair 1 on B10p with the secure port
specified as 443 and the clear port specified as 880.

NOTE: Upto 4 such port pairs can be configured on B10p. The maximum value of each port cannot exceed 1000. Each of the 8 ports in the 4 port pairs should be unique.

2.11 create service, e.g.,

Enter service name: svc1
Enter key name: key1
Enter server IP Address (0.0.0.0): 192.50.50.2
Enter cipher (export/best/optimal/high/medium/low) (best): best
Enter portpair number (1..4) (1): 1
Service svc1 created.

       - This creates a service "svc1" on B10p with the key "key1"
associated with it. The service is offered at the IP address
192.50.50.2. The "best" cipher is chosen for this service and port
pair 1 (with secure port 443 and clear port 880) is configured for
the service. Use "show service" to display all the services
configured on the B10p blade.

NOTE: Unique keys/certificates should be used for each service configured on a
B10p blade. The same key/certificate should be used for the same service
configured on multiple B10ps.

3) Router Setup (Using a ServerBlade as a router)

I)
* rm /etc/notrouter
* ndd -set /dev/ip ce0:ip_forwarding 1
* ndd -set /dev/ip ce10000:ip_forwarding 1

II)

3.1 ifconfig ce0 plumb 192.60.60.254 netmask 255.255.255.0 broadcast + up
clients are in 192.60.60.0 subnet. VLAN's are not configured on client

3.2 ifconfig ce0 addif 10.4.142.78 netmask 255.255.255.128 broadcast + up
10.4.142.78 is assigned for SWAN access (Optional step )

3.3 ifconfig ce10000 plumb 192.50.50.254 netmask 255.255.255.0 broadcast + up
On client/data VLAN, i.e., 10, for route from client to VIP on VIP side,
e.g., 192.50.50.254
NOTE: the VIPs are in the 192.50.50.0 subnet.

3.4 ifconfig ce10000 addif 192.100.100.254 netmask 255.255.255.0 broadcast + up
On client/data VLAN, i.e., 10 for server and cougar route to client, e.g.,
192.100.100.254.
NOTE: This is the address configured as the outbound router on B10p.

3.5 ifconfig ce30000 addif 192.4.142.78 netmask 255.255.255.0 broadcast + up
B10n , cougar & server blades communicate on 192.4.142.0 subnet
Configured 192.4.142.78 on router for testing purpose ( optional step)

4) B1600 Switch (SSC0/SWT) Setup:

4.1 Creating VLANs:

       Console#configure vlan database

       Console(config-vlan)#

Console(config-vlan)#vlan 30 name mgmt-vlan media ethernet
- Creates the management VLAN 3o

Console(config-vlan)#vlan 10 name client-vlan media ethernet
- Creates the client/data VLAN 10

       Console(config-vlan)#vlan 50 name service-vlan media ethernet
- Creates the service VLAN 50

4.2 Configure slot with B10n (e.g., S15) to allow the management, client and
service VLANs

       Console#configure
Console(config)#interface ethernet SNP15
Console(config-if)#
Console(config-if)#switchport allowed vlan add 30 tagged
Console(config-if)#switchport allowed vlan add 10 tagged
Console(config-if)#switchport allowed vlan add 50 tagged

4.3 Configure slot with B10p to allow the management, client and
service VLANs.
- Repeat steps in 4.2 for SNPxx, where xx is the slot # for B10p.

4.4 Configure slots with server blades to allow the management, client and
service VLANs.
- Repeat steps in 4.2 for SNPxx, where xx is the slot # for ServerBlade.

4.5 Configure uplink slot with Router to allow the management and client
VLANs.
- Repeat steps in 4.2 for SNPxx, where xx is the slot # for the Router.
- Repeat steps in 4.2 for NETPxx, where xx is the PORT # where external
router is connected.
- Service VLAN should not be configured on Router(external router or
serverblade acting as router)

4.6 Configure uplink slot with client to allow the management and client
VLANs.
- Repeat steps in 4.2 for NETPxx, where xx is the PORT # where external
router for the client side is connected.

5) Blade Servers' Setup:

Note: This example uses SPARC(R) Serverblade ( B100s)

5.1 Download/Install the clbmod packages.
cd <location of the clbmod packages>
pkgadd -d .

5.2 Configure the interfaces on the server (Assuming, switch 0 is active, so
interface ce0 is being configured):

    ifconfig ce0 plumb 10.4.142.71 netmask 255.255.255.128 broadcast + up
- 10.4.142.71 is assigned to access the server blade from SWAN

    ifconfig ce30000 plumb 192.4.142.71 netmask 255.255.255.0 up
- Configure the real IP on the management vlan 30
- ce3000 will be configured when vlan 30 is used

    ifconfig ce50000 plumb 188.88.8.5 netmask 255.255.255.0 up
- Configure any (unique) IP on the service vlan 50
- There is no use of the 188.88.8.5 IP address

    ifconfig ce10000 addif 192.100.100.71 netmask 255.255.255.0 up
- Configure IP on the client/data vlan 10 to reach the client
through the router

    ifconfig lo0:1 plumb 192.50.50.2 netmask 255.255.255.0 up
- Configure the VIP(s) on the loopback interface

5.3 Add the interfaces to the clbmod:
/opt/SUNWclb/bin/clbconfig add ce30000
/opt/SUNWclb/bin/clbconfig add ce50000
/opt/SUNWclb/bin/clbconfig add ce10000
- add ce30000, ce50000, ce10000 to /etc/opt/SUNWclb/clb.conf,
one on each line, to automatically add the interfaces to
clbmod accross reboots.

5.4 Load the module:
/etc/init.d/clbctl start

5.5 Check the interfaces on which the module is plumbed:
/opt/SUNWclb/bin/clbconfig list

5.6 Make sure the servers are not routing, i.e., /etc/notrouter file should
be present.

5.7 kstat clbmod
- use this command to monitor open/close connections on clbmod

5.8 Configure the bundled Apache Web Server
- Add the server ipaddress, 192.4.142.71, to /etc/hosts
- cp /etc/apache/httpd.conf.example /etc/apache/httpd.conf
- Add the line "ServerName 192.4.142.71" at the end of Servername section.
- Add the lines "Listen 80" and "Listen 880" at the end of Listen section.
- Use "/usr/apache/bin/apachectl start" to start the Web Server

5.9 Repeat the above steps to configure more ServerBlades (192.4.142.80..74.
.75..77 etc)

NOTE: Serverblade sends unencrypted response traffic to B10p blade encryption.

6.) Clients (a ServerBlade can also act as client)

- Using external sun machine as a client

6.1) Configure client IP in this example
ifconfig ge0 plumb 192.60.60.253 netmask 255.255.255.0 broadcast + up

6.2) Add static route to VIPs
route add -net 192.50.50.0 192.60.60.254 -static
(NOTE: Here we assume the VIPs are in the 192.50.50.0 subnet).

6.3) Launch web browser to access https://192.50.50.2 web page

This solution has no attachment