IMG 1010 - GCEMS redundancy - Utilizing Bonding
To aid in the GCEMS redundancy fail-over situation, a technique labeled Bonding can be applied to the Network Cards on any GCEMS servers utilizing GCEMS or Host Redundancy. Bonding along with a specific network setup (both displayed in diagrams below), can be utilized so the IMG has full GCEMS redundancy in the event a failure occurs. The Bonding technique being employed on the GCEMS is labeled active-backup mode. Bonding is supported on the GCEMS servers that are running Red Hat Enterprise Linux 5.5 and above. Refer to the topics listed in the Related Items heading below for more information. Refer to the information below.
Network Diagram (One subnet scenario)
Configuration (One Subnet)
The configuration below describes the configuration needed to set up bonding on the GCEMS servers when there is only one subnet involved. All modifications below are to be done as root user only.
Create the bond0 and bond1 files
Configure the IP address, netmask, gateway, and bonding options on the server running the primary GCEMS. Using the vi editor, create the ifcfg-bond0 file and enter the following information. (Update values displayed below to match your network configuration)
[root@localhost]# vim /etc/sysconfig/network-scripts/ifcfg-bond0 DEVICE=bond0 IPADDR=192.168.4.10 NETMASK=255.255.255.0 GATEWAY=192.168.4.1 USERCTL=no BOOTPROTO=none ONBOOT=yes IPV6INIT=no BONDING_OPTS=”mode=1 miimon=100 use_carrier=0 primary=eth0 primary_reselect=2”
Additional Information:
IPADDR = IP Address being configured on active network interface on primary GCEMS server.
NETMASK = Subnet Mask being configured on the active network interface on primary GCEMS server.
GATEWAY = IP address of the network gateway in your network
mode=1 - Sets mode to be active-backup.
miimon=100 - Sets the link monitoring interval to 100 ms
use_carrier=0 - Sets the method used for miimon and improves link detection by miimon
primary=eth0 - Defines eth0 as the “active” port
primary_reselect=2 (failure) - The primary slave becomes the active slave only if the current active fails and the primary slave is up.
Configure the IP address, netmask, gateway, and bonding options on the server running the standby GCEMS. Using the vi editor, create the ifcfg-bond1 file and enter the information displayed in step 1 above using values that match your network configuration).
Modify the existing Ethernet configuration for bonding on the server running primary GCEMS
Create a backup of the existing ifcfg-eth0 and ifcfg-eth1 files located in the /etc/sysconfig/network-scripts directory. See examples below.
Create backup of ifcfg-eth0 file.
[root@localhost]# cp ifcfg-eth0 backup.ifcfg-eth0
Create backup of ifcfg-eth1 file.
[root@localhost]# cp ifcfg-eth1 backup.ifcfg-eth1
Using vi editor, comment out the parameters in the ifcfg-eth0 file that will be configured in bond0. See example below.
$ vim ifcfg-eth0 DEVICE=eth0 BOOTPROTO=none ONBOOT=yes MASTER=bond0 SLAVE=yes # TYPE=Ethernet # HWADDR=xx:xx:xx:xx:xx:xx # IPADDR=192.168.4.2 # NETMASK=255.255.255.0 # GATEWAY=192.168.4.0 # USERCTL=no # IPV6INIT=no # PEERDNS=yes
Do the same for ifcfg-eth1. See example below.
$ vim ifcfg-eth1 DEVICE=eth1 BOOTPROTO=none ONBOOT=yes MASTER=bond0 SLAVE=yes # TYPE=Ethernet # HWADDR=xx:xx:xx:xx:xx:xx # IPADDR=192.168.4.3 # NETMASK=255.255.255.0 # GATEWAY=192.168.4.0 # USERCTL=no # IPV6INIT=no # PEERDNS=yes
Configure Bonding
Modify the /etc/modproble.conf file. Follow examples below.
Create a backup of the modprobe.conf file.
[root@localhost]# cd /etc [root@localhost]# cp modprobe.conf backup.modprobe.conf
Using vi editor, add the parameters below to the modprobe.conf file.
$ vim /etc/modprobe.conf alias bond0 bonding options bond0 mode=active-backup miimon=100 primary=eth0 primary_reselect=2
Activate Bonding and restart the service
Load the bond module and restart the service. See example below.
[root@localhost]# /sbin/modprobe bonding [root@localhost]# /sbin/service network restart
Enter the commands below to verify the Bonding settings are applied correctly
[root@localhost]# cat /proc/net/bonding/bond0 Ethernet Channel Bonding Driver: v3.4.0 (October 7, 2008) Bonding Mode: fault-tolerance (active-backup) Primary Slave: eth0 (primary_reselect failure) Currently Active Slave: eth0 MII Status: up MII Polling Interval (ms): 100 Up Delay (ms): 0 Down Delay (ms): 0 Slave Interface: eth0 MII Status: up Link Failure Count: 4 Permanent HW addr: xx:xx:xx:xx:xx:xx Slave Interface: eth1 MII Status: up Link Failure Count: 2 Permanent HW addr: xx:xx:xx:xx:xx:xx
Network Diagram (Two subnet scenario)
The dashed blue/red lines indicate that the GCEMS can exist on multiple VLANs, not just the vlan for OAM&P. For example:
Red = Network Management
Blue = SIP, RTP, RADIUS
Configuration (Two Subnets)
The configuration below describes the configuration needed to set up bonding on the GCEMS servers when there are two subnets involved. Note the diagram that extra hardware is needed. Before starting the procedure, verify the following:
The NICs on the server must support IEEE Standard 802.1Q, VLAN Tagging.
The switch must be a managed switch with support for port-based VLAN and VLAN Tagging
The procedure below configures two VLANS. One with VLAN ID=101 and another with VLAN ID=102. OAMP traffic will use VLAN ID 101 and VoIP traffic will be on VLAN ID 102 ports. See table below.
OAMP private subnet | Remotely accesible subnet |
IP Address = 192.168.4.20 | IP Address = 10.129.44.10 |
Netmask = 255.255.255.0 | Netmask = 255.255.255.224 |
Network Gateway = 192.168.4.1 | Network Gateway = 10.129.44.1 |
VLAN ID = 101 | VLAN ID = 102 |
Create the bond0 and bond1 files
Configure the IP address, netmask, gateway, and bonding options on the server running the primary GCEMS. Using the vi editor, create the ifcfg-bond0 file and enter the following information. (Update values displayed below to match your network configuration)
[root@localhost]# vim /etc/sysconfig/network-scripts/ifcfg-bond0 DEVICE=bond0 IPADDR=192.168.4.10 NETMASK=255.255.255.0 GATEWAY=192.168.4.1 USERCTL=no BOOTPROTO=none ONBOOT=yes IPV6INIT=no BONDING_OPTS=”mode=1 miimon=100 use_carrier=0 primary=eth0 primary_reselect=failure”
Additional Information:
IPADDR = IP Address being configured on active network interface on primary GCEMS server.
NETMASK = Subnet Mask being configured on the active network interface on primary GCEMS server.
GATEWAY = IP address of the network gateway in your network
mode=1 - Sets mode to be active-backup.
miimon=100 - Sets the link monitoring interval to 100 ms
use_carrier=0 - Sets the method used for miimon and improves link detection by miimon.
primary=eth0 - Defines eth0 as the “active” port
primary_reselect=2 (failure) - The primary slave becomes the active slave only if the current active fails and the primary slave is up.
Configure the IP address, netmask, gateway, and bonding options on the server running the standby GCEMS. Using the vi editor, create the ifcfg-bond1 file and enter the information displayed in step 1 above using values that match your network configuration).
Modify the Bond Interface created
The procedure below describes how to modify the ifcfg-bond0 for use with VLAN tagging. Comment out the following parameters (IPADDR, NETMASK, GATEWAY) from ifcfg-bond0 created above. The commented out parameters will be configured in another file.
Go to the /etc/sysconfig/network-scripts directory and using a vi editor, comment out the IPADDR, NETMASK, and GATEWAY parameters. See example below.
[root@localhost]# cd /etc/sysconfig/network-scripts [root@localhost]# vim ifcfg-bond0 DEVICE=bond0 # IPADDR=192.168.4.10 # NETMASK=255.255.255.0 # GATEWAY=192.168.4.1 USERCTL=no BOOTPROTO=none ONBOOT=yes IPV6INIT=no BONDING_OPTS=”mode=1 miimon=100 use_carrier=0 primary=eth0 primary_reselect=failure”
Bond configuration for VLAN ID 101 and VLAN ID 102
For different VLAN ID's create bond device bond0.y where the y value determines the VLAN ID. Create two new bond interfaces for VLAN tagging: Ifcfg-bond0.101 and ifcfg-bond0.102.
Using the vi editor, create the ifcfg-bond0.101. Configure IPADDR, NETMASK, and GATEWAY, and then enable VLAN tagging. See example below.
Create ifcfg-bond0.101
[root@localhost]# cp bond0 bond0.101 [root@localhost]# vim bond0.101 DEVICE=bond0.101 IPADDR=192.168.4.10 NETMASK=255.255.255.0 GATEWAY=192.168.4.1 ONBOOT=yes VLAN=yes
Using the vi editor, create the ifcfg-bond0.102. Configure IPADDR, NETMASK, and GATEWAY, and then enable VLAN tagging. See example below.
[root@localhost]# cp bond0 bond0.102 [root@localhost]# vim bond0.102 DEVICE=bond0.102 IPADDR=10.129.44.248 NETMASK=255.255.255.224 GATEWAY=10.129.44.225 ONBOOT=yes VLAN=yes
Configure Bonding
Check /etc/modprobe.conf for the following bonding components.
[root@localhost]# cd /etc [root@localhost]# vim /etc/modprobe.conf alias bond0 bonding options bond0 mode=active-backup miimon=100 primary=eth0 primary_reselect=2
Activate Bonding and Restart Network
Load the bond module and restart the network.
[root@localhost]# /sbin/modprobe bonding [root@localhost]# /sbin/service network restart
Verify the bonding settings were applied by entering the following command.
[root@localhost]# cat /proc/net/bonding/bond0 Ethernet Channel Bonding Driver: v3.4.0 (October 7, 2008) Bonding Mode: fault-tolerance (active-backup) Primary Slave: eth0 (primary_reselect failure) Currently Active Slave: eth0 MII Status: up MII Polling Interval (ms): 100 Up Delay (ms): 0 Down Delay (ms): 0 Slave Interface: eth0 MII Status: up Link Failure Count: 0 Permanent HW addr: 00:xx:xx:xx:xx:xx Slave Interface: eth1 MII Status: up Link Failure Count: 1 Permanent HW addr: 00:xx:xx:xx:xx:xx
[root@localhost]# /sbin/ifconfig
bond0 Link encap:Ethernet HWaddr 00:xx:xx:xx:xx:xx
inet6 addr: fe80::xxx:xxxx:xxxx:xxx/64 Scope:Link
UP BROADCAST RUNNING MASTER MULTICAST MTU:1500 Metric:1
RX packets:1518004 errors:0 dropped:0 overruns:154 frame:0
TX packets:1037592 errors:0 dropped:0 overruns:0 carrier:150503
collisions:0 txqueuelen:0
RX bytes:496743533 (473.7 MiB) TX bytes:83336021 (79.4 MiB)
bond0.101 Link encap:Ethernet HWaddr 00:xx:xx:xx:xx:xx
inet addr:192.168.4.20 Bcast:192.168.4.255 Mask:255.255.255.0
inet6 addr: fe80::xxx:xxxx:xxxx:xxx/64 Scope:Link
UP BROADCAST RUNNING MASTER MULTICAST MTU:1500 Metric:1
RX packets:213882 errors:0 dropped:0 overruns:0 frame:0
TX packets:150722 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:40249275 (38.3 MiB) TX bytes:14092960 (13.4 MiB)
bond0.102 Link encap:Ethernet HWaddr 00:xx:xx:xx:xx:xx
inet addr:10.129.44.10 Bcast:10.129.44.31 Mask:255.255.255.224
inet6 addr: fe80::xxx:xxxx:xxxx:xxx/64 Scope:Link
UP BROADCAST RUNNING MASTER MULTICAST MTU:1500 Metric:1
RX packets:1196 errors:0 dropped:0 overruns:0 frame:0
TX packets:626 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:57170 (55.8 KiB) TX bytes:35370 (34.5 KiB)
eth0 Link encap:Ethernet HWaddr 00:xx:xx:xx:xx:xx
inet6 addr: fe80::xxx:xxxx:xxxx:xxx/64 Scope:Link
UP BROADCAST RUNNING SLAVE MULTICAST MTU:1500 Metric:1
RX packets:745291 errors:0 dropped:0 overruns:154 frame:0
TX packets:483650 errors:0 dropped:0 overruns:0 carrier:150503
collisions:0 txqueuelen:1000
RX bytes:263721522 (251.5 MiB) TX bytes:40600774 (38.7 MiB)
Interrupt:185 Base address:0xec00
eth1 Link encap:Ethernet HWaddr 00:xx:xx:xx:xx:xx
inet6 addr: fe80::xxx:xxxx:xxxx:xxx/64 Scope:Link
UP BROADCAST RUNNING SLAVE MULTICAST MTU:1500 Metric:1
RX packets:772713 errors:0 dropped:0 overruns:0 frame:0
TX packets:553942 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:233022011 (222.2 MiB) TX bytes:42735247 (40.7 MiB)
Interrupt:185
Additional Information
This procedure was tested using Red Hat Enterprise Linux (RHEL) 5.5. The procedure was created using Red Hat Enterprise Linux (RHEL) v5.5 or later.
For additional information on bonding and the different bonding procedures, refer to the following links.
http://unixfoo.blogspot.com/2007/10/yet-to-add.html
http://www.linuxfoundation.org/collaborate/workgroups/networking/bonding