Like this article? We recommend
Like this article? We recommend
Like this article? We recommend
Configuration
The following sections describe the steps required to configure an IP network multipathing group with two active interfaces. These sections use the specifications defined in TABLE 1 and cover the following topics:
Enabling unique Ethernet MAC addresses.
Defining TCP/IP addresses.
Disabling routing (if applicable).
Configuring the network interfaces.
Enabling Unique Ethernet MAC Addresses
To support the setting of a unique MAC address automatically, the adapter must have a MAC address stored in its Fcode PROM. Not all network adapters (particularly on-board adapters) have this feature. In this situation, the MAC addresses must be set manually.
The definitive way to determine if a MAC address will be set automatically is to try it and check the outcome. Follow the procedure below:
Set the EEPROM variable local-mac-address? to true as follows:
# eeprom local-mac-address?=true
NOTE
With the preceding command, the server must be rebooted for the change to take effect.
If working at the OpenBoot_ Prompt (OBP), enter the following command:
ok setenv local-mac-address? True
Boot the server and plumb each of the interfaces in the IP network multipathing group by issuing the following command for each interface:
# ifconfig <interface> plumb
Execute the following command to determine the MAC address for each interface. The following codebox also includes an example of the output:
# ifconfig -a hme0: flags=1000843<BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 2 inet 0.0.0.0 netmask 0 ether 8:0:20:f7:c3:f hme1: flags=1000842<BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 8 inet 0.0.0.0 netmask 0 ether 8:0:20:f7:c3:fThe ether line shows the MAC address. If any interfaces are showing identical MAC addresses and are connected to the same subnet, then set the MAC addresses manually with the following command:
# ifconfig <interface> ether <MAC address>
It is suggested that the last three octets of the MAC address be made the same as the last three octets of the test address. This address convention helps make the MAC address unique within a subnet—for example, 192.168.49.106 – 08:00:20:a8:31:6a. Although the chance of a MAC address conflict is extremely small, it can be checked for by using the snoop command to search for the chosen MAC address, while using the ping command to verify connection to the broadcast address of the subnet.
NOTE
If a MAC address is to be set manually, insert the ether parameter when the interface is configured in the following sections.
Additionally, it is possible to determine whether the Fcode PROM of some Ethernet adapters has a local MAC address with the following command:
# prtconf -vp | grep local-mac-address
Therefore, if local-mac-addresses are displayed for all interfaces, except the on-board adapter, then the setting of local-mac-address provides unique addresses. Because this command has not been tested against all adapters, it is not a recommended method, however, if used diligently it may be helpful.
Defining TCP/IP Addresses
The interface data addresses and IPMP test addresses should be added to /etc/hosts file. It is assumed that no existing network configuration information exists on the server. For example (using the example addresses defined in TABLE 1), the following lines should be included in /etc/hosts file:
# # # IP Network Multipathing Group - Production # 192.168.49.42 camelot # Data Address 192.168.49.7 camelot-1 # Additional Data Address 192.168.49.105 camelot-qfe0 # Test Address for qfe0 192.168.49.106 camelot-qfe4 # Test Address for qfe4
The netmask setting for the subnet should be set in the /etc/netmasks file. Using the example addresses defined in TABLE 1, the following line should be included in the /etc/netmasks file:
192.168.49.0 255.255.255.0
Disabling Routing
If the node is not intended to perform network routing, enter the following command:
# touch /etc/notrouter
NOTE
The server must be rebooted for this change to take effect unless the IP driver parameter, ip_forwarding is set to zero using the ndd /dev/ip command.
Configuring the Network Interfaces
After the network address information has been added to the /etc/hosts and /etc/netmasks files, the network interfaces can be configured. The following commands perform the configuration dynamically. In addition, the following paragraphs describe how to create a permanent configuration. In the following examples, the addresses defined in TABLE 1 are used.
Create the network interfaces:
# ifconfig qfe0 plumb # ifconfig qfe4 plumb
Create an IP network multipathing group named production, which consists of network interfaces qfe0 and qfe4:
# ifconfig qfe0 group production # ifconfig qfe4 group production
After executing the commands above, the following syslog messages may be issued. The messages simply warn that failures cannot be detected, until test addresses are established on the interfaces.
May 21 14:14:15 camelot in.mpathd[430]: Failures cannot be detected on qfe0 as no IFF_NOFAILOVER address is available May 21 14:14:15 camelot in.mpathd[430]: Failures cannot be detected on qfe4 as no IFF_NOFAILOVER address is available
Use the following commands to create an address on each interface for data transmission and add the failover flag. The failover flag allows the address to migrate if an interface failure is detected.
# ifconfig qfe0 camelot netmask + broadcast + failover up # ifconfig qfe4 camelot-1 netmask + broadcast + failover up
The following commands configure a test address on each network interface; these addresses are used by mpathd to detect interface failures. Test addresses should not be used by host applications for data communication; hence, they should be marked with the deprecated flag. In addition, test addresses must not failover and should also be marked with the -failover flag. It is the presence of this -failover flag that causes in.mpathd to use the address as a test address, because it is tied to the interface. If IPMP was used to detect failures and not to provide failover, these test addresses could be omitted and the -failover flag could be used to define the data addresses.
# ifconfig qfe0 addif camelot-qfe0 netmask + broadcast + \ -failover deprecated up # ifconfig qfe4 addif camelot-qfe4 netmask + broadcast + \ -failover deprecated up
To enable the interface configuration to persist after a reboot, the files hostname.qfe0 and hostname.qfe4 must be created in the /etc directory. The files should look as follows:
For hostname.qfe0:
camelot netmask + broadcast + failover up \ group production \ addif camelot-qfe0 netmask + broadcast + \ deprecated -failover up
For hostname.qfe4:
camelot-1 netmask + broadcast + failover up \ group production \ addif camelot-qfe4 netmask + broadcast + \ deprecated -failover up
NOTE
If the MAC address of an interface must be set manually do this as a separate RC script, the ether parameter cannot be included in the above files.
Bug ID: 4710499, "Synopsis: rpc.bootparamd cannot send out reply with physical interface flag DEPRECATED
The advice in the original article was to place the test addresses on the physical interfaces. That is, on qfe0 and qfe4 and not on the logical interfaces qfe0:1 and qfe4:1 as is now described. The reasoning behind tying the test addresses to the physical interface was as follows.
It simplifies management; the test addresses will not failover, and therefore, they can always be guaranteed to be the first address on an interface. An interface may have more than one data address and this allows for a company standard. For example, test addresses are always found on the physical interface. The logical interfaces can be unplumbed (for example, ifconfig qfe0:1 unplumb), whereas unplumbing the physical interface also removes all logical addresses. While data addresses may need to be unplumbed, the test addresses should remain for as long as an interface is wired to a particular subnet.
Unfortunately, following the above, original, advice of placing a test address on the physical interface reveals Bug ID: 4710499, see http://sunsolve.sun.com/ in a small number of circumstances. Namely, where the server is being used as a bootp server. Typically, this is when the machine is acting as a JumpStart_ server. The client sends out a bootp request, but never receives a reply. There is also a problem where network file system (NFS) is run over user datagram protocol (UDP). However, given that resilience is unlikely to figure in this configuration it would be anticipated this will be extremely rare case.
ping
If for any reason, a message similar to the following is generated from using the ping command to verify connection to one of the data addresses, consult "Addendum – The Problem With ping" on page 17.
ICMP Protocol Unreachable from gateway camelot (192.168.49.42)for icmp from
clusterclient00 (192.168.49.4) to camelot (192.168.49.42)