- Recommendations for Applying Preferred Practices
- Principals of Mission-Critical Implementations
- Physical Environment
- Internal Network Planning
- External Network Planning
- System Controller Configuration
- Platform and Domain Administration
- Security
- Error Analysis and Diagnosis
- Platform and Domain Configuration
- Dynamic Reconfiguration
- References
- Related Resources
Like this article? We recommend
Like this article? We recommend
Like this article? We recommend
Internal Network Planning
The Sun Fire 15K/12K servers have two internal networks within the frame of each server. These are called the management network (MAN) I1 and I2 networks. The MAN I1 provides private point-to-point network connections between the system controller and each domain. The MAN I2 network provides the point-to-point connections between the primary and secondary system controllers. The overview of the Sun Fire 15K/12K management networks are shown in FIGURE 1 on page 7. The main thing to remember about these networks is that they are dedicated and serve specific functions for internal communications. They are not general purpose networks, therefore, no external packets should be routed across them.
)
FIGURE 1
Management Network Overview
The connections that make up the MAN I1, as built from the factory, are designed to provide maximum availability, redundancy, and security for system controllers to accomplish domain management. This environment has redundant system controllers with dedicated separate point-to-point connections to each system board on the frame. If there is a problem with any of the components that make up the MAN I1 network, the network will automatically switch to the redundant system without any service interruption to domains running applications.
The Sun Fire 15K/12K servers are designed with twenty internal network connections that make up the MAN I1 and I2 networks, as shown in FIGURE 2 on page 8. Eighteen of these connections are connected between system controllers and domain expander boards, and two are dedicated for connections between the system controllers. For a detailed description of the system controller's internal network configurations, see the SMS Administrators Guide, Chapter 6 "Domain Services."
)
FIGURE 2
MAN I1 Network Overview
I1 Network
The MAN I1 network is a private network, not a general purpose network. The network provides functions such as domain consoles, message logging, dynamic reconfiguration, booting from the network, and network time synchronization (NTP). Internal to the dman processes (management network driver) is the ability of the I1 network to detect failures and provide path switch-over in the event of a failure. No packets addressed to one domain can be routed along the network connection between the system controller and another domain. Access to MAN is restricted to the system controller and the domains, and this configuration must not be changed. MAN software also enforces domain isolation of network traffic on the I1 network. Similar software operates on the domain side.
The MAN I1 network is designed to be a completely separate and dedicated point-to-point network for communicating between system controllers and domains on one Sun Fire 15K/12K frame. The I1 network is connected in a point-to-point fashion to network interfaces located on each of the 18 expander I/O slots. The I1 is a 100 megabit, half-duplex configuration that uses internal hubs to connect to each I/O board. Using this design, the number of point-to-point Ethernet links between a system controller and a given domain is based on the number of I/O boards configured in that domain. Each NIC from the system controller connects to a NIC on the I/O board through the hub. The NIC is an internal part of the I/O board, and is not a separate adapter card.
Access to MAN is restricted to the system controller and the domains, and this configuration must not be changed. MAN software also enforces domain isolation of network traffic on the I1 network. Similar software operates on the domain side. For added security purposes the address resolution protocol (ARP) can be disabled on this network to remove the ability of ARP spoofing attacks and other IP attacks on this network. The procedure is well documented in the Sun BluePrints OnLine article "Securing the Sun Fire 15K/12K Domains."
I2 Network
The MAN I2 internal network consists of the two system controllers internal redundant NICs. The MAN I2 network, shown in FIGURE 3, is a private system controller-to-system controller network, which is entirely separate from the MAN I1 network. It is used for heartbeat communication between system controllers to initiate failover, when needed, as well as data synchronization between system controllers.
)
FIGURE 3
I2 Network Detail
The I2 network is configured for failover using the System Management Services software (SMS) software configuration tool. The scman pings the other system controller every 10 seconds and checks for activity every 30 seconds. If no activity is found, the scman initiates a failover. Even if the entire I2 network fails, the failover mechanism can still take place using the high availability static rapid access memory (HASRAM). Because system controllers are some of most important components of the platform, they should be among the first components tested before completing additional installation tasks. This testing can be done manually by forcing a system controller to failover using SMS commands and verifying that the secondary controller has full control of the platform.
The virtual network adapter on the system controller presents itself as a standard network adapter. It can be managed and administered just like any other network adapter (for example, qfe and hme). The usual system administration tools, such as ndd, netstat, and ifconfig, can be used to manage the virtual network adapter. Certain operations with these tools (for example, changing the Ethernet address) should be disallowed, for security reasons.
MAN operates and is managed as an IP network with special characteristics. For example, IP forwarding is disallowed by the MAN software. As such, the MAN operation is the same as any other IP network, with the noted exception documented above.