Securing the Sun Enterprise 10000 System Service Processors

Like this article? We recommend

Backup and Restore Practices for the Enterprise

Learn More Buy

Like this article? We recommend

Like this article? We recommend 

Backup and Restore Practices for the Enterprise

Learn More Buy

Building a Secure Sun Enterprise 10000 System

Building a secure system requires that entry points onto the system be limited and restricted, in addition to limiting how authorized users obtain privileges.

To effectively secure an SSP, changes are required to the Solaris OE software running on the SSP and, to a lesser degree, the Sun Enterprise 10000 system domains.

Properly securing the primary and backup SSP on a Sun Enterprise 10000 system requires the following:

• "Modifying Network Topology"

• "Installing Main SSP Detection Script"

• "Adding Security Software"

• "Creating Domain Administrator Accounts"

Although optional, for those who are administrating sites requiring the most secure configurations, we recommend that you add a host-based firewall on both SSPs. Refer to "Adding Host-Based Firewalls" on page 30.

By performing these procedures there is considerable improvement in the security and domain separation of the Solaris OE images running on SSPs and domains.

Modifying Network Topology

Modify the network topology of the SSP management network (recommended and documented in the Sun Enterprise 10000 documentation) to provide separation of each domain to SSP connection.

1. Isolate domains by implementing a separate and private network connection between the SSP and each domain.

   By providing separate networks for each domain, you make it impossible for a rouge domain user to use the SSP management network to attack other domains.

   NOTE

   If some of the domains are in the same security zone and connected on the public-side network already, then you might not need to separate those domains. For example, if two of six domains in a Sun Enterprise 10000 system are application servers providing the same services on the same network and managed by the same organization, then these systems have the same security exposures and are in the same security zone. You could place these two domains on the same private SSP management network—in a secured configuration—without compromising the security of the environment.

2. Repeat Step 1 for all domains that are present in the Sun Enterprise 10000 system.

By implementing these recommendations there is no network connection between multiple domains. Correspondingly, the weakest link is now the SSP and its Solaris OE configuration. (Recommendations on how to mitigate these risks are described later in this article.)

The number of security zones and separate SSP management networks required can impact the hardware used for the SSPs. For example, an Ultra 5 system has three PCI slots: one slot is typically used for a monitor and two slots are available for Sun Quad FastEthernet_ cards, which amounts to nine network ports. These ports can be configured as follows:

• two ports for control boards

• one port for a production network connection

• six ports for private SSP management networks

A Sun Enterprise 250 system has an additional PCI slot, supporting four more private SSP management networks than the Ultra 5 system.

The following sample configuration isolates each domain onto a separate network. This configuration has: two domains, domain_a and domain_b; two SSPs, ssp_a and ssp_b, and two control boards, control_board_0 and control_board_1. Each domain and SSP has one Sun Quad FastEthernet card. The networks connected to the Sun Quad FastEthernet ports are listed next to each component.

The following are the network segments for our sample configuration:

• control board 0 mngt network - 192.168.151.0

• control board 1 mngt network - 192.168.152.0

• domain_a ssp mngt network - 192.168.153.0

• domain_b ssp mngt network - 192.168.154.0

These four network segments all use a 24-bit netmask that has an entire Class C IP address space in it. You can subnet the SSP-domain management networks into parts of Class C networks. You must not subnet the SSP domain on the control board networks; subnetting to control board networks is not supported

The following figure illustrates our configuration.

FIGURE 1 Modified Network Topology Sample

Installing Main SSP Detection Script

This script detects the main SSP in a redundant SSP environment. Use it for configurations where a floating SSP name and IP are either not valid or not supported.

The script is required for SSP failover to function properly on SSP versions 3.4 and 3.5. It is not required on SSP version 3.3. Before running the script, some simple preparation work needs to be done on the domains; see the comment section of the script for details.

1. Download the script from the Sun BluePrints Tools web site at:

   http://www.sun.com/blueprints/tools

2. Install the script on each domain of a hardened Sun Enterprise 10000 system.

   NOTE

   This script poses negligible impact on the domain running it.

3. Before running the script, manually edit the file, /etc/ssphostname, to contain the resolvable uname of the main SSP.

4. Set up a cron job to run the script periodically.

   We recommend running this script every 3 minutes.

5. Refer to the crontab(1) manual page for additional information on how to create crontab entries.

The script runs as a cron job. No argument is needed. The following is a portion of a sample root crontab setting:

    0 * * * * /find_main_ssp.ksh > /dev/null 2>&1
    3 * * * * /find_main_ssp.ksh > /dev/null 2>&1
    6 * * * * /find_main_ssp.ksh > /dev/null 2>&1

When using this script with host-based firewalls, the SSPs may generate error messages to SYSLOG. We encountered these error messages when testing SunScreen 3.1 software rulesets as the SSP firewall. The SYSLOG errors generated are similar to the following:

Jan 7 14:22:34 xf4-ssp2 SSP Startup : [ID 702911 local0.info] Error: Failed to 
receive acknowledgement from cb xf4-cb0
Jan 7 14:22:34 xf4-ssp2 SSP Startup : [ID 702911 local0.info] Error: Failed to 
receive acknowledgement from cb xf4-cb1

If you encounter these error messages, they can be ignored. The root cause of the errors is the SunScreen 3.1 software; there is no apparent failure of the SSP or control board network.

Adding Security Software

The next stage in hardening an SSP requires downloading and installing additional software security packages. This section covers the following tasks:

• "Install Toolkit Software"

• "Download Recommended Patch Software"

• "Download FixModes Software"

• "Download OpenSSH Software"

• "Download MD5 Software"

Install Toolkit Software

The Toolkit version 0.3.5 software must be downloaded first, then installed on the SSP. Later, you'll use the Toolkit to automate installing other security software and implementing the Solaris OE modifications for hardening a Sun Enterprise 10000 system.

The primary function of the Toolkit is to automate and simplify building secured Solaris OE systems based on the recommendations contained in this and other security-related Sun BluePrints OnLine articles.

1. Download the source file (SUNWjass-0.3.5.pkg.Z) from the following web site:

   http://www.sun.com/security/jass

2. Extract the source file into a directory on the server using the uncompress command:

   # uncompress SUNWjass-0.3.5.pkg.Z

   
   

3. Install the Toolkit onto the server using the pkgadd command:

   # pkgadd -d SUNWjass-0.3.5.pkg SUNWjass

   
   

Executing this command creates the SUNWjass subdirectory in /opt. This subdirectory contains all Toolkit directories and associated files. The script make-pkg—included in Toolkit releases since version 0.3—allows administrators to create custom packages using a different installation directory.

Download Recommended Patch Software

Patches are regularly released by Sun to provide Solaris OE fixes for performance, stability, functionality, and security. It is critical to the security of a system that the most up-to-date patch is installed. To ensure that the latest Solaris OE Recommended and Security Patch is installed on the SSP, this section describes how to download the latest patch cluster.

Downloading the latest patch cluster does not require a SunSolve^SM program support contract.

1. Download the latest patch from the SunSolve Online_ web site at:

   http://sunsolve.sun.com

2. Click on the "Patches" link, at the top of the left navigation bar.

3. Select the appropriate Solaris OE version in the "Recommended Solaris Patch Clusters" box.

   In our example, we select Solaris 8 OE.

4. Select the best download option, either HTTP or FTP, with the associated radio button, then click "Go."

   A "Save As" dialog box is displayed in your browser window.

5. Save the file locally.

6. Move the file securely to the SSP with the ftp command.

7. Move the file to the /opt/SUNWjass/Patches directory and uncompress it as follows:

   # cd /opt/SUNWjass/Patches # mv /var/tmp/8_Recommended.zip . # unzip 8_Recommended.zip Archive: 8_Recommended.zip creating: 8_Recommended/ inflating: 8_Recommended/CLUSTER_README inflating: 8_Recommended/copyright inflating: 8_Recommended/install_cluster [. . .]

   
   

   Later, using the Toolkit, you'll install the patch after downloading all the other security packages.

   NOTE

   If you do not place the Recommended and Security Patches software into the /opt/SUNWjass/Patches directory, a warning message displays when you execute the Toolkit.

Download FixModes Software

FixModes is a software package that tightens the default Solaris OE directory and file permissions. Tightening these permissions can significantly improve overall security of the SSP. More restrictive permissions make it even more difficult for malicious users to gain privileges on a system.

1. Download the FixModes pre-compiled binaries from:

   http://www.Sun.COM/blueprints/tools/FixModes_license.html

   The FixModes software is distributed as a precompiled and compressed tar file formatted for SPARC-based systems. The file name is FixModes.tar.Z.

2. Save the file, FixModes.tar.Z, in the Solaris Security Toolkit Packages directory in /opt/SUNWjass/Packages.

   CAUTION

   Leave the file in its compressed state.

   Later, using the Toolkit, you'll install the FixModes software after downloading all the other security packages.

Download OpenSSH Software

In any secured environment, the use of encryption in combination with strong authentication is required to protect user-interactive sessions. At a minimum, user interactive sessions must be encrypted.

The tool most commonly used to implement encryption Secure Shell (SSH) software, whether a commercial or open source (freeware) version. To implement all the security modifications performed by the Toolkit and recommended in this article, you must implement a SSH software product.

The Toolkit disables all non-encrypted user-interactive services and daemons on the system, in particular daemons such as in.rshd, in.telnetd, and in.ftpd. Access to the system can be gained with SSH similarly to what is provided by RSH, TELNET, and FTP.

• Obtain the following online article and use the instructions in the article for downloading the software.

  A Sun BluePrints OnLine article about how to compile and deploy OpenSSH titled: Building and Deploying OpenSSH on the Solaris Operating Environment is available at:

  http://www.sun.com/blueprints/0701/openSSH.pdf

  Later, using the Toolkit, you'll install the OpenSSH software after downloading all the other security packages.

Download MD5 Software

The MD5 software validates MD5 digital fingerprints on the SSP. Validating the integrity of Solaris OE binaries provides a robust mechanism to detect system binaries that are altered or trojaned (hidden inside something that appears safe) by unauthorized users. By modifying system binaries, attackers provide themselves with back-door access onto a system; they hide their presence and cause systems to operate in unstable manners.

To install the MD5 program (Intel and SPARC Architectures), follow these steps:

1. Download the MD5 binaries from http://www.sun.com/blueprints/tools/md5_license.html

   The MD5 programs are distributed as a compressed tar file.

2. Save the downloaded file, md5.tar.Z, to the Solaris Security Toolkit Packages directory in /opt/SUNWjass/Packages

   NOTE

   Do not uncompress the tar archive.

   Later, when you execute the Solaris Security Toolkit software, the MD5 software is installed.

   After the MD5 binaries are installed, you can use them to verify the integrity of executables on the system through the Solaris Fingerprint Database. More information on the Solaris Fingerprint Database is available in the Sun BluePrints OnLine article titled The Solaris™ Fingerprint Database - A Security Tool for Solaris Software and Files:

   http://www.sun.com/blueprints/0501/Fingerprint.pdf

3. (Optional) Download and install Solaris Fingerprint Database Companion and Solaris Fingerprint Database Sidekick software from the SunSolve Online web site at:

   http://sunsolve.sun.com

We strongly recommend that you install these optional tools, and use them with the MD5 software. These tools simplify the process of validating system binaries against the database of MD5 checksums. Use these tools frequently to validate the integrity of the Solaris OE binaries and files on the main and spare SSPs.

These tools are described in the same The Solaris™ Fingerprint Database - A Security Tool for Solaris Software and Files article mentioned previously.

Install Downloaded Software and Implement Modifications

The Solaris Security Toolkit version 0.3.5 provides a driver (starfire_ssp-secure.driver) for automating the installation of security software and Solaris OE modifications. The driver for the Sun Enterprise 10000 SSPs performs the following tasks:

• installs and executes the FixModes software to tighten filesystem permission

• installs the MD5 software

• installs the Recommended and Security Patch software

• implements almost 100 Solaris OE security modifications for the Sun Enterprise 10000 system

The Toolkit focuses on Solaris OE security modifications to harden and minimize a system. Hardening means modifying Solaris OE configurations to improve the security of the system. Minimization means removing unnecessary Solaris OE packages from the system, thus reducing the components that must be patched and made secure. Reducing components potentially reduces entry points to an intruder. However, minimization is not addressed, recommended, or supported on Sun Enterprise 10000 SSPs at this time.

• Execute the starfire_ssp-secure.driver script as follows:

  # cd /opt/SUNWjass # ./jass-execute -d starfire_ssp-secure.driver ./jass-execute: NOTICE: Executing driver, starfire_ssp-secure.driver ============================================================ starfire_ssp-secure.driver: Driver started. ============================================================ [...]

  
  

To view the contents of the driver file and obtain information about the Solaris OE modifications, refer to the Solaris Security Toolkit documentation available either in the /opt/SUNWjass/Documentation directory or through the web at:

For information about other scripts in the Toolkit, refer to the Sun BluePrints OnLine article titled Solaris Security Toolkit Internals: Updated for Version 0.3.

Each Solaris Security Toolkit run creates a run directory in /var/opt/SUNWjass/run. The names of these directories are based on the date and time the run is initiated. In addition to displaying the output to the console, the Toolkit creates a log file in the /var/opt/SUNWjass/run directory.

The files stored in the /var/opt/SUNWjass/run directory track modifications performed on the system and enable the jass-execute undo feature. You can undo arun, or series of runs, with the jass-execute -u command. For example, on a system where two separate Toolkit runs are performed, you could undo them by using the following command:

# pwd
/opt/SUNWjass
# ./jass-execute -u
Please select from one of these backups to restore to
1. September 25, 2001 at 06:28:12 (/var/opt/SUNWjass/run/20010925062812)
2. April 10, 2001 at 19:04:36 (/var/opt/SUNWjass/run/20010410190436)
3. Restore from all of them
Choice? 3
./jass-execute: NOTICE: Restoring to previous run 
//var/opt/SUNWjass/run/20010410190436

============================================================
undo.driver: Driver started.
============================================================
[...]

Refer to the Toolkit documentation for details on the capabilities and options available in the jass-execute command.

Now that all the software is installed—including an alternative administrator access mechanism with either OpenSSH or SSH—the Solaris OE image running on the Sun Enterprise 10000 SSP can be secured.

Creating Domain Administrator Accounts

The default SSP configuration provides SSP administrators with root privileges to all Sun Enterprise 10000 domains through the SSP administration role. In secured environments, and particularly in those organizations where different administrators are responsible for different domains, it is beneficial to have a separate and more restrictive account. This account is referred to as the domain administrator. Create domain administrator accounts to establish restricted domain administrator accounts on each SSP.

Domain administrators use these accounts to access the console and any other SSP domain-specific functionality for a domain by logging into the appropriate account as a domain administrator.

1. For each domain, create a shell script called domain_console in the directory:

   /var/opt/SUNWssp/adm/<domain_a>

   where domain_a is the name of domain running on the Sun Enterprise 10000 system.

2. Create a shell script for each domain that supports the restricted shells.

   1. Assign permission mode 0555.

   2. Designate ownership as root of group root.

   3. Include the following in the script:

      #!/bin/sh setenv SUNW_HOSTNAME <domain_a> source /export/home/ssp/.cshrc /opt/SUNWssp/bin/netcon_wrapper

      
      

3. Set the permissions and user/group ownerships with the following command:

   # chown root:root /var/opt/SUNWssp/adm/<domain_a> # chmod 0555 /var/opt/SUNWssp/adm/<domain_a>

   
   

4. For each domain, create a domain administrator account with the following command:

   # useradd -m <domadma> -u 12 -g 10 -s /var/opt/SUNWssp/adm/<domain_a>

   
   

   where domadma is the account name.

5. For each account, first set, then expire, the password using the following commands:

   # passwd <domain_a> New password: Re-enter new password: passwd (SYSTEM): passwd successfully changed for root # passwd -f <domadma>

   
   

   This step ensures that the administrator has to enter a new password immediately after logging into the account for the first time.

   NOTE

   The examples use domain_a as the domain name and domadma as the account name. Use unique user ids (uid) for each account as well.

6. Repeat Step 1 through Step 5 for each domain on the Sun Enterprise 10000 system.

Adding Host-Based Firewalls

For some environments, you might want to implement host-based firewalls on the Sun Enterprise 10000 SSPs. Host-based firewalls control a systems network access to protect against malicious misuse. These firewalls provide another layer of protection for the SSP against network-based attacks.

Based on the recommendations made up to this point—network separation, addition of security tools, and hardening of the SSP—the SSP management environment is now considerably more secure than the default configuration and any previously available and supported configuration.

For customers requiring the most-secure and best-instrumented configuration, we recommend installing and implementing host-based firewall software on the SSPs. The goal of this recommendation is to provide additional controls to the services that must be run on the SSPs.

The following information provides an example of how to install a host-based firewall on the SSP. Choose a firewall software product that best fits your environment. Additionally, adapt the rule sets to fit the firewall product you choose.

Using SunScreen Software Version 3.1

For example purposes, we test a SunScreen 3.1 software configuration and recommend rulesets. For more information about SunScreen 3.1 software—including debugging tips and how to manage a firewall from its command line interface—refer to the Sun BluePrints Securing Systems with Host-Based Firewalls - Implemented With SunScreen Lite 3.1 Software at:

Our configuration is based on a two-domain Sun Enterprise 10000 system. The firewall software allows traffic to flow freely between the SSPs and the domains on any management segment.

Only certain traffic is allowed to originate from the domain destined for the SSPs. This traffic is SYSLOG and the failover check traffic. All other traffic from the domain to the SSP is not permitted—including administrative access to SSH on the SSP.

The only access to the Secure Shell daemons running on the SSPs is over the production network segments connected to the SSPs. Secure Shell is the only one permitted to access the SSP over these production network segments. No other protocols may access the SSPs. Of course, the SSPs can request information as appropriate.

Establishing Rulesets

In our sample configuration, we propose rulesets that are point-to-point for all authorized systems. Because the rulesets explicitly define the source and destination of each permitted data stream, unauthorized IP addresses are not able to communicate with any of the authorized devices.

The following table lists rulesets that correspond to ssp_a (shown in FIGURE 1). Modifications are required before deploying them on ssp_b.

TABLE 1 Rulesets for ssp_a Domain

Denying Protocols and Services on Management Networks

The proposed firewall rulesets deny many of the protocols that may have been used to manage SSPs, including some domain installation capabilities from the SSPs.

The following services are denied: TELNET, FTP, remote X display, R* services, and all user-interactive administrative type access over the SSP management networks.

Denying these services enforces domain separation in the architecture. The only user-interactive protocol permitted to access the SSPs is Secure Shell, from the production network connected to the SSPs.

Although the SSPs are permitted to send any protocol to the domains, some services require an additional firewall rule such as FTP. For FTP to function properly, the domains must be allowed to open a high-port connection back to the SSPs. This connection represents a serious risk to the security of the SSP management network and is strongly discouraged. We recommend using alternatives such as Secure Shells version 2, FTP mode, or FTP in PASV mode.

We disable the use of JumpStart_ software from the SSPs to install an OS on a domain.

Internet Control Message Protocols (ICMP) messages are not permitted within the management network, which means that commands such as ping would not be allowed. If you need ping functionality within the environment, enable it by adding the following rules:

Simple Mail Transport Protocol (SMTP) on the management network, including the SSPs ability to receive SMTP messages, is disabled.

The use of traceroute on the SSP management network is disabled. Normally it would not be expected that this protocol would be used on the SSP management network. Traceroute still works when directed against the production networks attached to the SSPs and domains.

The use of Network Time Protocol (NTP) over the SSP management networks is disabled. Instead of using the SSP as the NTP master for the domains, we recommend that the domains and the SSP function as an NTP client of a separate NTP time server—with the appropriate stratum classification. Additional information on NTP and how it can be securely configured is available in Sun BluePrints OnLine articles (refer to "Bibliography and Recommended Reading" on page 40).

The use of X-based window managers on the SSPs is disabled. When X-based applications must be run on the SSPs, we strongly recommend that you use the Secure Shell to tunnel the X traffic back to the local desktop. The capability to tunnel is available in UNIX_ platform based SSH clients.

The use of Sun Management Center (Sun MC) was also disabled. If Sun MC software is to be used add the following rules functionality within the environment, enable it by allowing UDP traffic to ports 166 on the SSPs and 1161 on the domains from the Sun MC server. Port 1161 is used on the domains as the default port, 161, is already in use.