Testing Tasks
Most test procedures were created for previous releases of the software image included with fixed configuration platforms. Some were obtained from other groups. See each individual test procedure document for details. The following tests are run:
Hardware version audits and health check (SunVTS™)
Operation's installation tests (verify operation's procedure for installing the system)
System installation and configuration tests (per the platform installation guide)
Hardware and software acceptance tests (audits)
Basic Solaris operating environment test procedure
VERITAS Test Suite
Basic Oracle test procedure on HA
Basic Oracle test procedure on Oracle9i RAC
TPC test procedure
I/O fault tolerance tests
File system stress tests
VeritasTS test suite
Defect correction verification
Cluster tests, including failover scenarios for Oracle HA and Oracle9i RAC
Shutdown and resumption tests
TPC-C Testing
TPC-C validation is used for two reasons. First, this benchmark test performs both read/write operations. Second, this benchmark puts a good load on the system. This load is used as a background process while faults are injected in the foreground to ensure that the system remains operational and no date is lost or corrupted.
The TPC-C web page is http://www.tpc.org/tpcc.
VERITAS Test Suite
The VERITAS Test Suite performs the most common functions of the VERITAS file system and volume manager applications by exercising as much of the code as possible on different versions of the Solaris operating environment.
The main goal is to ensure that bug fixes or new features added to the kernel in new versions of the Solaris operating environment do not break the VERITAS products. The extent of the testing depends directly on the hardware provided in the test machine. Some tests are only performed if the required hardware is present.
Also, the test suite is self-contained. It can detect and configure the hardware present in the test machine and, with this information, configure the tests to run. Some of the tests are not started if the required hardware is missing. For example, a mirror across disk controllers can only be tested if the test machine has two controllers.
The VERITAS Test Suite provides the necessary functionality to test most of the common functions performed by a system administrator in charge of a Sun system running VERITAS software. This functionality includes:
Creating and removing striped and concatenated volumes
Creating and removing mirrors, RAID5 configurations
Performing re-layouts on a live system
Removing and replacing disks in a running system
Growing and shrinking file systems
Adding and removing disk quotas
Mounting and unmounting the VERITAS File System (VxFS)
Defragmenting file systems
Taking snapshots of the file system
Encapsulating root disk
Creating a second boot disk.
During the test execution, simulations of disk usage and system load are run in parallel, thus adding extra reality to the tests. These simulations include:
File system 80 percent full with one single large file.
Multiple small files and directories.
Number of files growing and shrinking during the test execution.
Simulation of users accessing, reading, and writing files. This functionality of the test suite is not yet incorporated. It will be part of the planned second phase of Sun's work on the VERITAS Test Suite.
Sun ONE Software Testing
This testing validates the Sun ONE products (Sun ONE DS, Sun ONE AS, Sun ONE CS, Sun ONE ES and Sun ONE PS), and looks for any integration problems.
Sun ONE Directory Server (Sun ONE DS)
QA tests the Directory Server (Sun ONE DS) by testing other Sun ONE products that depend on it. These Sun ONE products are: Sun ONE AS and Sun ONE PS.
The Sun ONE DS tests are:
Automatic functional tests (for example, schema creation, update, deletion, connectivity standards, SSL communication, and so forth)
Automatic stress tests
Testing Directory Server by testing other Sun ONE products that depend on it (Sun ONE AS. Sun ONE MS and Sun ONE PS).
Sun ONE Application Server (Sun ONE AS)
The Sun ONE AS uses:
Sun ONE ES 6.0 SP1 as a front end
Sun ONE Directory Server
Oracle9i RAC (currently Oracle9i)
Currently QA performs the following tests on the Sun ONE AS:
Automatic product installation and Solaris kernel tuning
Sample application using Oracle9i
Automatic functional test using a Java™ Technology Bookstore sample application
Automatic stress test with load increase, monitoring system behavior and measuring time until system overloads
Future test software development:
Functional tests
Additional sample applications
Sun ONE Calendar Server (Sun ONE CS)
The currently tested version is 5.1. The Sun ONE CS uses the Sun ONE Directory Server.
Currently QA performs the following tests on the Sun ONE CS:
Automatic installation of the Sun ONE DS and creation of the calsrv Lightweight Directory Application Protocol (LDAP) database via the ldif file
Automatic functional tests as defined within the Sun ONE calsrv test plan (see http://sims.red.Sun ONE.com/calendar/qa/functional_testing.html)
Future test software development:
Administration tests
Automatic user interface tests
Stress tests (48 hour runs)
Sun ONE Enterprise Server (Sun ONE ES)
The currently tested version is 6.0 with Service Pack 2. The Sun ONE ES uses the Sun ONE DS.
Currently QA performs the following tests on the Sun ONE ES:
Automatic product installation and configuration
Automatic stress tests, monitoring system behavior and measuring time until the system overloads
Future test software development:
Automatic functional tests
Sun ONE Portal Server (Sun ONE PS)
The currently tested version is Sun ONE PS 3.0 with Service Pack 4. The Sun ONE PS uses the Sun ONE ES and Sun ONE DS.
Currently QA performs the following tests on the Sun ONE PS:
Automatic product installation
Automatic functional testing of the ipsadmin command line interface, specifically:
Usage string
Domain creation, modification, and deletion using several right and wrong examples
Role creation, modification, and deletion using several right and wrong examples
User creation, modification, and deletion using several right and wrong examples
Future test software development:
Test software for alternative configurations, for example, Open Portal, gateway on a separate machine and so forth.
Automatic stress test
Future Development Plans Across All Test Suites
Integration in a test harness so that tests can be run automatically as part of a test suite covering large software stacks
Failure recovery tests on clusters as appropriate and needed; guided by Sun ONE and customer requirements