- Microsoft Test Manager
- Test Plans
- Testing Configurations
- Assigning Testers
- Test Case Planning Workflow
- Common Scenarios
- Summary
This chapter is from the book
This chapter is from the book
This chapter is from the book
Testing Configurations
Testing configurations are configurable and can have an impact on the number of tests that need to be executed (mentioned previously). Test configurations specify any particular piece of information needed to ensure that your software is tested against all possible configuration options users could have on their machine.
The most typical example is using different browsers to ensure the rendering works correctly. Added to that may be the operating system those browsers run on. The two default configuration options are Operating System and Browser; to this you can add other things such as a Silverlight version or a particular piece of hardware, such as a webcam.
The biggest benefit to using test configurations is reporting results. All your test results can be broken down into configurations. In addition you have to write the Test Cases only one time, but this presents other issues, such as that the actions you take on one configuration may not be valid on another configuration. In some cases the differences may be so great it doesn't make sense to use the same Test Case. Consider these items when deciding on how to use test configurations.
Managing Test Configurations
You can access the Test Configuration Manager in two ways. The first is to go to Testing Center, Plan, Properties and select the drop-down arrow next to the configuration; then click Manage. The easier way is to go to Testing Center, Organize, Test Configuration Manager. This brings up the screen shown in Figure 3-12.
)
Figure 3-12 Test Configuration Manager
The Manage Configuration Variables option enables to you create new configuration categories. You can also add new values to an existing configuration variable.
Exercise 3-3. Adding a New Configuration Variable
To add a new configuration variable, follow these steps:
- Click Manage Configuration Variables.
- Click New Configuration Variable.
- Enter Silverlight Version for the name.
- Enter Default Silverlight Versions for the description.
- In Allowed Values, enter the following (shown in Figure 3-13): 1, 2, 3, and 4.
Figure 3-13 Silverlight Version Configuration Variable
- Click Save Configuration Variables.
)
The variables themselves cannot be used directly. You need to create an actual configuration composed of one or more configuration variables.
Exercise 3-4. Create a New Test Configuration
To create a new test configuration, follow these steps:
- Click New from the Test Configuration Manager.
- Enter the Name as Vista, IE7, and Silverlight 2.
- (Optional) Enter an appropriate description.
- Select the Add button, and notice that you can add configuration variables from the three existing categories. You can add only one variable from each category in a given configuration.
- Click Operating System, and select Vista for the value.
- Click Add, Browser, and select Internet Explorer 7.0 for the value.
- Click Add, Silverlight Version, and select 2 for the value.
- Click Save and Close.
You now have a new test configuration that can be assigned to plans. You can also delete test configurations if they are not being used or have not been used by previous Test Plans. If you try to delete a test configuration that is in use, you are prompted to set it to Inactive instead.
Assigning Test Configurations
To assign configurations to Test Cases, you have a few options. The first is to go to the Properties page of the plan and change the configuration. This can instantly apply the changes to all Test Cases contained within the plan and any Test Cases you add to the plan at a later date. The next option is to change the Default configurations from the Plan Contents tab (see Figure 3-9 just below the Test Suite name in the Test Suite Details pane). To make a change here, uncheck the Use Configurations from Parent Test Suite option, and check any additional test configurations you want to include. Changes you make here apply to the individual suite and any suites contained in the currently selected suite. For example, looking at Figure 3-9, if you select the Iteration 1 node and change the default configurations, the new set of configurations apply to all Test Suites in Iteration 1. If, however, you change the default configurations at Test Suite 1 (log onto the blog engine) the change applies only to this suite. Changing the configuration here is not automatically reflected on the Test tab. To illustrate this, after making one of the previous changes, select the Test tab; notice the same number of tests to be run as there are Test Cases. You see how to change this in a minute.
){kind=link}
Another option is to assign test configurations at the suite level for existing Test Cases. To do this, right-click the suite in the left pane of the Contents tab, and choose Select Test Configurations for all Tests. This shows the screen in Figure 3-14.
)
Figure 3-14 Assign test configurations to specific tests
One option available to you is the Reset Defaults option. If you have previously changed the default configuration at the Suite level and want to apply it to all existing Test Cases, selecting the Reset Defaults button will do this for you. (As shown in Figure 3-14, pressing this button automatically selects both configurations for all tests listed.)
After assigning one or more Test Cases to different configurations and applying the changes, you return to the Plan Contents page. The one apparent difference is the configurations column now has a value greater than 1. This column notes how many configurations are assigned to a given Test Case; you might see the Tester for a Test Case listed as Multiple. (You revisit this when assigning testers to Test Cases is discussed.) You see the changes when you select the Test tab. You can execute two more tests than there are Test Cases; these additional tests have different configurations, as shown in Figure 3-15.
)
Figure 3-15 Testing multiple configurations
An additional option for setting test configurations is to select one or more tests and click the Configurations button. This enables you to set configurations just for the specific tests selected.
So far you have seen how to set test configurations for a plan. Options can be set at the Plan, Suite, and Test Case level, and generally they cascade down. The next step is to assign and manage testers in the context of the plan.