This chapter is from the book
This chapter is from the book
This chapter is from the book
Scorecards
We use this hierarchy in this text to discuss accountability for getting the right things done at the right time:
- Gate or milestone requirements
- Gate or milestone deliverables
- Tasks
- Tools, methods, and best practices
This four-level flow is well-suited to measurement by using an integrated system of scorecards. A system of scorecards can be designed and linked so that each subordinate level adds summary content up to the next level.
The most basic level of scorecard is filled out by technical team members, who actually use specific sets of tools, methods, and best practices to help complete their within-phase tasks. This is called a tool scorecard. Tool scorecards are easy to fill out and should not take more that 15 minutes or so to complete at the end of a tool application. Typically, they are filled out in collaboration with the team leader for supervisory buy-in. When finished using a certain tool, the person or set of team members who apply that tool should account for the measurable items in Table 2.1.
Table 2.1. Tool Scorecard
|
SS-R&TD Tool |
Quality of Tool Use |
Data Integrity |
Results vs. Requirements |
Average Score |
Data Summary, including Type & Units |
Task Requirement |
Quality of tool use can be scored on a scale of 1–10 based upon the suggested criteria and levels in Table 2.2. You can adjust these ranks as you see fit for your applications.
Table 2.2. Quality of Tools
|
Rank |
Right Tool |
Fullness of Use |
Correct Use |
|
10 |
x |
High |
High |
|
9 |
x |
Medium |
High |
|
8 |
x |
High |
Medium |
|
7 |
x |
Low |
High |
|
6 |
x |
Medium |
Medium |
|
5 |
x |
Low |
Medium |
|
4 |
x |
High |
Low |
|
3 |
x |
Medium |
Low |
|
2 |
x |
Low |
Low |
|
1 |
Wrong tool |
The integrity of the data produced by the tool's use can be scored using the suggested but modifiable ranking structure in Table 2.3.
Table 2.3. Integrity of Data
|
Rank |
Right Type of Data |
Proper Units |
Measurement System Capability |
Percentage of Data Gathered |
|
10 |
Excellent |
Direct |
High |
High % |
|
9 |
Excellent |
Direct |
High |
Medium % |
|
8 |
Excellent |
Direct |
Medium |
High % |
|
7 |
Good |
Close |
High |
High % |
|
6 |
Good |
Close |
Medium |
Medium % |
|
5 |
Good |
Close |
Medium |
Low % |
|
4 |
Weak |
Indirect |
Medium |
High % |
|
3 |
Weak |
Indirect |
Low |
Medium % |
|
2 |
Weak |
Indirect |
Low |
Low % |
|
1 |
Wrong |
Wrong |
None |
— |
You can adjust the nature of the scoring criteria as you see fit for your applications. The key is to make very clear delineation among various levels of measurable fulfillment of the criteria.
The capability of the tool results to fulfill the task requirements is scored with the help of the following criteria:
- 10 = Results deliver all data necessary to completely support the fulfillment or lack of fulfillment of the task requirements.
- 9–8 = Results deliver a major portion of the data necessary to support the fulfillment or lack of fulfillment of the task requirements.
- 7–4 = Results deliver a moderate amount of the data necessary to support the fulfillment or lack of fulfillment of the task requirements.
- 3–1 = Results deliver a very limited amount of the data necessary to support the fulfillment or lack of fulfillment of the task requirements.
Here we want to account for how well our data fulfills original requirements. It is acceptable to find that, through a full set of data, we cannot meet the requirement a task was designed to fulfill. We have to reward technical professionals for doing good work that, unfortunately, tells us the truth about bad results. The intent is to avoid false positives and false negatives when making decisions about a project's viability. This metric helps quantify the underdevelopment of data and facts that can lead to poor decisions.