- Key Stakeholders
- Business Objectives
- Measuring Success
- Governance
- Content Management
- Rollout
- Key Points
This chapter is from the book
This chapter is from the book
This chapter is from the book
Measuring Success
Even if you have carefully defined and documented your business objectives and engaged all levels of stakeholders in the solution definition and strategy process, the time will come when you will need to assess the value of your organization's MOSS investment. This process may have already started as part of the justification to build out the MOSS environment in the first place. Once the MOSS investment has been approved, however, a metrics program should be established so that you can demonstrate that the solution is meeting the business objectives. The metrics program should include more than simply capturing usage statistics, though usage statistics are one important proxy measurement for MOSS value. A good metrics program includes both qualitative and quantitative measures that tie the MOSS solution as directly as possible to business success. The metrics program also needs to start with established baselines for all measures. Portals and collaboration tools have started to become a more common part of IT infrastructure—sometimes even considered an infrastructure application, like email. However, despite increasingly wide acceptance, MOSS solution builders should be keenly aware of the likelihood that management will want to understand how MOSS is delivering against expectations. Having a metrics program in place provides an opportunity to monitor the solution deployment to ensure that usage is optimized. It also provides a basis for justifying enhancements in the future.
If you worked with your key stakeholders to document business objectives, you've already accomplished the first important part of your metrics program—understanding the business and stakeholder objectives. The next key step is to identify a potential set of qualitative and quantitative measures that can be used to identify whether and how MOSS is addressing these objectives. Qualitative metrics can be "discovered" from user feedback during quality assurance testing and initial training and on an ongoing basis. Solicit users' "stories" or anecdotes that describe how using the portal and/or collaboration tools (finding a person with an unknown skill, rapidly accessing previously difficult-to-find information, and so on) helped contribute to increased revenue or profitability, increased client satisfaction, or other metrics that are already reflected in the key performance measures for the organization. The following sidebar provides an example of an anecdote describing the value of a knowledge management portal for a consulting firm. Note that while the story is an example of a qualitative measure, it is a good story because it includes a quantitative value estimate that is defined by the "actor" in the story. With a quantitative "punch line," this story provides a much clearer demonstration of value than it would if no bottom line were presented. You should try to ensure that all of your anecdotal evidence of value includes a realistic value estimate.
Quantitative metrics can often be obtained from usage analysis reports, and while these metrics may not be a direct measure of value, certain measures can be used as a proxy for value. Table 1.1 suggests some possible quantitative and qualitative metrics associated with several of the business objectives described earlier.
Table 1.1. Suggested MOSS Metrics
|
Objective |
Possible Measure |
Capture Frequency |
Issues and Challenges |
Target |
|
Maximize the reuse of best practices across the enterprise, enabling the organization to replicate successful business practices in all geographies. |
Quantitative: Number of downloads of best practice or reusable assets. Qualitative: Usage anecdotes where users can describe in quantitative terms how a MOSS asset that they reused contributed to business objectives. |
Monthly |
Frequent downloads are a proxy for content value, indicating that the content is delivering value to users. Gathering anecdotes is a labor-intensive process and may require some creativity to obtain. You may want to consider a success story contest (with prizes) to get MOSS users to share high-quality success stories. |
Look for an upward trend in the number of downloads for new content or new portals. Look for steady state activity in more mature environments. Targets should be set based on the maturity of the solution and the strategic importance of the content. Targets for success stories might be based on total "value" represented in the stories collected and/or the number of stories documented. |
|
Improve time to market for proposals and contracts. |
Quantitative: Average proposal or contract development time. |
Ideally captured for each proposal or contract and then compiled (averages) on a semiannual or annual basis. |
This measure will be easiest to capture if it is already a key performance measure for the enterprise. |
Trend downward from baseline. Target might also be a specific percentage of time reduction. |
|
Reduce training costs for enterprise applications. |
Quantitative: Total training costs for enterprise applications. |
Annual |
Some organizations justify their MOSS investment solely on the reduction in training costs. The assumption is that most users are not "power users" of enterprise applications. Instead of investing in full training programs for these users, you only need to train them in the use of the portal, not each enterprise application. |
Percentage or absolute reduction in training expenses for enterprise applications. |
|
Provide an organized "one-stop shop" for information for MOSS users that helps users reduce information overload. |
Qualitative: Usage anecdotes where users can describe in quantitative terms how using MOSS has improved their productivity. |
Monthly |
Gathering anecdotes is a labor-intensive process and may require some creativity to obtain. Consider using the built-in MOSS survey capability. |
Targets for success stories might be based on total "value" represented in the stories collected or the number of stories documented. |
Your organization will most likely have a preferred format for documenting metrics. This might be a balanced scorecard, documented key performance indicators, or a simple "report card." Your goal will be to ensure that you are capturing metrics that, as well as possible, directly relate MOSS's value to the business. Keep in mind that you also want to be sure that you are not spending more time counting metrics than you are doing real work, so look for measures that are both meaningful and practical.