Home > Articles

This chapter is from the book

Measuring Return on Content

In the volumes of literature on knowledge management (KM), the study of return on investment has yielded less than compelling results. For some, the issue is no longer relevant; while others have tried to attack it from myriad perspectives—both quantitative and qualitative. As we discussed in the Preface, the connection between content management and knowledge management is obvious. As a result, the ROI problems of knowledge management have been carried over to content management, and to understand them we need to look at why the ROI on knowledge management has been so difficult to nail down. With that understanding, we will try to cut through this dead-end discussion to arrive at a more useful view of how effective management of content (as a form of knowledge) can yield positive qualitative and quantitative returns.

Health maintenance organizations (HMOs) are struggling with very thin profit margins (if they are making any profits at all) because of the rising costs of healthcare. These costs are largely out of the HMOs' control, so they must look elsewhere in the organization for cost containment. Administrators struggle mightily with information management. Every area, from sales to contracts to service, is intensely dependent on accurate information. More often than not, this information should be shared across the departments, but it isn't.

For example, 200,000 letters communicating a policy change have gone out in mass mailings to HMO's insured members, but member services knows nothing of the change, not to mention the letter. Confused (and irate) members call looking for clarification, while member services representatives remain in the dark. Table 1.1 details the total costs of communicating the policy change.

Table 1.1 Communicating Policy Change Cost

Number of letters

200,000

Production and mailing cost of one letter

$1.50

Total Cost of Mailing

$300,000

Number of calls to call center

10,000

Duration of call

15 minutes (.25 hours)

Hourly cost of member service representative

$22

Total Cost of Response

$55,000

TOTAL

$355,000


A simple process change in which member services is notified three days before a mailing goes out can help the rep prepare for the inevitable spike in phone calls. Better yet, preparing a Frequently Asked Questions document ahead of time can make it easy to answer members' questions, further reducing the additional $55,000 of lost productivity.

Knowledge Management and Measurement: A Literature Review

The origins and development of knowledge management as a business topic have hindered a fruitful discussion of how knowledge can yield bottom line results that can be validated through accurate measurement. A look at the seminal books on this subject helps us understand why this is the case.

The first problem that knowledge management faced was its presentation, inevitably, as a new management philosophy akin to TQM, re-engineering, learning organizations, and the like. This was because the key books on the topic were preoccupied with convincing readers that knowledge needed to be treated as an asset that should be managed just as land, labor, and capital were. While the management techniques and "best practices" for managing these familiar assets were mature, those for knowledge were introduced as something wholly different and autonomous.

Thomas H. Davenport and Laurence Prusak wrote about "knowledge markets" in Working Knowledge: How Organizations Manage What They Know; Ikujiro Nonaka and Hirotaka Takeuchi elaborated on the "knowledge spiral" in The Knowledge-Creating Company. Along with Thomas A. Stewart's Intellectual Capital: The New Wealth of Organizations, these books set the parameters for how knowledge management would be subsumed into the vocabulary of management in the United States. As a result, they profoundly affected our ability to understand and measure its ROI.

The language of knowledge management has been remarkably consistent. The topic came into vogue after 1995 with the publication of The Knowledge-Creating Company. "Intellectual Capital" appeared in 1997 and was quickly followed in 1998 by "Working Knowledge."1 In these books and articles, the central argument was that the collective intelligence (i.e., knowledge) of an organization was a new kind of asset that must be managed for the organization's strategic benefit. The focus was therefore on describing the special management techniques necessary to harness this new asset to create "new wealth" and "innovation."

With this focus, the authors had no choice but to start with some discussion of the nature of knowledge. The first chapter of Working Knowledge, What Do We Talk about When We Talk about Knowledge? spends a great deal of time separating data, information, and knowledge. The Knowledge-Creating Company begins with a long discussion of the competing epistemologies of Plato and Aristotle, moves through the Enlightenment, and ends up with a brief discussion of Heidegger's notion of Dasein. At issue was grounding the discussion in knowledge as something unique and special. Only by establishing this could discussion of the proper management techniques for this most unique asset begin.

Because these books spent so much time defining the qualities of knowledge, business strategy took a back seat, creating confusion as the topic matured. For instance, Davenport and Prusak, after defining knowledge, make a fairly simple statement: "Knowledge can and should be evaluated by the decisions or actions to which it leads." Here was a step in a necessary direction: the need to attach knowledge management practices to strategic business objectives. Knowledge is valuable only insofar as it leads to decisions and actions deemed valuable and productive to the business. The authors continue with their central argument that knowledge is a unique asset that needs special management techniques: "We've observed and analyzed over a hundred attempts to manage knowledge in organizations. To the managers of most of them we've posed the question, 'How do you make the distinction between data, information, and knowledge?' " (p. 6). However, if their interest was truly with the value of knowledge as an asset, we would have expected a different question: "How do you know what knowledge is of most value to the organization?"

Stewart makes a similar move in Intellectual Capital: "There's a vital lesson here: Knowledge assets, like money or equipment, exist and are worth cultivating only in the context of strategy" (p. 71; emphasis in original). But this point is immediately dropped and no systematic argument is made for showing how to link these assets (i.e., intellectual capital) to business strategy. In fact, his next chapter, Recognizing Tacit Knowledge, abruptly returns to the central concern, which is to define the essential qualities of knowledge.

We aren't criticizing these books for not pursuing a link between knowledge management and strategy. Their authors were making the necessary first steps in introducing knowledge as a strategic asset, and they rightfully spent their time discussing the special qualities of intellectual capital and how to manage it. Still in their attempts to add force to their arguments, the authors were unnecessarily confused and confusing in the new assets' positioning. They elevated knowledge in such a way that it could be seen as more important than strategy. Stewart claims, for instance, that "managing intellectual capital should be business's first priority" and that "Knowledge has become the preeminent economic resource—more important than raw material; more important, often, than money" (p. 6).

Davenport and Prusak add to the confusion. They begin by talking about their involvement with managers who "were admitting that they didn't have any effective methods and approaches for managing and understanding how to better use information themselves." They then turn their research to "why it isn't managed well, what managing information actually means, and what kinds of specific improvements our clients could make in how they obtained and used it" (p. xii). These are very reasonable problems to address when the focus is to understand the unique nature of knowledge and information as assets. However, in citing economist Sidney Winter, Davenport and Prusak state, "If 'knowing how to do things' defines what a firm is, then knowledge actually is the company in an important sense" (p. xiii; emphasis in original).

So, the discussion of knowledge management as it was introduced simultaneously offered promise and confusion. The promise was that knowledge and information are intimately related to strategy and that, with new technology for better managing them, companies could improve their top and bottom lines. The confusion came with statements about knowledge as somehow more important than strategy. As a result, knowledge management could be thought of as an independent undertaking within a business—as if the management of knowledge could proceed unattached from strategic concerns.2

At that point, measuring ROI became nearly impossible. Typical financial metrics seemed inappropriate—return on capital employed (ROCE) being the classic example. Proving that a new piece of capital equipment could yield cost reductions through improved productivity was easy, showing that automating accounts receivable could reduce errors and shorten "days sales outstanding" (DSO) was a no-brainer. However, the benefits of knowledge management were touted as competitive advantage, employee learning and retention, and other so-called intangibles. In fact, in many cases effects such as collaboration and communication were offered as valuable ends in and of themselves rather than as the means to other more strategic objectives. But how do you measure the benefit of improved collaboration? Email systems allow more prolific communication, but how do you calculate their ROI? In fact, conventional measures would put email systems at a complete loss with no positive financial return.

Content Services and Business Performance Measurement

Fortunately, the art and science of business performance measurement have matured over the last few years. The work in this area—balanced scorecards (BSC) and Six Sigma specifically—has helped us refocus on the link between strategy, performance, measurement, and management. These new disciplines thus help us intelligently resituate the ROI discussions of knowledge management (and consequently content services).

Beyond ROI

First, we need to distinguish between performance measurement and ROI. On the one hand, performance measurement is a discipline that focuses on identifying the metrics to precisely gauge the execution of strategy. On the other hand, ROI makes no necessary connection between the investment being measured and the direction of the business; ROI studies tend to be done in isolation. The key question is always whether or not the specific investment has returned more financial benefit to the organization than its cost. A positive return alone is assumed to be beneficial.

The essential challenges in determining ROI are twofold:

  • The boundaries around the investment itself. Can we completely account for all costs associated with the investment—human, opportunity, capital, and so forth?

  • The boundaries around the benefits of the investment. Can we completely understand all of the benefits—financial and nonfinancial—associated with the investment?

If you can adequately answer these challenges, you can determine whether or not your gain is more than your investment. However, this kind of analysis doesn't take into account the overall benefit to the business. For instance, a new investment in upgrading a piece of equipment would allow for an increase in the capacity of a factory to turn out product, but what if the sales force's compensation plan didn't provide adequate incentives to move the additional volume? The ROI analysis would show a positive return—fewer inputs to produce greater outputs—unless you took into account the additional inventory carrying costs necessary to handle the additional output that wasn't being sold.

ROI analyses tend to let management off the hook. In the example we just cited, what if this investment were made without knowing whether or not the market could absorb the additional capacity? A narrow view would indicate a positive impact; a broader view would show a possible negative impact as inventory stagnated in the warehouse. Neither view—narrow or broad—would reveal whether or not the investment contributed to an overall plan.

Recent work in performance measurement has pointed out the shortcomings of financial indicators like ROI and ROCE as measures of business performance. It isn't that they are irrelevant but rather that they need to be part of an overall strategy. Instead of being isolated numbers in a report, they should become key performance indicators (KPIs) that tell management how well the business is doing in executing its chosen strategy to achieve its objectives. Translating this into the example above, the ROI on the capital investment makes sense insofar as the company has determined that they are going to actively grow the market share of the product. They have set up a sales and marketing plan, which the investment in the equipment directly supports; so the ROI will be measured appropriately against two KPIs:

  • Manufacturing capacity. Did the investment allow for meeting the capacity requirements set forth in the plan?

  • Market share. Was the desired market share increase achieved?

The first KPI lets you know if the manufacturing manager hit his objectives. The second tells you whether or not investment in the new equipment materially benefited the business.

Return on Investment versus Business Performance

A DVD player manufacturer's customer support center has implemented a new call-tracking system for $1.5 million, and within the first year the result has been a 40 percent reduction in talk time. Another investment of $750,000 will complete the job with an expected drop in callbacks of about 75 percent. The problem is that customer satisfaction has dropped precipitously over the same time. Studies show that customers feel rushed through the support process and are showing up in the stores to get their problems fixed. This is preventing sales associates from helping new customers.

Measuring Content Services Performance

As we discussed in the Preface, content services is a form of knowledge management. Consequently, it is trapped in the shortcomings of the ROI on knowledge management discussions we have been analyzing. Let's rehabilitate this discussion in the context of content services at this point.

The challenge of determining the return on content services is understanding how it affects the organization's financial KPIs. Therefore, we need to establish a cause-and-effect relationship between content services and financial performance, and the only way to do this is to first know the financial goals of the organization and the operational plan and KPIs that support them. Then and only then can we begin to make sense of how content services can contribute to financial performance.

There are many ways to create business strategy, as the ever-growing literature on this topic illustrates. Our starting point is the balanced scorecard as developed originally by Robert S. Kaplan and David P. Norton in The Balanced Scorecard: Translating Strategy into Action. We like the BSC because it provides a clear method for translating goals into metrics at all levels of the business. Kaplan and Norton are only tangentially concerned with how content can impact performance; but we see in their approach a way to, first, link knowledge management and content services initiatives to strategic initiatives and, second, measure that performance appropriately.

Kaplan and Norton articulate the different levels of strategy that allow you to set up reliable KPIs:

  • Financial. What does the company hope to achieve in the given timeframe—revenue growth, profitability, cash flow neutrality?

  • Customers. What segment or segments of the target customer base will be addressed to achieve financial goals?

  • Internal business processes. What business processes are key to the organization in achieving financial and customer objectives?

  • Innovation and learning. What employee skills and technical infrastructure need to be in place for the strategy to be successful?

We are going to use these levels to show how content services can be a key contributor to business strategy and thus yield a measurable and meaningful benefit. We will do this by showing how and where better access to relevant, accurate, and useful content positively affects the performance measures linked to financial goals.

Figure 1.2 shows this cause-and-effect relationship. In it we have groupedthe financial and customer perspectives as strategic and the Internal process and innovation and learning perspectives as operational , which highlights the different roles that content services play depending upon which perspective is relevant. At the strategic level, content services performance is measured largely by the ability to deliver necessary information (e.g., metrics, reports, competitive intelligence, customer satisfaction surveys) to decision makers. It is also measured by the ability to communicate strategy and metrics to those who need to know (e.g., shareholders, employees, partners).

Figure 1.2Figure 1.2 Measuring the Performance of Content Services

At the operational level, content services is responsible for lower-level KPIs as well as for timely, accurate content that promotes the following:

  • Improved productivity, timeliness, flexibility, and quality of key processes

  • Improved innovation and decision making at the front lines

  • Alignment of employee skills with strategic direction

Each of these is measurable, depending on the specific goals of the organization.

InformIT Promotional Mailings & Special Offers

I would like to receive exclusive offers and hear about products from InformIT and its family of brands. I can unsubscribe at any time.

Overview


Pearson Education, Inc., 221 River Street, Hoboken, New Jersey 07030, (Pearson) presents this site to provide information about products and services that can be purchased through this site.

This privacy notice provides an overview of our commitment to privacy and describes how we collect, protect, use and share personal information collected through this site. Please note that other Pearson websites and online products and services have their own separate privacy policies.

Collection and Use of Information


To conduct business and deliver products and services, Pearson collects and uses personal information in several ways in connection with this site, including:

Questions and Inquiries

For inquiries and questions, we collect the inquiry or question, together with name, contact details (email address, phone number and mailing address) and any other additional information voluntarily submitted to us through a Contact Us form or an email. We use this information to address the inquiry and respond to the question.

Online Store

For orders and purchases placed through our online store on this site, we collect order details, name, institution name and address (if applicable), email address, phone number, shipping and billing addresses, credit/debit card information, shipping options and any instructions. We use this information to complete transactions, fulfill orders, communicate with individuals placing orders or visiting the online store, and for related purposes.

Surveys

Pearson may offer opportunities to provide feedback or participate in surveys, including surveys evaluating Pearson products, services or sites. Participation is voluntary. Pearson collects information requested in the survey questions and uses the information to evaluate, support, maintain and improve products, services or sites, develop new products and services, conduct educational research and for other purposes specified in the survey.

Contests and Drawings

Occasionally, we may sponsor a contest or drawing. Participation is optional. Pearson collects name, contact information and other information specified on the entry form for the contest or drawing to conduct the contest or drawing. Pearson may collect additional personal information from the winners of a contest or drawing in order to award the prize and for tax reporting purposes, as required by law.

Newsletters

If you have elected to receive email newsletters or promotional mailings and special offers but want to unsubscribe, simply email information@informit.com.

Service Announcements

On rare occasions it is necessary to send out a strictly service related announcement. For instance, if our service is temporarily suspended for maintenance we might send users an email. Generally, users may not opt-out of these communications, though they can deactivate their account information. However, these communications are not promotional in nature.

Customer Service

We communicate with users on a regular basis to provide requested services and in regard to issues relating to their account we reply via email or phone in accordance with the users' wishes when a user submits their information through our Contact Us form.

Other Collection and Use of Information


Application and System Logs

Pearson automatically collects log data to help ensure the delivery, availability and security of this site. Log data may include technical information about how a user or visitor connected to this site, such as browser type, type of computer/device, operating system, internet service provider and IP address. We use this information for support purposes and to monitor the health of the site, identify problems, improve service, detect unauthorized access and fraudulent activity, prevent and respond to security incidents and appropriately scale computing resources.

Web Analytics

Pearson may use third party web trend analytical services, including Google Analytics, to collect visitor information, such as IP addresses, browser types, referring pages, pages visited and time spent on a particular site. While these analytical services collect and report information on an anonymous basis, they may use cookies to gather web trend information. The information gathered may enable Pearson (but not the third party web trend services) to link information with application and system log data. Pearson uses this information for system administration and to identify problems, improve service, detect unauthorized access and fraudulent activity, prevent and respond to security incidents, appropriately scale computing resources and otherwise support and deliver this site and its services.

Cookies and Related Technologies

This site uses cookies and similar technologies to personalize content, measure traffic patterns, control security, track use and access of information on this site, and provide interest-based messages and advertising. Users can manage and block the use of cookies through their browser. Disabling or blocking certain cookies may limit the functionality of this site.

Do Not Track

This site currently does not respond to Do Not Track signals.

Security


Pearson uses appropriate physical, administrative and technical security measures to protect personal information from unauthorized access, use and disclosure.

Children


This site is not directed to children under the age of 13.

Marketing


Pearson may send or direct marketing communications to users, provided that

  • Pearson will not use personal information collected or processed as a K-12 school service provider for the purpose of directed or targeted advertising.
  • Such marketing is consistent with applicable law and Pearson's legal obligations.
  • Pearson will not knowingly direct or send marketing communications to an individual who has expressed a preference not to receive marketing.
  • Where required by applicable law, express or implied consent to marketing exists and has not been withdrawn.

Pearson may provide personal information to a third party service provider on a restricted basis to provide marketing solely on behalf of Pearson or an affiliate or customer for whom Pearson is a service provider. Marketing preferences may be changed at any time.

Correcting/Updating Personal Information


If a user's personally identifiable information changes (such as your postal address or email address), we provide a way to correct or update that user's personal data provided to us. This can be done on the Account page. If a user no longer desires our service and desires to delete his or her account, please contact us at customer-service@informit.com and we will process the deletion of a user's account.

Choice/Opt-out


Users can always make an informed choice as to whether they should proceed with certain services offered by InformIT. If you choose to remove yourself from our mailing list(s) simply visit the following page and uncheck any communication you no longer want to receive: www.informit.com/u.aspx.

Sale of Personal Information


Pearson does not rent or sell personal information in exchange for any payment of money.

While Pearson does not sell personal information, as defined in Nevada law, Nevada residents may email a request for no sale of their personal information to NevadaDesignatedRequest@pearson.com.

Supplemental Privacy Statement for California Residents


California residents should read our Supplemental privacy statement for California residents in conjunction with this Privacy Notice. The Supplemental privacy statement for California residents explains Pearson's commitment to comply with California law and applies to personal information of California residents collected in connection with this site and the Services.

Sharing and Disclosure


Pearson may disclose personal information, as follows:

  • As required by law.
  • With the consent of the individual (or their parent, if the individual is a minor)
  • In response to a subpoena, court order or legal process, to the extent permitted or required by law
  • To protect the security and safety of individuals, data, assets and systems, consistent with applicable law
  • In connection the sale, joint venture or other transfer of some or all of its company or assets, subject to the provisions of this Privacy Notice
  • To investigate or address actual or suspected fraud or other illegal activities
  • To exercise its legal rights, including enforcement of the Terms of Use for this site or another contract
  • To affiliated Pearson companies and other companies and organizations who perform work for Pearson and are obligated to protect the privacy of personal information consistent with this Privacy Notice
  • To a school, organization, company or government agency, where Pearson collects or processes the personal information in a school setting or on behalf of such organization, company or government agency.

Links


This web site contains links to other sites. Please be aware that we are not responsible for the privacy practices of such other sites. We encourage our users to be aware when they leave our site and to read the privacy statements of each and every web site that collects Personal Information. This privacy statement applies solely to information collected by this web site.

Requests and Contact


Please contact us about this Privacy Notice or if you have any requests or questions relating to the privacy of your personal information.

Changes to this Privacy Notice


We may revise this Privacy Notice through an updated posting. We will identify the effective date of the revision in the posting. Often, updates are made to provide greater clarity or to comply with changes in regulatory requirements. If the updates involve material changes to the collection, protection, use or disclosure of Personal Information, Pearson will provide notice of the change through a conspicuous notice on this site or other appropriate way. Continued use of the site after the effective date of a posted revision evidences acceptance. Please contact us if you have questions or concerns about the Privacy Notice or any objection to any revisions.

Last Update: November 17, 2020