Disaster Recovery Planning: Preparing For The Unthinkable

This chapter is from the book

Disaster Recovery Planning: Preparing for the Unthinkable, 3rd Edition

Learn More Buy

This chapter is from the book

This chapter is from the book 

Disaster Recovery Planning: Preparing for the Unthinkable, 3rd Edition

Learn More Buy

Building Management Consensus for Disaster Recovery Planning

This brief survey demonstrates that the disaster recovery planning project is propelled by a number of considerations, ranging from a common-sense business impetus to safeguard corporate assets from loss or damage to a natural desire to reduce legal exposure and personal loss. This is not to say, however, that corporate management is aware of all of the legal penalties, or even the risks, associated with not having an effective disaster recovery capability. In some cases, management consensus must be cultivated by an information manager or auditor.

The above observation may seem out of step with the current elevated level of attention being paid to disaster preparedness in the wake of 9/11. However, history has demonstrated over and over again that there is a tendency for interest in DR to spike in the aftermath of a disaster, then to diminish rather quickly as the memory of the disaster fades.

Even after a consensus is built to support the planning for business interruption and recovery, sustaining the consensus when it comes time to implement the paper plan—to install the recovery capability—can be difficult. The reasons for the breakdown of the consensus are numerous. In some cases, management exhibits reluctance to spend money acquiring the services and products that are intrinsic to the plan. This often occurs when management does not fully understand the risks and exposures a company faces without a recovery capability.

To address this dilemma, DR consultants recommend that a formal risk analysis process be undertaken early in the disaster recovery planning project. An assessment should be made of the impact of unplanned interruption events on the business as a whole—including a detailed assessment of tangible and intangible costs accrued to the unplanned interruption of each automated application that supports a mission-critical business process.

As an adjunct to this analysis, any legal requirements that compel management to undertake planning should be cited and clearly communicated. Where the law does not directly mandate disaster recovery planning, other strategies have to be found to convince reluctant senior managers of plan benefits. Effective strategies are often difficult to find.

Convincing corporate management to shoulder the costs of a disaster recovery capability can often be a greater challenge than surmounting the technical problems involved in backing up critical systems and networks. In the final analysis, however, management will play the most critical role in the planning effort—the role of underwriter.

Following are some typical problems reported by the information managers who had to sell their plans to senior management, and the successful strategies they developed to overcome them.

• "You haven't cost-justified the plan."

  This criticism usually reflects the disaster recovery planner's failure to document adequately the exposures and risks of not having a plan. Often, planners first encounter this objection when they seek management approval to conduct an analysis of risks and exposures. That is, management may ask the planner to cost-justify the plan before they have authorized an investigation of whether such a disaster recovery capability is necessary!

  How does one cost-justify a capability that, in the best circumstances, will measure its success in nonevents? This dilemma was perhaps best expressed in a humorous anecdote that followed January 1, 2000 in which a chief executive officer railed at his IT manager, "You mean we spent all that money on Y2K preparations and nothing happened?!" A good disaster recovery plan, after all, sets the stage for disaster avoidance by providing the means to detect and react to potential problems, in many cases, before they become disasters. Strategies for successfully addressing this problem generally fall into two categories.

  One strategy for responding to this criticism is to assign a dollar value to an hour of downtime. Calculate the average hourly earnings of employees who use an application that provides a business process for which a recovery capability will be planned. This provides an estimate of the cost of an hour of lost productivity were an application outage to occur. Assuming that the data could be entered at a later time without other adverse consequences, repeat the above calculation for 1 hour of average overtime salary (i.e., the amount of time all system users would need to work to make up for lost time). Then, add the two dollar costs as the total average labor cost for one hour of downtime.

  For applications used by greater numbers of end users, this number by itself could be sufficient to demonstrate the benefit of a disaster recovery capability that would avoid outages or minimize the duration of unavoidable outages. By multiplying the number to reflect 10 hours, 24 hours, 48 hours, and so on, the statistic could be quite compelling.

  This approach, while simplistic, can be augmented by citing any documented costs associated with outages that have previously been experienced by the company. However, the planner must keep in mind what was said earlier about the tendency of outage costs to decline over time.

  In some cases, the better case can be made by referencing intangible costs. The loss of customer satisfaction, the abrogation of service level commitments, potential legal liability, lost sales opportunities, negative press, and a host of other non-dollar-based factors may hold significant meaning for management. Arguments based on these non-quantifiable factors may actually be more compelling than cost-justification and other quantitative methods. The latter are inherently flawed, in any case, by the lack of reliable data on the likelihood of occurrence for any given disaster potential.

  Another way to justify the plan is to demonstrate the collateral benefits of such a plan. An effective disaster recovery capability can actually reduce business insurance costs in some situations. Premiums for facility or business interruption insurance may be substantially reduced in many cases where disaster recovery planning identifies the specific coverages required and blanket insurance policies are replaced with targeted and less-expensive plans.¹⁵

  Some disaster recovery capabilities, such as uninterruptible power supplies (UPS), actually do double-duty. They can sustain critical business systems or network devices for a time so that organized shutdowns can be accomplished or independent generators started up in the event of a disastrous power outage. In addition to its use in a disaster, the UPS also supports equipment during the occasional surges, dips, and flickers of a typical business day. In so doing, the UPS can actually prolong the useful life of connected equipment.

  Another dual-use scenario may apply to an end-user or workgroup operations center—a facility designed to house users if the main business facility becomes uninhabitable for a period of time. Such a facility may also be used during non-disaster periods as a training or conference facility.

  A practical example of dual-use may be found in the case of a certain Midwestern state government's IT department. In the wake of 9/11, the politicians in the state made compelling speeches about the need for disaster recovery but allocated no money for building such a capability. The IT manager for the government did, however, manage to receive funding for an "application development and testing center." Determined to build a redundant facility for disaster recovery, the savvy IT manager replicated virtually all critical production systems and networks at the alternate site and created an internal hot site while also providing a location for developing and testing new systems before they were deployed into production for the state. Sometimes, as this case suggests, planners may need to become downright sneaky to overcome obstacles to effective planning.

  Dual-use benefits can usually be discerned for most disaster recovery plan components, given sufficient time and creative energy. When a particularly compelling benefit cannot be found to outweigh a cost, the impact of the cost may be softened if it is examined from the perspective of tax deductions, health and safety of personnel, or good corporate citizenship.

• "Our insurance will cover an outage, so why do we need the plan?"

  Even if a consensus exists for developing a plan on paper, management may resist spending money for implementation, especially if the disaster recovery capability is viewed as just so much more insurance. According to spokespersons for two data processing insurers, the right insurance policy will cover operating costs that are above the normal costs of business operations, provided that the appropriate "extra" costs are spelled out in the policy. This business interruption insurance, however, should not be viewed as business resumption insurance.¹⁶ An information manager may need to educate management in the following facts.

  Insurers can readily cover the costs of facility damage (and, in some cases, replacement), and they can provide coverage for hardware and media. However, while insurers are willing to underwrite the reconstruction of data lost to a natural or manmade disaster, it would be cost prohibitive to the client to underwrite the value of the data.

  Without a disaster recovery plan, the client would be hard pressed to estimate the cost of reconstructing data or to buy adequate insurance for doing so. In all likelihood, without a disaster recovery capability, there would be nothing with which to reconstruct the data: no extant records, no systems, no location for personnel to work. Purchasing extra coverages under these circumstances would be pointless.

  In a disaster situation, a company protected only by business interruption insurance is placed in the unenviable position of relying on the progress of the claims adjustment cycle to drive the recovery. While top insurers generally provide excellent turnaround on disaster claims and may even provide support services to facilitate the insured's recovery activities, this is generally less desirable than controlling recovery within the business itself and capitalizing on the determination and commitment of trained recovery teams who are employees of the company.

  If management does not understand the problems inherent in depending on insurance to recover from disaster, planners should provide information about the experience of companies that have experienced disasters firsthand. The aftermath of Hurricane Andrew—the subsequent flight (and, in some cases, bankruptcies) of property and casualty insurers from the state of Florida—attests to the impact of a regional disaster on insurance companies as well as the potential problems companies confront in obtaining the timely handling of insurance claims.

  The bottom line is that business interruption insurance is properly regarded only as a supplement to a tested disaster recovery capability. It is never a substitute for planning.

• "The purpose of the plan is to satisfy the auditors."

  While it may seem blasphemous, managers often express this sentiment in spoken or unspoken terms. One manager of a national financial concern remarked, off the record, that the best disaster recovery plan he could get his company to fund was an up-to-date resume. Sadly, in the absence of rigidly enforced laws, auditors' comments are often the only incentive for management to undertake disaster recovery planning. Auditors seldom have the power to compel management to do anything it is not inclined to do.

  One strategy for surmounting management indifference is to barrage corporate officers with news clippings about business disasters, although this may ultimately cost the sender some prestige or power. The object of this strategy is not to aggravate or frighten, but to create awareness in senior management that disasters do happen and that those who prepare for them generally recover normal operations far more readily than those who do not.

  Another method for reducing senior management indifference is to demonstrate that the planner understands and participates in management's priorities and objectives. This may be reflected in the methods used to create and articulate the plan. For example, every effort should be made to maximize the plan's protection while minimizing its cost. Despite his or her personal investment in systems and networks, the planner should strive to assess IT resources dispassionately for their criticality to the corporation. Certain applications are more vital than others, a fact that is underscored by carefully analyzing the impact of an unplanned interruption on business applications. It needs to be clearly communicated to senior management that the plan will target the largest share of budget expenditures for the most important applications.

  Furthermore, plans must ultimately encompass not only the recovery of IT and network resources but also the user departments. It makes little sense to restore applications if no provision is made to restore the user community. By involving the managers of user departments in the planning project, the planner may be able to cultivate a corporate climate of support for disaster recovery planning. This, in turn, may reinforce senior management's perception of the value of the disaster recovery planning effort and result in a more comprehensive and effective recovery capability.

  Only a few years ago, the Y2K Bug, and the legal exposure it created for a company, captured the attention of senior management. In some companies, it provided an opportunity for DR planners to demonstrate to senior management that what they do is motivated by concern for the business and for management. While Y2K was merely another software-related disaster potential from the DR perspective, some savvy DR planners were able to leverage management concern to gain political capital that could be used later to expand the scope of DR planning into other areas. The same opportunities exist in the wake of 9/11 and should not be overlooked.

These are only a few of the common problems and strategies used by DR planners to obtain senior management approval for disaster recovery planning costs. Other problems may develop that reflect the particular circumstances of a business, the distribution of information systems, or even the individual personalities of senior managers themselves. Whenever possible, disaster recovery planning should be depoliticized and depersonalized. Since the initial focus of the planning effort is on information systems, the information manager will play an important role in setting the stage for the entire corporate disaster recovery plan.