The information technology role in disaster recovery and business continuity: governments, and particularly IT departments, need

When disaster happens, who is in charge in your city, county, or school? Depending on the severity of a disaster, different areas and groups of employees are affected; however, the information technology department should be among a government's first responders, because it must be ready on a moment's notice to activate a pre-determined alternate site if normal facilities become unavailable.

Disasters can be as small as a few flooded offices, a fire that destroys a room or building, or even a labor dispute, or they can be as extensive as hurricanes or tornadoes. Disasters that shut down a government's mission critical applications for any length of time could have devastating direct and indirect costs to the government and its economy For these kinds of disasters, a disaster recovery and business continuity plan essential. (1)

The first steps the IT department should take depend on how seriously a disaster affects resources. Does it require a few desktops and a room off site to provide a temporary recovery solution? Or does a larger plan need to be activated to move PCs and servers to a "hot site" to restore entire applications and set up temporary work facilities for a limited number of key workers to operate until normalcy is restored?

But what good does it do for IT to restore applications and data if there is no one there to run things? It is only half the solution, albeit the first half. The second half is the contact information for the business continuity piece. Recovering from disaster is less a solution than a process. Governments must take control of their own destinies. In the event of a disaster, a core team of people across all departments is typically designated to continue business operations pending the restoration of a normal work environment. These people need accurate information with which to call on IT and on vendors for technical support or to report to work at a temporary site.

BUSINESS CONTINUITY PLANNING

One of the most basic steps is backing up. Turning a light switch off and on is a common everyday occurrence. Just as frequently, IT should be backing up all the data and applications and storing the tapes off site in a secure environment. Tapes should never remain on site longer than one or two business days. A secure facility with environmental controls and a method for transporting tapes off site, as well as returning them if restoration of corrupted data is needed, is as critical to any government data center as health insurance is to an individual.

In addition to regular tape backup activity, a disaster recovery (DR) "bible" is an essential source of information. The DR bible is a reference book in which you compile all the information necessary to put a recovery plan in place. This includes technical documentation, contact information for critical IT staff, and process documentation. The book should be written specifically for your current work environment, tested at least annually, and updated as often as every three months. Copies of it should be kept off site as a backup, just as with any other critical data.

Technical documentation in the DR bible should include specific start-up instructions for each application and a list of any processes to run jobs on a daily basis. Sometimes steps must be followed sequentially to restore the data off site, which occurs because full backups are only done on a weekly basis with incremental backups done nightly in between the weekly backup process. The DR bible should also contain copies of the applications themselves.

The DR bible also should contain contact information for all of the people, both internal and external, who are essential to keep IT running. That includes home and cell phone numbers or other contact information for all IT people involved as well as key vendors such as equipment and service providers.

Lastly, the DR bible documents the process of disaster recovery. Process documentation is often overlooked when planning for disaster. Damage to facilities and/or dislocation of employees mean that it is often not possible for a full IT department and full staff to operate on site, and as a result people often find themselves doing unfamiliar jobs. Process documentation enables those people to function efficiently by providing the instructions necessary for them to do their work, which in a disaster often means covering for someone else on short notice and with little training.

PRACTICE IDENTIFIES WEAKNESSES

Training is a crucial part of preparing for disaster recovery. With accurate and up-to-date documentation, a disaster drill once a year will validate the accuracy of the technical and contact information and process documentation in the DR bible, and help both IT staff and end users to understand their roles and responsibilities during an actual disaster. Your goal with training is to prepare staff to restore data and perform skeleton business activities off site at a predetermined place. Rehearsing core function restoration results in successful understanding of roles and processes during any kind of disaster, allowing the mistakes identified during the testing activity to be corrected, both in process and documentation.

The IT department plays a key role in business continuity planning by validating the hardware and equipment needed to make the government's plan successful. The government also must affirm the limited amount of services IT can provide in such a disaster, and not expect IT to replicate the full environment for government. However, there are exceptions in the event of long-term disasters that effectively change that "full environment."

Designating leadership roles in the event of a disaster is integral to the recovery process. That is because a disaster at least temporarily changes the priorities that must be set in order to keep government running functionally Notably, while IT normally does not "dictate" how other departments operate, the IT department must take charge in the beginning when setting up an emergency environment, because it has, or should have, the technical expertise to restart operations under extremely trying conditions. You cannot function without technology and only the IT department has the ability to restart that technology.

With the information in place to do the job of keeping technology running, it is important not to forget the need to actively engage all partners in this process in training and disaster preparation. Tasks include working with Internet and telephone providers to locate a suitable "hot site" for disaster recovery and as a backup for lost or damaged government office buildings; including vendors and service providers in the documentation and training process; and contracting with vendors for emergency services and equipment replacement in the event of a catastrophic loss.

Once again, it is important to emphasize the need for regular updates to plans and regular training. Documentation and contact information must be kept updated. Systems must be tested on a regular basis. With these measures in place, a government is far less likely to be caught unprepared when disaster occurs. Disasters ranging from floods, fires, accidents, and even sabotage can be handled efficiently Preparations such as these also enhance your ability to deal with the most geographically widespread disasters, even though they pose special challenges ranging from the scale of the damage to the tendency of state and federal disaster support systems to be geared for more localized events.

Fortunately, governments increasingly recognize the need for IT preparations in disaster planning, including ensuring data security In one survey, 84 percent of the respondents indicated cyber security is part of their overall business continuity plan. (2)

CONCLUSION

The key reason that IT is an important part of disaster planning and preparation is its vital role in ensuring business continuity in the event of a disaster. Constituents and employees depend on continuity in emergencies just to survive, and government is the first place people turn to when disaster strikes.

The first step in disaster preparation is to ensure the information technology department is an active participant. Governments depend on IT to manage their hardware and software needs in the event of a disaster, but often do not document the business continuity activities necessary to operate in a disaster mode. Governments need to document the steps needed to continue operations in the event of a minor or critical disaster. The key to success is participation from all sides, working together as teams to restore data and perform skeleton business activity off site. The IT department is a key player in the business continuity planning to make sure the hardware and equipment needed will be ready to make the government's plan successful. Processes need to be in place to keep the emergency documentation and lists off premises as well in case a disaster makes facilities inaccessible.

Being prepared involves practice. It also involves resources both inside and outside of government. Proper documentation of procedures, processes, and contact information as well as regular training and updates of information dramatically enhance governments' ability to ensure the continuity of essential services through a critical disaster situation.

Planning for Disaster

In 2006, the GFOA worked with Newport News, Virginia, Public Schools to secure an ASP contract (application services provider) for ERP services. One of the benefits of an ASP model is frequent data or file backup and restoration since the tasks often occur off site. The Newport News agreement included a complete IT business continuity plan designating a chain of notification and process for remote communications. It also addressed what would need to happen to recover from a disaster at the main data center.

Print Get the Mag Weekly Updates