Let’s be honest. Nobody in your organization looks forward to the annual test of your disaster recovery plan. It’s similar to the annual agony of physical inventories. Getting your full team to work through the weekend, fuelled by coffee and donuts, is not a recipe for a joyful happening.
But having grumpy and resentful team members is only part of the problem. There are real costs involved as well. Everybody needs to be paid overtime (or time off in-lieu), perhaps including extra effort for all the pre-test preparations that need to take place. External contractors such as recovery site, or data storage services need to be paid as well.
Furthermore, you run the outside risk of data corruption or failure to restore properly, and you may have a service disruption or “limitations” for the course of the testing. Even though it may be a weekend, having your web presence down is not a good thing!
Thankfully, there are alternatives to the big, annual outage.
Cyclical and Continuous DR Testing
Instead of performing your DR testing all at once, cyclical and continuous DR testing involves doing smaller bits of less disruptive testing, spread out over time. Consider, for example, your three or four mission-critical applications. Being top applications, they are most likely often subject to major change such as upgrades.
An application upgrade provides a perfect opportunity to test that application in DR recovery mode at the same time. With your application specialists already engaged and in place, it is the optimal time to perform the additional tasks of DR testing for that application.
Cyclical and continuous DR testing involves changing the focus and scope of your testing processes. In the absence of major upgrades to any one application, schedule smaller tests less frequently, then every year. Test an application or two, or one or two elements of the core infrastructure at a time.
Spread the testing out, so that the most important aspects of your suite of applications are done more frequently, and the less important (or less volatile) ones are done every two or three years. Continuous testing of smaller segments of the DR plan provides a way to prioritize more critical, highly changing elements for frequent or annual verification.
Embed DR Practices Into Your Normal Routines
In an ideal situation, team members throughout your organization will be able to make system or application DR testing part of their normal systems development routines. Smaller teams will work more efficiently in the areas they are most familiar with. Start the testing processes with the most important systems, applications, and tiers then gradually increase the scope of the testing.
As DR practices become embedded in normal IT practices, and updating is occurring regularly, then full-blown DR plans need to be tested less. And that may mean a lot of money saved, and a lot fewer grumpy, caffeine and sugar-fuelled team members working all weekend.
