How Big Data Changes Your Disaster Recovery Plans

close up of business team with coffee and papers

Big Data increases the challenges of disaster recovery. New performance and availability needs brought on by Big Data will push your DR design and architecture, as well as your hardware and software choices. Continuity and recovery strategies become more complex as Big Data processes expand, because the data sets are bigger, there are more records and steps in the processing, and more advanced components.

Here are some of the implications of Big Data for DR analysis, strategies, implementation, testing and maintenance. All of it implies more work for your organization.

Analysis

The presence of Big Data means your organization needs to clearly understand where it is being used, how it is being used and by whom, and how it relates to your overall IT capabilities in terms of interconnections between applications, shared (and uniquely-specialized) infrastructure resources, and so on.

Where business typically think of an RPO in terms of quantity – how much time or data was lost – the addition of Big Data to the equation means you need to be aware of the quality, accuracy and integrity of the data at issue. When you have a multi-step process in your cycle, Big Data makes it more difficult to determine what got “processed” and what didn’t in the case of a serious event, unless you do a complete “damage assessment” prior to restart.

Strategies for Continuity and Recovery

Your DR strategies will require extra selection, evaluation and design efforts. On the continuity side, you will need to look more at stop-gaps for any obvious single points of failure, and decide whether you can harden your unique processes with better management/monitoring tools, expandable communication links, or logger redundancy, in order to avoid minor disasters.

When you get to the recovery side, because Big Data brings on increased criticality of information, you will need to consider higher level data synchronization, replication, mirroring and duplication than you did before. More data means more storage capacity somewhere (i.e. at machine-level, logger, queue, data file, etc.) more specialized software and tools, and more communication bandwidth to accommodate alternate or high volume bursts of traffic to catch-up.

Implementation, Testing and Maintenance

Big data may require a little more focus: more rigorous end-state diagnosis, more elaborate restart and recovery procedures, all of which require more diligence and accuracy in “as-built” configuration documentation and up-to-date operating procedures (e.g. outlining data sources, storage points, arrival frequencies, transaction-volume storage capacities for interim steps, etc.). Special re-processing and likely some backlog processing need to be built into the plan in order to make a more robust restart take place smoothly.

Technical processes may now require unique skills and new service architectures due to new volume, bandwidth and special licensing requirements. Since some Big Data originates from 3rd party sources, certain data access arrangements will need to be revisited. Unique production and processing schedules may require specialized plans for simulated or actual failover/failback testing. Embedded DR checkpoints, decisions and documentation will be more important for the Big Data –based critical applications.

More than anything else, the introduction of Big Data makes your Recovery Point Objective (RPO) more relevant than ever before. Given large stores of crucial, rapidly-changing data, the threat of data loss, latency and inaccuracy make challenging disaster recovery decisions even harder to make.

Steve Tower

With many years of professional IT experience, and training as a Certified Management Consultant, a Project Management Professional, a Professional Engineer and a Member, Business Continuity Institute, Steve Tower has the skills and abilities required to assist with even the most complex disaster recovery planning initiatives. Below, Steve discusses the necessary tools involved in setting up a disaster recovery plan and program.