In the near future many business computing applications will depend heavily on the use of Big Data. In this three-article series, we will explore how the characteristics of Big Data may end up forcing your organization to alter its IT capabilities over time, including adjustments to your disaster recovery planning.
What Is Big Data?
Big Data is the accumulation of large volumes of information (sometimes in real time), to be captured, reformatted, processed, interpreted, and represented to end users and customers in a variety of forms. Google and Facebook are the pioneering giants of Big Data, but Big Data is now an important decision-making and operational tool for many other sectors, including the power grid, weather and environmental services, banking and financial services, Internet and cable providers, and other retail and consumer businesses.
Characteristics of Big Data
When we talk about Big Data, we are talking about an amount of information that is too big to fit on one single computer, server, disk drive or even site, to be processed. This is information that arrives quickly and constantly, and has to be manipulated through multiple pieces of IT equipment and machinery. But it is more than simply a large amount of frequently arriving information. Big Data involves more complex distribution and presentation than typical inventory, order processing, or other transaction systems.
Unlike traditional databases, Big Data grows perpetually, picking up ad hoc and scheduled information feeds 24/7, and in multiple formats. Its distribution may involve a multi-step pipeline from source to ultimate end users, and then back again. Or, you could have multiple sources feeding a Big Data application before syndicating it out to multiple users at the back end, like a spreading fan at both ends.
The processing of Big Data typically involves three or more steps before the information is ready to be used by its users. After being captured at source, the data is “ingested” in any number of ways: extracted, compressed, anonymized, filtered, formatted, cleaned, or normalized.
The data is then sliced and diced and analyzed into the ultimate decision-making analytics or application functions. It may be split, aggregated, summarized and combined, after which it is transferred, distributed and delivered back to users in various forms, including graphic visualizations, dashboards, tables, alerts and other forms of presentations.
Business Impacts of Big Data
Big Data is becoming a fact of life for many businesses, and your organization is at times obligated to accept, process and respond to it through its normal course of activity. At other times, you may subscribe to big data in one form or another. Insurance companies, banks and brokerage houses all take transaction data in large volumes, they anonymize it and process it in other ways before feeding it back to their external suppliers, customers and partners.
This data can be highly valuable once processed. It may become part of your organization’s official record, and may form the basis of obligatory audit trails. Big Data will inevitably also supply unforeseen future needs, and involve information that may or may not result in privacy, confidentiality or sensitivity issues down the line (e.g. records of your messages, texts, calls or images caught-on-video).
For these reasons and more, the growth of Big Data ought to generate a serious rethink of your organization’s IT capabilities, not the least of which concern is its impact on your disaster recovery plans. More on how Big Data presents challenges for disaster recovery next week.