Major Hurdles in Big Data: Risks and Threats

In her white paper Big Data, Bigger Opportunities (April 2013), author Jean Yan, Program Manager at, U.S. General Services Administration, has brought the risks and threats associated with big data to the forefront, which are so often overlooked by the profit-hungry world of business.

Privacy/Security: The biggest risk that anyone familiar with big data knows is privacy concerns and security issues that emerge from such concerns. Privacy, in the big data world, indicates any or all identifiable information blocks that may be used to establish an individual’s identity. The issues concerning privacy most often surfacing in big data applications are

  • What types of personal information can be deemed sharable and with whom
  • Whether cyber communication can be transmitted without anyone else viewing
    such communication, where concerns of lawful interception may also be included
  •  Anonymous communication how

The big-data champions may discover great business opportunities hidden under the wraps; but the privacy groups or advocates are increasingly getting uncomfortable about the endless possibilities of data usage by personal data collection agents. The biggest worry is not about the profile data, but the personal data that may be getting tweaked by analytics engines to produce completely erroneous results.

Although conceptually, cyber-security comprises the entire range of knowledge, tools, or technologies developed to protect networks, computers, programs and data from attack, damage or unauthorized access, in reality, those same technologies and tools can create tremendous security risks by making volumes of real-time data available! So, in a way, cyber security policies or processes act are designed to police the security risks and threats posed by the latest data management and analytics technologies. As mentioned before, cyber-security is often an afterthought in big data applications.

Federal agencies have the additional burden of federal policies on privacy and cyber-security that stop them from harvesting the fullest benefits from big data. Though bid data has managed to revolutionize data gathering, storing, and analysis in businesses, the Federal bodies have remained behind the iron curtain of strong security policies, data- protection policies, and information-sharing standards.

Cyber security and Data Privacy: Big Data and the Law

Deriving conclusions from erroneous data patterns: In big data analytics, very large volumes of data involving many variables have a high probability of displaying bogus patterns or correlations, thereby establishing relationships between variables by the sheer volume of sample data, where such relationships do not exist. These types of spurious results will mislead and misguide decision makers. Often as a result of such erroneous analytics, managers are misled into making wrong decisions.

Huge datasets have two severe limitations:

  1. They evoke false confidence in the predictive capability of analysts.
  2. Results from Big data analytics can be easily manipulated, misinterpreted, or misused by individuals or organizations to present a strong case in their favor

Too much reliance on data: Being overly dependent on data only for decision making can also be risky in a research or business environment. Many factors outside the realm of data, which are as critical as data, such as strategic goals, trust, respect, instinct etc. are often overlooked in an overarching desire to blindly rely on data. The real concern in big data analytics should be a thorough understanding of the data being used to gather insights.

Limitations of big data: Big data, no doubt, has the capability to provide decision-makers with more timely, rich information, but without a deep understanding of the associated  context, analysts will fail to relate the data to the story.

Big data, though powerful, cannot solve all types of business data problems. Big data is more suited to scalable, variable data environments. In many cases, data warehousing and big data have to work together to solve a single business problem.

Building Confidence in Big Data through Context


Konkel, F. (March 11, 2013). Big data’s big hurdle: Federal policy.

  • Top articles, research, podcasts, webinars and more delivered to you monthly.

  • Leave a Comment
    Next Post

    Leave a Reply

    Your email address will not be published. Required fields are marked *

    Big Data, Cloud & DevOps
    Cognitive Load Of Being On Call: 6 Tips To Address It

    If you’ve ever been on call, you’ve probably experienced the pain of being woken up at 4 a.m., unactionable alerts, alerts going to the wrong team, and other unfortunate events. But, there’s an aspect of being on call that is less talked about, but even more ubiquitous – the cognitive load. “Cognitive load” has perhaps

    5 MINUTES READ Continue Reading »
    Big Data, Cloud & DevOps
    How To Refine 360 Customer View With Next Generation Data Matching

    Knowing your customer in the digital age Want to know more about your customers? About their demographics, personal choices, and preferable buying journey? Who do you think is the best source for such insights? You’re right. The customer. But, in a fast-paced world, it is almost impossible to extract all relevant information about a customer

    4 MINUTES READ Continue Reading »
    Big Data, Cloud & DevOps
    3 Ways Businesses Can Use Cloud Computing To The Fullest

    Cloud computing is the anytime, anywhere delivery of IT services like compute, storage, networking, and application software over the internet to end-users. The underlying physical resources, as well as processes, are masked to the end-user, who accesses only the files and apps they want. Companies (usually) pay for only the cloud computing services they use,

    7 MINUTES READ Continue Reading »