Avoiding the reference data quagmire

Maxim Lukichev Maxim Lukichev
February 15, 2019 Big Data, Cloud & DevOps

Ready to learn Big Data? Browse Big Data Training and Certification Courses developed by industry thought leaders and Experfy in Harvard Innovation Lab.

A company can put master data management (MDM) in place and make a tremendous effort to ensure its data quality. It can take pains to develop good data governance procedures and invest in leading-edge analytics technologies to make the most of its data. But any errors in its reference data can undermine all of these initiatives.

Reference data is a non-volatile and slow-moving subset of enterprise data. It is often standardized by external bodies and businesses generally use the same reference data throughout their operations. Examples include country codes, SIC codes, currencies and measurement units.

Typically, companies make use of several applications that feature drop-down lists of reference data, and may have various forms that are prefilled with certain reference data. To keep this data synchronized, some companies rely on reference data management or RDM.

experfy-blog

Any inconsistencies in the reference data that a business relies on can result invalid transactions, incorrect reporting, and may require manual fixes that result in higher operational costs. In a recent survey by Accenture, 70 percent of the companies polled acknowledged that reference data quality issues had increased their costs. The study put the total impact at more than $6 Billion for 2015.

These are some of the issues that can result from poorly managed reference data:

1. Higher maintenance costs: Inconsistent reference data across applications results in invalid mapping, which can cause data corruption. Rectifying the problem is costly—especially if transcoding is required during ETL to handle the discrepancies. This may require creating new data objects within MDM or creating custom code.

If mapping procedures aren’t standard and automated, corrective action will be even more expensive. Reference data discrepancies can also affect system stability and extend the time it takes to roll out new applications.

2. Operational Inefficiencies: When reference data such as ZIP codes or country codes don’t match across different applications, transactions may rejected, leading to system-wide alarms and the need for extensive manual intervention. And as companies expand into different international markets, it becomes more challenging to ensure that the business’ refence data remains consistent.

3. Inaccurate analytics and poor data governance: Reference data discrepancies degrade a company’s master data, which affects all of its operational and analytical systems. This will also lead to inaccurate reporting and analytics. And since good data governance depends on standardization to facilitate data mapping, auditing and management, any reference data inconsistencies will significantly undermine these efforts.

4. Non-compliance: Regulatory initiatives such as GDPR require companies to manage all of their internal and external data in an integrated fashion. If a business’ reference data is inconsistent, the requisite data management transparencies are difficult to achieve, exposing the company to a variety of the risk of regulatory and non-compliance risks.

To avoid or correct these issues, a company should make RDM part of its master data management and an integral part of its data strategy.

Managing reference data successfully

To manage its reference data effectively, an organization needs a comprehensive data strategy and a complete RDM solution. A good RDM tool should support locale-specific values plus internationalization for full global compliance. The inclusion of a central, business-user friendly repository can simplify the complex mapping of different data hierarchies, while complete transcoding of incoming data across all operational systems without the use of custom code can cut down on master data maintenance costs.

Other features should include:

  • The ability to define and manage reference data across multiple functional areas
  • Easy integration with existing MDM, applications and third-party data sources
  • Cloud-based deployment for greater scalability, zero downtime and future-readiness
  • An easy to use data steward interface for creating, mapping managing and storing reference data

The software should also offer robust change management, audit controls and workflow functions for reviews, approvals and audit trails.

Reference data anchors a company’s data initiatives, and maintaining consistent, high-quality reference data is essential to successful data management. A sound RDM solution provides enterprise-wide benefits, including lower maintenance costs, greater operational efficiencies, more accurate analytics, reliable data governance and full compliance.

First appeared in Information Management

  • Experfy Insights

    Top articles, research, podcasts, webinars and more delivered to you monthly.

  • Maxim Lukichev

    Tags
    Data Science
    Leave a Comment
    Next Post
    The 2018 Sales Technology Landscape: Your Go-To Sales Tech Guide

    The 2018 Sales Technology Landscape: Your Go-To Sales Tech Guide

    Leave a Reply Cancel reply

    Your email address will not be published. Required fields are marked *

    More in Big Data, Cloud & DevOps
    Big Data, Cloud & DevOps
    Cognitive Load Of Being On Call: 6 Tips To Address It

    If you’ve ever been on call, you’ve probably experienced the pain of being woken up at 4 a.m., unactionable alerts, alerts going to the wrong team, and other unfortunate events. But, there’s an aspect of being on call that is less talked about, but even more ubiquitous – the cognitive load. “Cognitive load” has perhaps

    5 MINUTES READ Continue Reading »
    Big Data, Cloud & DevOps
    How To Refine 360 Customer View With Next Generation Data Matching

    Knowing your customer in the digital age Want to know more about your customers? About their demographics, personal choices, and preferable buying journey? Who do you think is the best source for such insights? You’re right. The customer. But, in a fast-paced world, it is almost impossible to extract all relevant information about a customer

    4 MINUTES READ Continue Reading »
    Big Data, Cloud & DevOps
    3 Ways Businesses Can Use Cloud Computing To The Fullest

    Cloud computing is the anytime, anywhere delivery of IT services like compute, storage, networking, and application software over the internet to end-users. The underlying physical resources, as well as processes, are masked to the end-user, who accesses only the files and apps they want. Companies (usually) pay for only the cloud computing services they use,

    7 MINUTES READ Continue Reading »

    About Us

    Incubated in Harvard Innovation Lab, Experfy specializes in pipelining and deploying the world's best AI and engineering talent at breakneck speed, with exceptional focus on quality and compliance. Enterprises and governments also leverage our award-winning SaaS platform to build their own customized future of work solutions such as talent clouds.

    Join Us At

    Contact Us

    1700 West Park Drive, Suite 190
    Westborough, MA 01581

    Email: support@experfy.com

    Toll Free: (844) EXPERFY or
    (844) 397-3739

    © 2023, Experfy Inc. All rights reserved.