The Horrors of Bad Data

Posted · Add Comment

Most organisations don’t realise that bad data is the base problem in every project. Bad data is anything ranging from missing values, malformed records, and improper file formats to unnecessarily collected data (that has no use but takes up space), lost data, and inaccessible data. The problem of big data is far bigger than an IT headache, and will continue to be as data collection within organization grows with big data.

So what can bad data do to your organisation? Think of it as a zombie feasting off of human flesh, but instead of flesh, it feasts off of your resources, your time, your efficiency, and often the health of your systems. In fact, Information Management even refers to data affected by bad data as decaying, stating; “data decays at alarming rates and requires regular and consistent data improvement processes to maintain and/or improve its value.”

Bad data can ruin your customer experience. Read the infographic Is Bad Data a Hazard to Your Customer Experience HERE.
Bad data can corrupt a system. According to Dataflux, “each year, a large number of data warehousing initiatives fail because of erroneous or incomplete data. Inaccurate or incomplete data can lead to faulty decisions once the data warehouse is polluted with bad data.”
Bad data can cost an organisation A LOT of money. The Data Warehousing Institute estimated that data quality problems cost U.S. businesses over $600 billion a year.
So what do organisations to about this FRIGHTFULLY serious issue to that their organisation doesn’t become a little shop of horrors? Well the first thing is to get rid of misconceptions about data quality. That is where Athena IT Solutions can help, as they have compiled a list of seven important, common, misconceptions about data quality, and they are as follows:

1. You Can Fix Data – Don’t make a bigger mess. This can either cause more errors or not solve the root problem; that the issue isn’t an error within the data, it is that the wrong data is being collected.

2. Data Quality is an IT Problem – Data quality affects the entire company and so it is everyone’s problem from IT to business users to analysts and customer service representatives. Data governance must be consistent throughout.

3. The Problem is in the Data Sources or Data Entry – “The larger issue is that you need to manage data from its creation all the way to information consumption. You need to be able to trace its flow from data entry, transactional systems, data warehouse, data marts and cubes all the way to the report or spreadsheet used for the business analysis. Data quality requires tracking, checking and monitoring data throughout the entire information ecosystem.” That is where Realise Data Systems comes into play to help your organization’s data quality with our data tracking software. Download our E-Book here.

4, 5, 6. The Data Warehouse, ERP System, and CPM System will Provide a Single Version of the Truth – “Multiple data silos mean multiple versions of the truth and multiple interpretations of the truth. Data quality has to be addressed across these data silos, not just in the data warehouse.” Again this is where Realise Data Systems can help. We track multiple systems, at the same time, to give you the FULL truth about your data quality.

7. BI Standardisation will Eliminate the Problem of Different “Truths” Represented in the Reports or Analysis – The issue of data tracking still remains the same as does the lack of consistency between data used and data transformations.

So now that we have cured your brainwash, you are probably wondering what you can do about your data quality, and preventing the horrors of bad data from wrecking havoc on your organization.

Well according to Information Management, this is quite simple:

One of the first means of preventing bad data is to examine the three most common problem areas and how these interact when data is moved between them. These typically are:

Business applications – customer relationship management programs, enterprise resource planning, customer information systems, etc.
Movement processes – extract, transform, load
Storage – enterprise data warehouse, integration and analysis
We like to think so too, but because we aren’t in the business of tricks, but rather treats, we would like to talk about one tasty treat right now for your business.

Observato is an independent data audit trail that tracks and archives all your data from multiple systems at the same time. So most of those issues we talked about above are gone quickly. So take 30 minutes out of your day to see what Observato can do for you.

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.