Discover where Bad Data Comes From and Learn How to Stop It

Posted · Add Comment
Bad Data

Bad data is the bane of any organization.  And if it isn’t, well it should be.  Why?  Because bad data causes operational inefficiencies, inaccurate business intelligence and forecasts, and can even mess with delivering high-class customer service as well.  In other words, bad data is well… BAD.  But the worst part about bad data is that a lot of times organizations don’t understand where it comes from.  This means bad data is infiltrating your systems and you have no plan to stop it, prevent it, or in most cases find it.  Here are three places where bad data comes from and how to stop it.  Make sure the issue of bad data is tackled as part of your data strategy plan, and ensure that you have the right software to help you track bad data to find it and stop it in those cases it can’t be prevents (i.e. human error) BEFORE it becomes more than an IT headache.

1.  Data Entry Mistakes

Oh the curse of human error!  If only humans could be perfect.  But alas that is not possible.  Data entry is the root cause of most bad data.  According to the Clean Data Blog these mistakes lead to bad data:

  • Entering data into the wrong fields
  • Typos (missing words, letters, html elements, etc.)
  • Using variations on certain data elements

While there is no complete fix for human error, mistakes can be prevented using the following techniques:

  • Providing proper training to employees
  • Monitoring user activity to increase accountability
  • Ensuring you are properly staffed so things are not rushed

2.  Old Data

The problem with old data is it doesn’t give you real-time insight.  That’s because it is out-dated, and degraded, like stale, moldy bread.  Life is fast moving and ever changing, and your data reflects that.  This is often referred to as data erosion.  After all, things change:

  • Customer information changes
    • People move
    • Get new phone numbers
    • Get married
    • Change their names
    • People die, etc.
  • Systems get updates
  • Exchange rates change

So old data doesn’t keep up with the times, or life’s changes, causing inaccuracies and an inability to provide accurate forecasts or insights.  What do you do, especially when you don’t want to just dump all the data you have worked so hard to collect?

  • Archive your data separately so it doesn’t bog down your systems (use a data tracking software like Observato)
  • Make sure the data you use for reporting and BI is fresh and new
  • Do regular system updates

3.  Web Forms

Oh customers and their need for instant gratification.  This leads to incomplete data fields or worse gibberish entered instead of real, actual data.  Not enough space for relevant info or systems and forms that have not been updated to match the business process also pose a challenge.  So to recap, web forms cause bad data when:

  • Inflexible CRMs don’t allow implementation of data rules
  • Systems and forms don’t match business processes
  • Web forms don’t require valid, accurate entry

So how do you prevent the thing that is supposed to make data collection easy from destroying the process?

  • Make sure your web form follows rules (valid phone # required, etc.)
  • Add a captcha to ensure an actual human is filling out the form
  • Make sure you have the proper workflows in place to put captured data in the right places in your systems

These are just three instances of where your bad data can come from. Unfortunately, there are many, many more, including bad data caused by customization and configurations as well as irrelevant or duplicate data you have stored in your systems.  The key for any organization is to be proactive. A proper data strategy can help you prevent bad data before it ever happens in many cases, but you need more than that.  You need visibility in order to see data errors, correct them, and improve your data quality.  A data audit log software can help you track your data, pinpoint bad data, and show you the who, what, where, why and when so you can fix it on the spot and put a plan in place to prevent it in the future.  Find out more below.

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.