As we deal with the deluge of data that is being collected and presented everyday do we spend too much time striving for perfection of said data when we could be spending our efforts better else where?
That is what I want to talk about. The idea that perfection in data is necessary in some instances but, getting past the potential issues with a set of data can help us to move towards understanding the big picture. What might this big picture be? Sudden drop off in sales of a specific product or product area. Increasingly negative feedback about a product, company or service.
These things do not require that we have perfect data. We rarely need to dig down into the details so much that it matters. What matters is you see a trend. People are not buy that item any more. Why?
You might also notice via the analysis of big data that sales in a certain region are very low compared to another. If you in the snow chain business you will likely sell a lot more in Michigan than say Florida or South California. Now you might see other trends that indicate that there is little movement in certain border snow areas. That might leads to evaluation of additional marketing in those areas. Maybe even evaluation of pricing or product placement. The details come following the identification of a trend. You might even identify untapped market areas or complimentary products you will want to sell.
Big data by its nature does not allow for detailed analysis. You have to much data your dealing with.
So that is where the idea of trying to be to perfect in the data might make you slow to react to a trend.
If your spending to much time worrying about the "quality" of the data you might be missing out on the chance to find a trend or insight. if you delay to long you might even miss the window of opportunity with a product.
One issues brought up was the merging together of multiple databases or sets of data from say 2 companies that just merged. Do you want to wait 6-12 months while you work out the details of getting the data merged together properly. Take the data set independent of each other, find the trends then go dig in the details when you notice some overlap.
Take the rainbow loom for example. If someone just now is deciding they need to jump on this band wagon that might be 6 -12 months late to the show and the opportunity likely has been missed.
As you might start to see anecdotal information indicating its popularity is weaning. New products launch with that information at hand can make a big difference.
You also have to make a concerted effort to look at the origin of the data in question. Are you compiling a bunch of customer data from say web page visits or are you tagging into internal sales figures. This will also help you decide how much time you need to spend making the data "perfect".
A great case example was given of Amazon. They compile piles and piles of data on related searches and click throughs on products. They have become very good at offering complimentary products and related items that might be of interest to the person on the site. They have learned you don't have to be perfect in the suggestions provided. Depending on which way it is being offered you can be fairly general and have a bunch of misses in the pile. Not so bad. On the other hand bundles need to be more on target.
"Amazon is good at this because they don’t worry about everybody. They develop a model where they’re eventually going to get a consistent model of the world, but at the moment they need to do it, they don’t care that they can’t roll it out for everyone. They’ve got hundreds of millions of clicks a day, and they figure, why don’t we just look at 20% of them? The key thing is to do it quickly and to make sure that whatever we conclude, there are many observations for it."
"This is when the term “analytics” becomes interesting. Analytics doesn’t have to be based on super-precise data. That doesn’t again mean wrong data, but it might mean some outcome that wins for the customer. If you profile a jazz CD that people didn’t know they wanted, and some people buy it, great. The fact that some of the 100,000 people that you showed it to didn’t buy that CD is irrelevant."
Sid Probstein:Why Companies Have to Trade “Perfect Data” for “Fast Info”
The whole concept here is whether you worry about getting that last 5-10% of the data right (so you can have some potential increase in sales for example) or just live with what you have and go for the 5-6% increase (generally lesser increase) in sales that can come from that data. The trade off is how much time and effort do you want to spend on "fixing" or getting right the last little bit of data for some small even minute additional increase?
Everything has to be in context. You need perfection in say medical records, and even in medical billing but, do you need that perfection before you can do something. To take action you have to decide at what level is this stuff have to be accurate. Can it be 70-80% or does it have to be 95-100%. Granted due to the ever changing issues with data and the dynamic nature of things you will find a "perfect" set of data may be 98% accurate.
IN some ways are we blessed by all the additional information at our finger tips. Have we gotten away form the big picture?
Think of our ancestors some 300-400 years ago. They may not have had the advantage of satellite imagery to help deal with a storm. They always looked at the big picture and then when they saw a trend they would go for the details so that could properly plan.
We definitely need to get back to this type of analysis. And not be overly jumpy to move on things if just a little bit more time or patience might be in order!
articles worth reading
Why Companies Have to Trade “Perfect Data” for “Fast Info”
Engineering The Perfect Big-Data Bra
Obamacare: a lesson in data entry design
5 Reasons Why Consumer Collaboration and Big Data Are a Perfect Match
Predictive Data Delivering The Perfect Pricing Fit
Healthcare Data’s Perfect Storm
Digging through the Data: How to Separate Quality from Quantity
Buaidh - NO -Bas
No comments:
Post a Comment