I have worked with data for most of my career. After college, I started out as a pricing analyst in the insurance sector. I spent hours poring over data–you could call it big data–to figure out the optimal price for a risk. I would look at historic cost data by a number of segments and try to predict what future costs would be.
In the last couple of years, many companies have been built on the idea of using “big data” and predictive analytics to make better decisions. None of this is new to me. However, my perspective on data has evolved. I have come to the realization that businesses have to evolve beyond pure data. Here’s what we need to know:
- Good judgment still matters.
- “Big data” will not identify all trends.
- Creativity will survive and thrive.
Good judgment still matters.
I am a Twitter user. I like the platform but I don’t love it. It’s noisy and it has some maturing to do. Too many relationships—and I use that word loosely—on thare shallow. People follow to be followed. My “Who to follow” list is littered with the most popular people to follow rather than the most relevant people to follow. And, here’s the thing, Twitter has a TON of data. Someone has actually written the algorithm that fuels my “Who to follow” list. It’s just all wrong.
It is as if the process that is used to process the big data did not take into account any goals for the organization. Quite frankly, I’m not convinced that anyone with big followings on Twitter has done any more than game the system. Follow a few people. Get them to follow you back. Unfollow a few people. Then repeat. (Maybe pay for a few ghost followers too.) Twitter is in the middle of a management shuffle. I suspect they are in search of leadership that empowers, motivates and rewards others to use good judgment. It matters…even when you have big data.
“Big data” will not identify all trends.
Media companies are in a frenzy today over cord-cutting. Subscribers have been falling quarter over quarter for years for many cable and satellite providers. For too many years, the providers’ answer was to raise prices; after all, they needed to replace the lost revenue. Of course, that just led to more subscribers bailing. While the media companies had millions of subscribers and terabyte after terabyte of data, they have been slow to get in on the megatrend of consumers watching video content online. These same companies have been touting their data scientists’ ability to predict blockbusters. Unfortunately, developing more content for online consumption did not showed up in the tea leaves early enough.
Similarly, trends like advances in the sharing economy (think Uber and Airbnb) hit the transportation and hospitality industries a bit late. And, why did Apple rather than a banking institution, with their troves of financial transactional data, innovate on the mobile payment systems first? Perhaps data isn’t really king.
Creativity will survive and thrive.
I have a deep appreciation for data and analysis but it is quite clear that we have to evolve beyond our data. Don’t get me wrong, I love math and science and we have made amazing advances in incorporating both to make better, stronger businesses. But, ultimately, we will stick as much of those advances into a system or process that can generate automated outcomes. What will survive and thrive is creativity. I’m not just talking innovation. Innovation builds upon some that exists. I would say it’s even an outcome of predictive analysis. But those who are creative, those who have an original idea that can be implemented to add value will be those who thrive.
Big data needs big thinking. Take your data gathering, analysis and interpretation to the next level with creativity and good judgment. You might just find yourself ahead of the trend.