The Perils of Predictive Analytics

For centuries people have been captivated by the idea of predicting the future. Crystal ball gazers and fortune tellers all promised to be able to do this. They played on our biases, weaknesses, and gullibility and counted on us attributing chance occurrences to their predictive powers.

The rise of predictive analytics gives us the ability to reduce uncertainty by applying statistics and determining the probabilities that future patterns will emerge in the behaviour of people and systems.

The Internet provides a platform for us to communicate, share, buy, play, and learn. And because people are largely creatures of habit and tend to repeat behaviours, our online activities when combined with today’s computing power and statistical knowledge, tell a lot about what we are likely to do. We can give odds, based on science, about what will most likely occur. To do this has required access to mountains of data about what we do, when we do it, how often we do it, and where we do it.

The rise of predictive analytics gives us the ability to reduce uncertainty says @kwheeler Click To Tweet

By tracking things such as our location, Facebook likes, retweets, where we check-in, what and when we buy, what we search for and so on, analysts are able to make reliable predictions on our future behaviour. This data is often called “data exhaust” by analysts as in and of itself it has no real meaning or value. However, when aggregated, correlated, and combined and then analysed with the tools of statistics this data becomes not only relevant but commercially valuable.

Privacy Invasion

We are being monitored and watched every time we log into any electronic device whether it is a computer, a mobile phone, a tablet, or a game. And everything we do is collected without us being aware. We do not give permission for it to be collected, in most cases, nor do we have any control over what is collected. And we have no way to turn off the monitoring.

For example, when we buy something, it is not hard to predict that we might buy more of it. It is even possible to narrow this down to specific types of items, the amounts we spend and the frequency we buy them. Or when we do something as simple as check into a restaurant or hotel, we leave a location trail as well as an economic trail. Combined with a profession, easily derived from a LinkedIn or Facebook profile, this data can predict with a high degree of certainty where we are likely to be at a given time. It can also predict how often we will be there, what kind of hotels we prefer, perhaps even the type of room we prefer, our income, and much more. And all of this can be sold to an hotelier or retailer, for example, without our knowledge or permission.

Commercialisation that Plays on Our Predilections

shopping-cartPredictive analytics has had tremendous commercial benefits. Firms such as Amazon are built on predictive analytics that help them predict what we will buy, how much of it and when so that they can stock warehouses and order products before they are needed. There has never been so much demand for statisticians and analysts.

Much of the work in developing predictive analytics has been paid for by Madison Avenue, Wall Street, and the retail world. We are marketed to heavily base on our location, age, socio-economic status, and past behaviour. Products are recommended to us based on a prediction about what we are likely to buy.

Shoshana Zuboff, a Harvard professor and no fan of predictive analytics, has focused her research on the study of the rise of the digital world, its individual, organisational, and social consequences, and its relationship to the history and future of capitalism. She is concerned that we are applying analytics to making money and toward turning us all into “slaves” of the commercial world.

She says, in her article entitled A Digital Declaration – “Now the focus has quietly shifted to the commercial monetisation of knowledge about current behaviour as well as influencing and shaping emerging behaviour for future revenue streams. The opportunity is to analyse, predict, and shape while profiting from each point in the value chain.”

Biases that Impede Truth

All humans have biases, and many that tend to impact human resource professionals and recruiters.

The selection and hiring of people are fraught with bias and subjectivity. Psychologists have assembled long lists of these biases which include our tendency to reject new evidence that contradicts something we believe to be true. Or the tendency to search for and remember information in a way that confirms our preconceptions.

For example, if we believe that people with high grades, for example, are better workers, then we will seek evidence to prove that and dismiss any that contradicts it.

Recruiters also often rely too heavily on one trait or piece of information when making decisions – often the first piece of information acquired or the information obtained from a trusted source. If someone recommends a candidate, for example, that recommendation may outweigh any facts that contradict or suggest that the person is not so good.

Analytics can help dispel biases only if we believe in the results of the analytics & act on them Click To Tweet

Many recruiters and hiring managers also suffer from what is called the “Hothand effect” which is the fallacious belief that someone who has experienced success doing something has a greater chance of further success in additional tries.

For a thorough discussion of how bias affects all of us and everything we do, grab a copy of Cathy O’Neil’s Weapons of Math Destruction. This is an insightful analysis of bias in algorithms by a Harvard Ph.D. and ex-hedge fund quant.

Analytics can help dispel many of these, but only if we believe the results of the analytics and act on them. There are many instances where we build our biases unconsciously into the algorithms that analyse our data.

Analytics can offer insight and help make sense of mountains of data that have been beyond our reach. Analytics can help us make choices based on facts. They can provide us insights and reduce uncertainty. But, as with everything, there are dangers. We need to troll the waters of data with care, ethics, and human judgment.

Images: Shutterstock

Leave a Reply