Two Reasons Why Big Data Projects Fail.

by Carla Sidoli on 14 July 2014

Technology analysts predict big data is about to move out of the early-adopter phase and into the business mainstream. It’s certainly possible. We see companies from every sector that are keen to seize data opportunities. But the issue blocking their efforts is often not technology – it’s talent.

 Our experience is that companies can perform better across every area of activity if they make decisions based on robust insights from data analysis. They can seize new opportunities faster, control risks better, and deploy their finances more effectively.

But to turn technological opportunity into business reality, they need the right talent in place – and they need that talent to be directed and managed. This is where data projects often disappoint.

Data projects are hard to deliver

When we talk to CEOs and other senior technology executives we often hear the same lament.

Many of them are keen to take their data projects out of the experimentation phase. They want to roll out the ideas that work and back them with serious investment. They don’t need anyone to convince them about the potential. But they say crunching actionable ideas from a mountain of information is harder than their colleagues from other parts of the business think.

Organisations want to become data- and insight-driven, but they are struggling to get there. Technology remains a big stumbling block. But we think the real issue lies elsewhere.

Companies need to change their approach

Companies need to raise their game in two areas. First, they need their senior leaders to step up and understand the nature of the challenge. It takes resources from across business functions to create an effective data management operation.

The CTO or CIO needs to organise data warehousing. The CFO needs to see a clear business case for investment – especially as the returns will likely be beyond the normal budget horizon. The CMO needs to contribute their knowledge of customer relationships. The chairman needs to mentor the CEO as they guide the board through the issues. Basically, all the senior executives in the business – from the chairman down – must champion the data agenda.

Second, companies need to manage the cultural blocks that can derail data projects.

A data-savvy chief marketing or digital officer is a powerful individual. They will be able to generate new insights about the business. Their analysis might expose underperformance and inefficiency, challenge assumptions, demand new ideas – in fact, that’s a key part of their job.

But this kind of knowledge can be dangerous. It might threaten the position of “old guard” executives. They could resist or stop new data projects in their tracks.

The worry that direct reports are blocking the upward flow of information is the kind of thing that keeps CEOs awake at night. In a well-run organisation, all the executives will welcome the new clarity that data-driven insights can bring to decision-making. Even so, those in charge of implementing data analysis projects must have the sensitivity and interpersonal skills needed to manage this risk.

The right person – in the wrong place

Sadly, this is one of the areas where we often see companies experience “organ rejection”. They appoint a data leader who they honestly believe to be the top candidate. But then all that person’s efforts to deliver change disappoint.

Either they are brought in too far down the organisational structure – at a point where they can’t get the kind of functional cooperation data projects demand. Or they can’t adapt to the cultural demands of the business – their insights are seen as threats and their projects become mired in a swamp of resistance.

The good news is that companies can avoid these pitfalls. It’s often a good idea to appoint a highly qualified interim executive to test the water. And it helps to keep one point in mind: yes, it’s important to select the right data technologies, but it’s just as important to invest in the talent that deploys and directs those technologies. Big data needs big talent.