Companies need to create innovative products adapted to their customers’ much quicker and much more regularly. These days it is the main vector for retaining or increasing market share in a constantly changing economic context. In-depth analysis of market data and consumer behaviours makes it possible to anticipate future requirements. Thus turned towards the identification of future uses, to meet customer expectations or to disrupt the market, a company will be able to take better operational decisions to define its strategy and position itself in its market segment. Data analysis is thus becoming more than ever a major driver of growth. In 2020, corporate investment in data projects is expected to exceed 203 billion dollars worldwide. But at a time when many are claiming to be Data Driven Companies, lots of data projects end in failure. Yet most of these failures are unnecessary and due to well-known causes! Focus on the recurrent pitfalls to avoid.
An approach focused exclusively on the technology
Many data projects concentrate on the implementation of technology solutions (Data Lake, installation of Hadoop clusters, use of a NoSQL database etc.) without considering their purpose, in other words, the requirements or the uses to which they are to be put. The investment is in fact, focused on IT rather than on the operational departments and provides few business benefits. Not all such projects are a waste of money as they at least contribute to developing the technology skills of the IT and IS teams, whilst providing little value at a company level.
Results built on non-industrialisable models
Data Scientists have the required knowledge of models (predictive analysis, machine learning etc.) but usually have little experience of development, especially in an industrial environment. The scripts they provide are often unusable for IT teams. On the IT side, whilst teams are expert in industrialisation aspects, they encounter certain difficulties in understanding the cogs and sequencing of the models proposed by the Data Scientists, whose constraints they don’t always understand. Most data initiatives end up with unsuitable results that are hard to use, both for the analysis of data and for the deployment of the use cases handled. The design of industrial, automatable methods requires a learning process in which the IT department and the Data Science team both participate. This is a preliminary to making effective deployment the rule.
A lack of perspective, analysis and preparation of the company
Companies that introduce data projects often assume that the model and strategies to be implemented are the same as those they have already come across in Business Intelligence projects, thinking that in the end that only the technology and tools will change. This encourages them to retain the same organisation and the same design methods as on previous projects. Yet data science is a discipline based on a prospective approach, with exploratory and trial and error stages. It is impossible to run a project today as we did ten years ago, on the basis of predetermined studies, without a learning phase and without the collaboration of the Data Scientists and IT.
Little collaboration due to in-house cultural differences
The companies, especially the major groups, currently operate with internal silos. There are few synergies between different departments where all work to their own cultures and habits. Confusion might arise when the operational departments, the data science team and IT are working towards distinct and exclusive objectives. Companies lack maturity in respect of the new approaches and working methods, both in terms of the organisation of the work and the way data is handled. To remove this continuing brake on innovation, the various teams must learn to work together more.
An organisation awaiting mobilisation
Despite loading lots of expectations on data projects, top management is still scarcely involved in them. Yet the success of data initiatives also depends on suitable investment in company management. This organisation must be reflected in the entire structure – from management to operational teams – to ensure that the players involved are working towards the same objectives. It must be arranged around a set of common practices capable of developing the skills of every team, from the operational departments to IT.
A data project is a complex initiative, there is a need for suitable architectures to handle the huge quantities of data (structured and non-structured), advanced statistical techniques (proven algorithms, integration of the predictive dimension, training) and access to multiple sources of information (stored and managed in-house or externally, or shared and accessible via initiatives such as Open Data or Open API).
But the key to success is elsewhere, residing in the structuring of the approach. The only way to avoid the usual pitfalls is to bring them to light upstream, by completing each of the various stages (concept, framing of use cases, trialling), within defined timescales and with precise objectives. Completing these three stages will prove the project’s business benefits before entering the industrialisation phase, giving it maximum chances of success!