Is the time ripe for appointing a Chief Business Intelligence Officer?

linkedin Business Intelligence Business Intelligence

Once more I have decided to pen this article based on a question that was raised on LinkedIn.com. The group in question on this occasion was Business Intelligence and the thread was entitled Is it time that the CBIO (Chief Business Intelligence Officer) position and organization become commonplace in today’s corporate structure? This was posted by John Thielman.

Standard note: You need to be a member of both LinkedIn.com and the group mentioned to view the discussions.
 
 
The case for a CBIO

The Office of the CBIO

I won’t republish all of John’s initial post, but for those who cannot access the thread these are the essential points that he raised:

  1. There is an ever-increasing need for more and better information in organisations
  2. Increasingly Business Intelligence is seen as a major source of competitive advantage
  3. A CBIO would bring focus and (more importantly) accountability to this area
  4. The CBIO should report directly to the CEO, with strong relations with the rest of the executive team
  5. The CBIO’s team would be a hybrid business / technical one (as I strongly believe the best BI teams should be)
  6. This team should also be at the forefront of driving change, based on the metrics that it generates

Now obviously creating a senior role with a portfolio spanning BI and change is going to be music that falls sweetly on my ears. I did however attempt to be objective in my response, which I reproduce in full below:

As someone who is (primarily) a BI professional, then of course my response could be viewed as entirely self-serving. Nevertheless, I’ll offer my thoughts.

In the BI programmes that I have run, I have had reporting lines into people such as the CIO, CFO or sometimes a combined IT / Operations lead. However (and I think that this is a big however), I have always had programme accountability to the CEO and have always had the entire senior leadership team (business and service departments) as my stakeholders. Generally my direction has come more from these dotted lines than from the solid ones – as you would hope would be the case in any customer-centric IT area.

I have run lots of different IT projects over the years. Things such as: building accounting, purchasing and sales systems; configuring and implementing ERP systems; building front-end systems for underwriters, marketing and executive teams; and so on. Given this background, there is definitely something about BI that makes it different.

Any IT system must be aligned to its users’ needs, that much is obvious. However with BI it goes a long way beyond alignment. In a very real sense, BI systems need to be the business. They are not there to facilitate business transactions, they are there to monitor the heartbeat of the organisation, to help it navigate the best way forward, to get early warning of problems, to check the efficacy of strategies and provide key input to developing them.

In short a good BI system should be focussed on precisely the things that the senior leadership team is focussed on, and in particular what the CEO is focussed on. In order to achieve this you need to understand what makes the business tick and you need to move very close to it. This proximity, coupled with the fact that good BI should drip business value means that I have often felt closer to the overall business leadership team than the IT team.

Please don’t misunderstand my point here. I have been an IT person for 20 years and I am not saying that BI should not be fully integrated with the overall IT strategy – indeed in my book it should be central to it as a major function of all IT systems is to gather information (as well as to support transactions and facilitate interactions with customers). However, there is something of a sense in which BI straddles the IT and business arenas (arenas that I have long argued should be much less distinct from each other than they are in many organisations).

The potentially massive impact of BI, the fact that it speaks the language of business leaders, the need for it to be aligned with driving cultural change and that the fact that the skills required for success in BI are slightly different for those necessary in normal IT projects all argue that something like a CBIO position is maybe not such a bad idea.

Indeed I have begun to see quite a few BI roles that are part of change directorates, or the office of the CEO or CFO. There are also some stand-alone BI roles out there, reporting directing to the board. Clearly there will always be a strong interaction with IT, but perhaps you have detected an emerging trend.

I suppose a shorter version of the above would run something like: my de facto reporting line in BI programmes has always been into the CEO and senior management team, so why not recognise this by making it a de jura reporting line.

BI is a weird combination of being both a specialist and generalist area. Generalist in needing to play a major role in running all aspects of the business, specialist in the techniques and technologies that are key to achieving this.
 
 
Over to the jury

Maybe the idea of a CBIO is one whose time has come. I would be interested in people’s views on this.
 

 

A recording of me being interviewed by Brian Roger of SmartDataCollective.com

SmartDataCollective.com

I have been a featured blogger on SmartDataCollective.com almost as long as I have been a blogger. SDC.com is Social Media Today’s community site, focussed on all aspects of Business Intelligence, Data Warehousing and Analytics, with a pinch of social media thrown in to the mix.

Brian Roger, the SDC.com editor, was recently kind enough to interview me about my career in BI, the challenges I have faced and what has helped to overcome these. This interview is now available to listen to as part of their Podcast series – click on the image below to visit their site.

sdc-podcast

SmartDataCollective.com Intervew

I would be interested in feedback about any aspect of this piece, which I am grateful to Brian for arranging.
 


 
Social Media Today LLC helps global organizations create purpose-built B2B social communities designed to achieve specific, measurable corporate goals by engaging exactly the customers and prospects they most want to reach. Social Media Today helps large companies leverage the enormous power of social media to build deeper relationships with potential customers and other constituencies that influence the development of new business. They have found that their primary metrics of success are levels of engagement and business leads. One thousand people who come regularly and might buy an SAP, Oracle or Teradata system some day is better than a million people who definitely won’t.

Social Media Today LLC, is a battle-tested, nimble team of former journalists, online managers, and advertising professionals who have come together to make a new kind of media company. With their backgrounds, and passions for, business-to-business and public policy conversations, they have decided to focus their efforts in this area. To facilitate the types of convresations that they would like to see Social Media Today is assembling the world’s best bloggers and providing them with an independent “playground” to include their posts, to comment and rate posts, and to connect with each other. On their flagship site, SocialMediaToday.com, they have brought together many of the most intriguing and original bloggers on media and marketing, covering all aspects of what makes up the connective tissue of social media from a global perspective.
 

A single version of the truth?

linkedin The Data Warehousing Institute The Data Warehousing Institute (TDWI™) 2.0

As is frequently the case, I was moved to write this piece by a discussion on LinkedIn.com. This time round, the group involved was The Data Warehousing Institute (TDWI™) 2.0 and the thread, entitled Is one version of the truth attainable?, was started by J. Piscioneri. I should however make a nod in the direction of an article on Jim Harris’ excellent Obsessive-Compulsive Data Quality Blog called The Data Information Continuum; Jim also contributed to the LinkedIn.com thread.

Standard note: You need to be a member of both LinkedIn.com and the group mentioned to view the discussions.
 
 
Introduction

A Calabi–Yau manifold

Here are a couple of sections from the original poster’s starting comments:

I’ve been thinking: is one version of the truth attainable or is it a bit of snake oil? Is it a helpful concept that powerfully communicates a way out of spreadmart purgatory? Or does the idea of one version of the truth gloss over the fact that context or point of view are an inherent part of any statement about data, which effectively makes truth relative? I’m leaning toward the latter position.

[…]

There can only be one version of the truth if everyone speaks the same language and has a common point of view. I’m not sure this is attainable. To the extent that it is, it’s definitely not a technology exercise. It’s organizational change management. It’s about changing the culture of an organization and potentially breaking down longstanding barriers.

Please join the group if you would like to read the whole post and the subsequent discussions, which were very lively. Here I am only going to refer to these tangentially and instead focus on the concept of a single version of the truth itself.

Readers who are not interested in the ellipitcal section of this article and who would instead like to cut to the chase are invited to click here (warning there are still some ellipses in the latter sections).
 
 
A [very] brief and occasionally accurate history of truth

The demise of a cherry tree

I have discovered a truly marvellous proof of the nature of truth, which this column is too narrow to contain.

— Pierre de Tomas (1637)

Instead of trying to rediscover M. Tomas’ proof, I’ll simply catalogue some of the disciplines that have been associated (rightly or wrongly) with trying to grapple with the area:

  • Various branches of Philosophy, including:
    • Metaphysics
    • Epistemology
    • Ethics
    • Logic
  • History
  • Religion (or more perhaps more generally spirituality)
  • Natural Science
  • Mathematics
  • and of course Polygraphism

Lie algebra

Given my background in Pure Mathematics the reader might expect me to trumpet the claims of this discipline to be the sole arbiter of truth; I would reply yes and no. Mathematics does indeed deal in absolute truth, but only of the type: if we assume A and B, it then follows that C is true. This is known as the axiomatic approach. Mathematics makes no claim for the veracity of axioms themselves (though clearly many axioms would be regarded as self-evidently true to the non-professional). I will also manfully resist the temptation to refer to the wrecking ball that Kurt Gödel’s took to axiomatic systems in 1931.

Physical science

I have also made reference (admittedly often rather obliquely) to various branches of science on this blog, so perhaps this is another place to search for truth. However the Physical sciences do not really deal in anything as absolute as truth. Instead they develop models that approximate observations, these are called scientific theories. A good theory will both explain aspects of currently observed phenomena and offer predictions for yet-to-be-observed behaviour (what use is a model if it doesn’t tell us things that we don’t already know?). In this way scientific theories are rather like Business Analytics.

Unlike mathematical theories, the scientific versions are rather resistant to proof. Somewhat unfairly, while a mountain of experiments that are consistent with a scientific theory do not prove it, it takes only one incompatible data point to disprove it. When such an inconvenient fact rears its head, the theory will need to be revised to accommodate the new data, or entirely discarded and replaced by a new theory. This is of course an iterative process and precisely how our scientific learning increases. Warning bells generally start to ring when a scientist starts to talk about their theory being true, as opposed to a useful tool. The same observation could be made of those who begin to view their Business Analytics models as being true, but that is perhaps a story for another time.

The Thinker

I am going to come back to Physical science (or more specifically Physics) a little later, but for now let’s agree that this area is not going to result in defining truth either. Some people would argue that truth is the preserve of one of the other subjects listed above, either Philosophy or Religion. I’m not going to get into a debate on the merits of either of these views, but I will state that perhaps the latter is more concerned with personal truth than supra-individual truth (otherwise why do so many religious people disagree with each other?).

Discussing religion on a blog is also a certain way to start a fire, so I’ll move quickly on. I’m a little more relaxed about criticising some aspects of Philosophy; to me this can all too easily descend into solipism (sometimes even quicker than artificial intelligence and cognitive science do). Although Philosophy could be described as the search for truth, I’m not convinced that this is the same as finding it. Maybe truth itself doesn’t really exist, so attempting to create a single version of it is doomed to failure. However, perhaps there is hope.
 
 
Trusting your GUT feeling

Physicists have a sense of humour too you know...
© xkcd.com

After the preceding divertimento, it is time to return to the more prosaic world of Business Intelligence. However there is first room for the promised reference to Physics. For me, the phrase “a single version of the truth” always has echoes of the search for a Grand Unified Theory (GUT). Analogous to our discussions about truth, there are some (minor) definitional issues with GUT as well.

Some hold that GUT applies to a unification of the electromagnetic, weak nuclear and strong nuclear forces at very high energy levels (the first two having already been paired in the electroweak force). Others that GUT refers to a merging of the particles and forces covered by the Standard Model of Quantum Mechanics (which works well for the very small) with General Relativity (which works well for the very big). People in the first camp might refer to this second unification as a ToE (Theory of Everything), but there is sometimes a limit to how much Douglas Adams’ esteemed work applies to reality.

For the purposes of this article, I’ll perform the standard scientific trick of a simplifying assumption and use GUT in the grander sense of the term.

Scientists have striven to find a GUT for decades, if not centuries, and several candidates have been proposed. GUT has proved to be something of a Holy Grail for Physicists. Work in this area, while not as yet having been successful (at least at the time of writing), has undeniably helped to shed a light on many other areas where our understanding was previously rather dim.

This is where the connection with a single version of the truth comes in. Not so much that either concept is guaranteed to be achievable, but that a lot of good and useful things can be accomplished on a journey towards both of them. If, in a given organisation, the journey to a single version of the truth reaches its ultimate destination, then great. However if, in an another company, a single version of the truth remains eternally just over the next hill, or round the next corner, then this is hardly disastrous and maybe it is the journey itself (and the aspirations with which it is commenced on) that matters more than the destination.

Before I begin to sound too philosophical (cf. above) let me try to make this more concrete by going back to our starting point with some Mathematics and considering some Venn diagrams.
 
 
Ordo ab chao

In my experience the following is the type of situation that a good Business Intelligence programme should address:

Fragmentation

The problems here are manifold:

  1. Although the various report systems are shown as separate, the real situation is probably much worse. Each of the reporting and analysis systems will overlap, perhaps substantially, with one or more or the other ones. Indeed the overlapping may be so convoluted that it would be difficult to represent this in two dimensions and I am not going to try. This means that you can invariably ask the same question (how much have we sold this month) of different systems and get different answers. It may be difficult to tell which of these is correct, indeed none of them may be a true reflection of business reality.
  2. There are a whole set of things that may be treated differently in the different ellipses. I’ll mention just two for now: date and currency. In one system a transaction may be recorded in a month when it is entered into the system. In another it may be allocated to the month when the event actually occurred (sometimes quite a while before it is entered). In a third perhaps the transaction is only dated once it has been authorised by a supervisor.

    In a multi-currency environment reports may be in the transactional currency, rolled-up to the currency of the country in which they occurred, or perhaps aggregated across many countries in a number of “corporate” currencies. Which rate to use (rate on the day, average for the month, rolling average for the last year, a rate tied to some earlier business transaction etc.) may be different in different systems, equally the rate may well vary according to the date of the transaction (making the last set of comments about which date is used even more pertinent).

  3. A whole set of other issues arise when you begin to consider things such as taxation (are figures nett or gross), discounts, commissions to other parties, phased transactions and financial estimates. Some reports may totally ignore these, others my take account of some but not others. A mist of misunderstanding is likely to arise.
  4. Something that is not drawn on the above diagram is the flow of data between systems. Typically there will be a spaghetti-like flow of bits and bytes between the different areas. What is also not that uncommon is that there is both bifurcation and merging in these flows. For example, some sorts of transactions from Business Unit A may end up in the Marketing database, whereas others do not. Perhaps transactions carried out on behalf of another company in the group appear in Business Unit B’s reports, but must be excluded from the local P&L. The combinations are almost limitless.

    Interfaces can also do interesting things to data, re-labelling it, correcting (or so their authors hope) errors in source data and generally twisting the input to form output that may be radically different. Also, when interfaces are anything other than real-time, they introduce a whole new arena in which dates can get muddled. For instance, what if a business transaction occurred in a front-end system on the last day of a year, but was not interfaced to a corporate database until the first day of the next one – which year does it get allocated to in the two places?

  5. Finally, the above says nothing about the costs (staff and software) of maintaining a heterogeneous reporting landscape; or indeed the costs of wasted time arguing about which numbers are right, or attempting to perform tortuous (and ultimately fruitless) reconciliations.

Now the ideal situation is that we move to the following diagram:

De-fragmentation

This looks all very nice and tidy, but there are still two major problems.

  1. A full realisation of this transformation may be prohibitively expensive, or time-consuming.
  2. Having brought everything together into one place offers an opportunity to standardise terminology and to eliminate the confusion caused by redundancy. However, it doesn’t per se address the other points made from 2. onwards above.

The need to focus on what is possible in a reasonable time-frame and at a reasonable cost may lead to a more pragmatic approach where the number of reporting and analysis systems is reduced, but to a number greater than one. Good project management may indeed dictate a rolling programme of consolidation, with opportunities to review what has worked and what has not and to ascertain whether business value is indeed being generated by the programme.

Nevertheless, I would argue that it is beneficial to envisage a final state for the information architecture, even if there is a tacit acceptance that this may not be realised for years, if at all. Such a framework helps to guide work in a way that making it up as we go along does not. I cover this area in more detail in both Holistic vs Incremental approaches to BI and Tactical Meandering for those who are interested.

It is also inevitable that even in a single BI system data will need to be presented in different ways for different purposes. To take just one example, if you goal is to see how the make up of a book of business has varied over time, then it is eminently sensible to use a current exchange rate for all transactions; thereby removing any skewing of the figures caused by forex fluctuations. This is particularly the case when trying to assess the profitability of business where revenue occurs at a discrete point in the past, but costs may be spread out over time.

However, if it is necessary to look at how the organisation’s cash-flow is changing over time, then the impact of fluctuations in foreign exchange rates must be taken into account. Sadly if an American company wants to report how much revenue it has from its French subsidiary then the figures must reflect real-life euro / dollar rates (unrealised and realised foreign currency gains and losses notwithstanding).

What is important here is labelling. Ideally each report should show the assumptions under which it has been compiled at the top. This would include the exchange rate strategy used, the method by which transactions are allocated to dates, whether figures are nett or gross and which transactions (if any) have been excluded. Under this approach, while it is inevitable that the totals on some reports will not agree, at least the reports themselves will explain why this is the case.

So this is my take on a single version of the truth. It is both a) an aspirational description of the ideal situation and something that is worth striving for and b) a convenient marketing term – a sound-bite if you will – that presents a palatable way of describing a complex set of concepts. I tried to capture this essence in my reply to the LinkedIn.com thread, which was as follows:

To me, the (extremely hackneyed) phrase “a single version of the truth” means a few things:

  1. One place to go to run reports and perform analysis (as opposed to several different, unreconciled, overlapping systems and local spreadsheets / Access DBs)
  2. When something, say “growth” appears on a report, cube, or dashboard, it is always calculated the same way and means the same thing (e.g. if you have growth in dollar terms and growth excluding the impact of currency fluctuations, then these are two measures and should be clearly tagged as such).
  3. More importantly, that the organisation buys into there being just one set of figures that will be used and self-polices attempts to subvert this with roll-your-own data.

Of course none of this equates to anything to do with truth in the normal sense of the word. However life is full of imprecise terminology, which nevertheless manages to convey meaning better than overly precise alternatives.

More’s Utopia was never intended to depict a realistic place or system of government. These facts have not stopped generations of thinkers and doers from aspiring to make the world a better place, while realising that the ultimate goal may remain out of reach. In my opinion neither should the unlikelihood of achieving a perfect single version of the truth deter Business Intelligence professionals from aspiring to this Utopian vision.

I have come pretty close to achieving a single version of the truth in a large, complex organisation. Pretty close is not 100%, but in Business Intelligence anything above 80% is certainly more than worth the effort.
 

I will be giving a Business Intelligence Masterclass at “Business Process Excellence in Financial Services” London, September 22-24

Business Process Excellenece in Financial Services
 
This event will be held in London’s Canary Wharf and has the strap-line: “Improving Business Agility and Performance Whilst Reducing Cost and Complexity”.

A selection of the organisations that seminar speakers work for appears below:
 

Axa Citi Co-operative Financial Services
Deutsche Bank First Direct HSBC
Kleinwort Benson Lloyds TSB Royal Bank of Scotland
Union Bancaire Privée UniCredit Group  

 
If you would like to find out more about this event then there are a variety of ways to do this:
 

Freephone Freephone: 0800 652 2363 or +44 (0)20 7368 9300
Fax Fax: +44 (0)20 7368 9301
Mail Mail: IQPC Ltd. Anchor House
15-19 Britten Street
London SW3 3QL
Internet Internet: www.bpefinance.com
e-mail e-mail: enquire@iqpc.co.uk

I hope to maybe see some of you there.
 


 
A selected list of my previous public speaking may be viewed here.
 

“Involving users in business intelligence strategy key for success” – Christina Torode on SearchCio-Midmarket.com

Search CIO Midmarket Christina Torode

While browsing the, slightly idiosyncratically named, infoBOOM! Must-know people, ideas and opinions for mid-sized business group on LinkedIn.com today, I came across a link to an artcile about Business Intelligence on SearchCio-Midmarket.com (part of the TechTarget stable). This was by Christina Torode and is entitled, Involving users in business intelligence strategy key for success (please note that registration is required to read the full article, though this is a relatively painless process).

Christina cites the opinions of a number of industry experts and practitioners (as one would expect, the latter are mostly from the mid-market) in making her case, which is one that I pretty much agree with. These include: Boris Evelson at Forrester Research; Rob Fosnaugh, BI lead at Brotherhood Mutual Insurance Co.; and Chris Brady, CIO at Dealer Services Corp. The experiences of Rob and Chris in particular provide some useful pointers to techniques that may be appropriate for you to use in your own BI projects.

Commitment vs Involvement

I do however have one minor quibble. This is to do with the use of the word “involvement” in this context. Some of my concern may be explained by recourse to a dictionary.

  involve /invólv/ v.tr. 1 (often foll. by in) cause (a person or thing) to participate, or share the experience or effect (in a situation, activity, etc.). (O.E.D.)  

The point that I want to make is perhaps more clearly stated in the rather earthy adage about the difference between involvement and commitment relating to breakfast; this being that a chicken was involved with it, but a pig was committed to it.

To me involving business people in a BI project is not enough. It implies that IT is in the driving seat and that the project is essentially a technological one. Instead what I believe is required is a full partnership. I have written about the lengths that I have gone to in trying to achieve this in Scaling-up Performance Management and Developing an international BI strategy.

Aside: It is worth noting that the former of these articles covers a 9-month collaboration with 30 business people to define the overall BI needs of an insurance organisation in 13 European countries. This contrasts with a 2-month process at another (rather different) insurance organisation, Brotherhood Mutual, that Christina cites.

I should mention that the exercise I describe resulted in nine major reporting and analysis areas (chronologically: Profitability, Broker Management, Claims Management, Portfolio Management, Budget Management, Dashboard, Expense Management, Exposure Management and Service & Workflow) as opposed to a single one (Claims) at Brotherhood Mutual; so maybe the durations are comparable.

Either way the main lesson is that it takes time to get good requirements in BI.

The real-life examples that Christina mentions in her article seem to also lean a little more towards partnership / commitment than to involvement. It may seem that I am splitting hairs on this issue (maybe this is a byproduct of the things that I learnt about semantics yesterday), but I have seen BI projects fail to deliver on their promise specifically because the IT team became too internally focussed and lost touch with their business users after an initial (and probably inadequately thorough) requirement-gathering exercise.

Indeed I was once brought in to act as an internal consultant for a failing BI project and my main diagnosis was precisely that the business people were semi-detached from it. They had been “involved”, but this was never to a great degree and had also occurred some time in the past. My recommendation was ongoing and in-depth collaboration, to the degree that the BI team becomes a joint IT / business one with the distinctions between people’s roles blurring somewhat at the edges.

This partnership approach has worked for me (the results may be viewed here) and I have seen the lack of an IT / business partnership lead to failure in BI on a number of occasions. Rather than being the minor point I initially mentioned, I think that the difference between involvement and commitment can be make or break for a BI project.
 


 
Christina Torode has been a high tech journalist for more than a decade. Before joining TechTarget, she was a reporter for technology trade publication CRN covering a variety of beats from security and networking to telcos and the channel. She also spent time as a business reporter and editor with Eagle Tribune Publishing in eastern Massachusetts. For SearchCIO.com and SearchCIO-Midmarket, Christina covers business applications and virtualization technologies.
 

“Does Business Intelligence Require Intelligent Business?” by George M. Tomko

CIO Rant George M Tomko

Introduction

George Tomko’s CIO Rant has been on my list of recommended sites for quite some time. I also follow George on twitter.com (http://twitter.com/gmtomko) and have always found his perspective on business and technology matters to be extremely interesting and informative.

George’s latest blog post is is on a subject that is clearly close to my heart and is entitled Does Business Intelligence Require Intelligent Business? I should also thank him for quoting my earlier artcile, Data – Information – Knowledge – Wisdom, in this. Being mentioned in the same breath as Einstein is always gratifying as well!

George acknowledges that this is something of a “What comes first – the chicken or the egg?” situation. He starts out by building on an article by Gerry Davis at Heidrick & Struggles to state:

  1. collecting [information about customers] is “easy”
  2. analyzing it is hard
  3. disseminating it is very hard

Kudos to the first reader to correctly identify the mountain

Both George and Gerry agreed that the mountains of data that many organisations compile are not always very effectively leveraged to yield information, let alone knowledge or wisdom. Gerry proposes:

identifying and appointing the right executive — someone with superb business acumen combined with a sound technical understanding — and tasking them with delivering real business intelligence

George assesses this approach through the prism of the the three points listed above and touches on the ever present challenges of business silos; agreeing that the type of executive that Gerry recommends appointing could be effective in acting across these. However he introduces a note of caution, suggesting that it may be more difficult than ever to kick-off cross-silo initiatives in today’s turbulent times.

I tend to agree with George on this point. Crises may deliver the spark necessary for corporate revolution and unblock previously sclerotic bureaucracies. However, they can equally yield a fortress mentality where views become more entrenched and any form or risk taking or change is frowned upon. The alternative is incrementalism, but as George points out, this is not likely to lead to a major improvement in the “IQ” of organisations (this is an area that I cover in more detail in Holistic vs Incremental approaches to BI).
 
 
The causality dilemma

Which came first?

Returning to George’s chicken and egg question, do intelligent enterprises build good business intelligence, or does good business intelligence lead to more intelligent enterprises? Any answer here is going to vary according to the organisations involved, their cultures, their appetites for change and the environmental challenges and evolutionary pressures that they face.

Having stated this caveat, my own experience is of an organisation that was smart enough to realise that it needed to take better decisions, but maybe not aware that business intelligence was a way to potentially address this. I spoke about this as one of three sceanrios in my recent artcile, “Why Business Intelligence projects fail”. Part of my role in this organisation (as well as building a BI team from scratch and developing a word-class information architecture) was to act as evangelist the benefits of BI.

The work that my team did in collaboration with a wide range of senior business people, helped an organisation to whole-heartedly embrace business intelligence as a vehicle to increasing its corporate “IQ”. Rather than having this outcome as a sole objective, this cultural transfomation had the significant practical impact of strongly contributing to a major business turn-around from record losses over four years, to record profits sustained over six. This is precisely the sort of result that well-designed, well-managed BI that addresses important business questions can (and indeed should) deliver.
 
 
Another sporting analogy

I suppose that it can be argued that only someone with a strong natural aptitude for a sport can become a true athlete. Regardless of their dedication and the amount of training they undertake, the best that lesser mortals can aspire to is plain proficiency. However, an alternative perspective is that it is easy enough to catalogue sportsmen and women who have failed to live up to their boundless potential, where perhaps less able contemporaries have succeeded through application and sheer bloody-minded determination.

I think the same can be said of the prerequisites for BI success and the benefits of successful BI. Organisations with a functioning structure, excellent people at all levels, good channels of communication and a clear sense of purpose are set up better to succeed in BI than their less exemplary competitors (for the same reason that they are set up better to do most things). However, with sufficient will-power (which may initially be centred in a very small group of people, hopefully expanding over time), I think that it is entirely possible for any organisation to improve what it knows about its business and the quality of the decisions it takes.

Good Business Intelligence is not necessarily the preserve of elite organisations – it is within the reach of all organisations who possess the minimum requirements of the vision to aspire to it and the determination to see things through.
 


 
George M. Tomko is CEO and Executive Consultant for Tomko Tek LLC, a company he founded in 2006. With over 30 years of professional experience in technology and business, at the practitioner and executive levels, Mr. Tomko’s goal is to bring game-changing knowledge and experience to client organizations from medium-size businesses to the multidivisional global enterprise.

Mr. Tomko and his networked associates specialize in transformational analysis and decision-making; planning and execution of enterprise-wide initiatives; outsourcing; strategic cost management; service-oriented business process management; virtualization; cloud computing; asset management; and technology investment assessment.

He can be reached at gtomko@tomkotek.com
 

“Why Business Intelligence projects fail”

Introduction

James Anderson bowls Sachin Tendulkar for 1 - England v India, 3rd Test, The Oval, August 12, 2007

In this blog, I have generally tried to focus on success factors for Business Intelligence programmes. I suppose this reflects my general character as something of an optimist. Of course failure can also be instructive; as the saying goes “we learn more from our mistakes than from our successes.” Given this, and indeed the Internet’s obsession with “x reasons why y fails”, I have also written on the subject of how Business Intelligence projects can go wrong a few times.

My first foray into this area was in response to coverage of the Gartner Business Intelligence Summit in Washington D.C. by Intelligent Enterprise. The article was entitled, “Gartner sees a big discrepancy between BI expectations and realities” – Intelligent Enterprise; I have always had a way with words!

Rather than simply picking holes in other people’s ideas on this topic, I then penned a more general piece, Some reasons why IT projects fail, which did what it said on the can. Incidentally I was clearly in the middle of a purple patch with respect to article headlines back then, I am trying to recapture some of these former titular glories in this piece.

So what has moved me to put fingertip to keyboard this time? Well my inspiration (if it can be so described) was, as has often been the case recently, some comments made on a LinkedIn.com forum. In breech of my customary practice, I am not going to identify the group, the discussion thread or the author of these comments, which were from a seasoned BI professional and were as follows:

Most BI projects fail because:

a) the business didn’t support it properly or
b) the business didn’t actually know what they wanted

I realise that I should be more emotionally mature about such matters, but comments such as these are rather like a red rag to a bull for me. Having allowed a few days for my blood pressure to return to normal, I’ll try to offer a dispassionate deconstruction of these suggestions, which I believe are not just incorrect, but dangerously wrong-headed. If the attitude of a BI professional is accurately reflected by the above quote, then I think that we need look no further for why some BI projects fail. Let’s look at both assertions in a little more detail.
 
 
What does “the business didn’t properly support [BI]” actually mean?

The British and Irish Lions scrum in South Africa - 1997

For a change I am going to put my habitual frustration at unhelpful distinctions between “the business” and “IT” to one side (if you want to read about my thoughts on this matter, then please take a look at the Business / IT Alignment and IT Strategy section of my Keynote Articles page).

To me there are three possible explanations here:

  1. The business did not need or want better information
  2. The business needed or wanted better information, initially supported the concept of BI delivering this, but their enthusiasm for this approach waned over time
  3. The business needed or wanted better information, but didn’t think that BI offered the way to deliver this

maybe there are some other possibilities, but hopefully the above covers all the bases. Here are some thoughts on each scenario:
 
 
1. The business did not need or want better information

So the rationale for starting a BI project was… ?

On a more serious note, there could be some valid reasons why a BI project would still make sense. First of all, it might be that a BI system could deliver the same information that is presently provided more cheaply. This could be the case where there are multiple different reporting systems, each needing IT care and support; where the technology that existing systems are based on is going out of support; or where BI is part of a general technology refresh or retrenchment (for example replacing fat client reporting tools with web-based ones, thereby enabling the retirement of multiple distributed servers and saving on the cost of people to maintain them). Of course it may well be IT that highlights the needs in these circumstances, but there should surely be a business case developed with some form or payback analysis or return on investment calculation. If these stand up, then an IT-centric BI project may be justifiable. While I would argue that such an approach is a wasted opportunity, if such an initiative is done properly (i.e. competently managed), then the the lack of business people driving the project should not be an obstacle to success.

Second, it could be that the business saw no need for better information, but IT (or possibly IT in conjunction with a numerate department such as Finance, Change or Strategy) does see one. While this observation is perhaps tinged with a little arrogance, one of IT’s roles should be to act as an educator, highlighting the potential benefits of technology and where they may add business advantage. Here my advice is not to start a BI project and hope that “if we build it, they will come”, but instead to engage with selected senior business people to explain what is possible and explore whether a technology such as BI can be of assistance. If this approach generates interest, then this can hopefully lead to enthusiasm with appropriate nurturing. Of course if the answer is still that the current information is perfectly adequate, then IT has no business trying to kick off a BI project by itself. If it does so, then accountability for its likely failure will be squarely (and fairly) laid at IT’s door.
 
 
2. The business needed or wanted better information, initially supported the concept of BI delivering this, but their enthusiasm for this approach waned over time

To me this sounds like a great opportunity that the BI team have failed to capitalise on. There is a pressing business need. There is a realisation that BI can meet this. Funding is allocated. A project is initiated. However, some where along the line, the BI team have lost their way.

Why would business enthusiasm wane? Most likely because delivery was delayed, no concrete results were seen for a long time, costs ballooned, the system didn’t live up to expectations, or something else happened that moved executive focus from this area. In the final case, then any responsible manager should be prepared to cut their coat according to their cloth. The BI team may feel that their project is all-important, but it is not inconceivable that another project would take precedence, for example integrating a merger.

Assuming that external events are not the reason for business disenchantment, then all of the other reasons are 100% the responsibility of the BI team themselves. BI projects are difficult to estimate accurately as I described in The importance of feasibility studies in business intelligence, but – as the same article explains – this is not an excuse for drastically inaccurate project plans or major cost overruns. Also the BI team should work hard at the beginning of the project to appropriately set expectations. As with any relationship, business or personal, the key to success is frequent and open communication.

Equally, BI projects often require a substantial amount of time to do well (anyone who tells you the opposite has never been involved with one or is trying to sell you something). This does not mean that the BI team should disappear for months (or years) on end. It is important to have a parallel stream of interim releases to address urgent business needs, provide evidence of progress and burnish the team’s credibility (I explore this area further in Holistic vs Incremental approaches to BI).

If the BI system delivered does not live up to expectations then there are two questions to be answered. In what way does it not meet expectations? and Why did it take until implementation to determine this? It could be that the functionality of the BI tool does not meet what is necessary, but most of these have a wide range of functionality and are at least reasonably intuitive to use. More likely the issue is in the information presented in the tool (which is not judged to be useful) or in an inadequate approach to implementation. The way to address both of these potential problems from the very start of the project is to follow the four-pillared approach that I recommend in many places on this blog; notably in one of the middle sections of Is outsourcing business intelligence a good idea?.

So rather than blaming the business for losing interest in BI, the BI team needs to consider where its own inadequacies have led to this problem. It is sometimes tempting to dwell on how no one really appreciates all of the hard work that us IT types do, but it is much more productive to try to figure out why this is and take steps to address the problem.
 
 
3.The business needed or wanted better information, but didn’t think that BI offered the way to deliver this

While I recognise some aspects of the first scenario above, this one is something that I am more intimately familiar with. Back in 2000, I was charged with improving the management information of a large organisation, in response to profitability issues that they were experiencing. No one mentioned data warehouses, or OLAP, or analytics. A business intelligence implementation was my response to the strategic business challenges that the organisation was facing. However I initially faced some scepticism. It was suggested that maybe I was over-engineering my approach (the phrase “we need a diesel submarine, not a nuclear one” being mentioned) when all that was required was a few tweaks to existing reports and writing some new ones.

First of all, a BI professional should welcome such challenges. Indeed they should continually ask the same questions of themselves and their team. If your proposed approach does not stand up to basic scrutiny with respect to cost effectiveness and timeliness, then you are doing a poor job. However good BI people will be able to answer such questions positively, having devised programmes and architectures that are appropriate for the challenges that they seek to meet. A BI solution should clearly not be more expensive than is needed, but equally it should not be cheaper, lest it fails to deliver anything.

A skill that is required in a situation such as the one I found myself in back in 2000 is to be able to lay out your vision and proposals in a way that is logical, compelling, attractive and succinct enough to engage business enthusiasm and engender the confidence of your potential stakeholders. In short you need to be able to sell. Maybe this is not a skill that all IT people have acquired over the years, but it is invaluable in establishing and maintaining project momentum (I cover the latter aspect of this area in three articles starting with Marketing Change).

Again, if the lead of the BI team is not able to properly explain why BI is the best way to meet the information needs of the organisation, then this is essentially the fault of the BI lead and not the business.
 
 
So far, the main conclusion that I have drawn is the same as in my earlier piece about the failure of BI projects. I closed this by stating:

I firmly believe that BI done well is both the easiest of IT systems to sell to people and has one of the highest paybacks of any IT initiative. BI done badly (at the design, development, implementation or follow-up stages) will fail.

The issue is basically a simple one: just how good is your BI team? If a BI implementation fails to deliver significant business value, then instead of looking for scape-goats, the BI team should purchase a mirror and start using it.

Let’s see if an exploration of the second suggested reason for problems with BI projects changes my stance at all.
 
 
What does “the business didn’t know what they wanted [from BI]” actually mean?

What do I need

Business intelligence, when implemented correctly, helps organisations to be more successful by offering a way to understand the dynamics of their operations and markets and facilitating better business decisions. So, almost by definition, good BI has to be a sort of model of the key things that happen in an company. This is not easy to achieve.

Again I will come back to my four-pillared approach and emphasise the first pillar. There is an imperative to:

Form a deep understanding of the key business questions that need to be answered.

In my opinion, it is the difficulty in managing this process that plays into the assertion that “the business didn’t know what they wanted.” Putting it another way, the BI team were not skilful enough, engaging enough, or business-savvy enough to help the business to articulate what they wanted and to translate this into a formal set of definitions that could then form the basis of IT work. This process can indeed lengthy, tough and difficult to get right because:

  1. Businesses are often complex, with many moving parts and many things that need to be measured
  2. Different business people may have different visions of what is important – each of these may have validity, depending on context
  3. Both IT and business people may be unaccustomed to talking about business phenomena in the required way (one that is self-consistent and exhaustive)
  4. IT may not have a proper understanding of business strategy, business terminology and business transactions
  5. There is often a desire to start with the current state and adapt / add to this, rather than take the more arduous (but more profitable) approach of working out what is necessary and desireable
  6. The process is typically iterative and requires an ongoing commitment to the details, sapping reserves of perseverance

I have written about the level of commitment that can be required in defining BI business requirements in a couple of articles: Scaling-up Performance Management and Developing an international BI strategy. Please take a look at these if you are interested in delving further into this area. For now it is enough to state that you should probably allow a number of months for this work in your BI project plan; more if your objective is to deliver an all-pervasive BI system (as it was in the work I describe in the articles). It is also helpful to realise that you are never “done” with requirements in BI, they will evolve based on actual use of the system and changing business needs. You will end up living this cycle, so it makes sense to get good at it.

Sometimes it may be tempting for either IT or the business people involved to short-cut the process, or to give up on it entirely. This is s sure recipe for disaster. It is difficult to make establishing requirements a fun exercise for all those involved, but it is important the the BI team continually tries to keep energy levels high. This can be done in a number of ways: by reminding everyone about the importance of their work for the organisation as a whole; by trying to use prototypes to make discussions more concrete; and – probably most importantly – by building and maintaining personal relationships with their business counterparts. If you are going to work for a long time with people on something that is hard to do, then it makes sense to at least try to get along. It is on apparently small things such as these essentially human interactions that the success or failure of multi-million dollar projects can hinge.

Helping the business to articulate what they want from BI is extremely important and equally easy to get wrong. Mistakes made at this stage can indeed derail the whole project. However, this is precisely what the BI team should be good at; it should be their core competency. If this work is not done well, then again it is primarily the responsibility of the professionals involved. A statement such as “the business didn’t know what they wanted” simply reflects that the BI team were not very good at running this phase of a BI project.
 
 
Conclusion

The case against the BI team

I find myself back at my previous position. Of course the idea of finding scape-goats for the failure of a BI project can be very tempting for the members of the team that has failed. However, this is an essentially futile process and one that proves that the adage about learning from your mistakes does not always apply.

To make things personal. Suppose that I am responsible for leading project in which it is obvious up-front that extensive buy-in and collaboration will be required from a group of people. If the project fails because neither of these things was obtained, then surely that’s my fault and not theirs, isn’t it?
 


 
For some further thoughts on this issue, take a look at an article by Ferenc Mantfeld entitled: Top 10 reasons why Business Intelligence Projects fail.