Jeffrey Archer	Joseph Conrad

Introduction

The context of the above bon mot was – as is often the case – a discussion on LinkedIn.com. I have been rather absent from the LinkedIn.com discussion groups for the same reasons that I have not been blogging and tweeting. In this case, my attention was drawn to the debate by a colleague.

	CIOs.com: Chief Information Officer Network

The particular thread was posted by Andy McKnight and is entitled What’s missing from Business Intelligence? and at the time of writing has attracted nearly 60 responses (you have to be a member of the group to view the discussion). It referred to an article published by EMC² which has the strap-line How CIOs can Reap the Benefits of BI Technology (note: this is a PDF document). Here is a pertinent quote:

The bad news is that only twenty-seven percent of respondents [to a survey of CIOs carried out by IDG Research] who use a BI solution report being extremely successful or very successful with it. Forty-five percent report being only somewhat successful, while seventeen percent say that they are not very, or not at all successful.

I’m not sure what happened to the other 11% of respondents, maybe they just hung up the ‘phone.
 
 
Blaming the users

Having stated that “BI has, too often, not lived up to expectations”, the paper goes on to list some reasons why. First on the list is the following:

• lack of adoption by users

You don’t have to be Einstein to realise that this is the result of a BI project failing, not the cause of it. The equivalent in athletics terms would be to say that you came last in the race because everyone else was faster than you. While obviously true this observation doesn’t help a lot with how to do better next time.

Of course hidden in the comment is the plaintive whine heard emanating from many an unsuccessful project (or indeed product launch), “the problem is the users”. This is arrant nonsense, returning to the start of article if you write a book that is panned by the critics and not bought by the public, then there is at least some chance that the fault lies with you and not them. It is the job of the IT professional to know their users, understand their needs and provide systems that cause delight, not disillusion.

A more interesting observation later on is:

Many BI initiatives falter because the analytics capabilities that are at the core of the system aren’t even used. Many users simply pull data from the warehouse and dump it in a spreadsheet. [...] A true BI implementation includes both reporting and analytics. CIOs indicate a much higher success rate with BI when users embrace both.

I think that there is some truth in this. Some of the BI failures I have seen have gone to the bother of building a warehouse only to front it with flat reports that are only marginally better than what they replaced.

In my career I have taken the opposite approach. While many people warn against analysis paralysis, I have deployed OLAP tools to all users, with fixed format reports de-emphasised, or used mostly for external purposes. This does mean that more effort needs to be put into training, but this is necessary anyway if you want your BI system to be an agent of change (and why else would you be building one if this is not the case?). I cover my general approach to driving user adoption in a series of three articles as follows:

• Marketing Change
• Education and cultural transformation
• Sustaining Cultural Change

This approach was very successful and we achieved user adoption of 92% – i.e. of those people who attended training, 92% remained active users (defined as using the BI system on average for at least two extended periods each week). We actually felt that the OLAP tools we were implementing were pretty intuitive and easy-to-use and so focussed mostly on how to use them in specific business scenarios. Overall we felt that training was 25% technical and 75% business-related.
 
 
Aiming for simplicity

Related to the above point, the EMC² article also mentions the following reason for failure:

• limited functionality/hard to use

This seems a little oxymoronic as normally it is depth of functionality that confuses people. I think I would disagree with both parts of this point. Out of the box, most BI tools have rich functionality and a reasonably intuitive to use. In one response to the LinkedIn.com thread I said the following:

I have been successful in getting users [...] weaned [...] off ad hoc reports, it wasn’t an easy process and required persistence and selling, but this paid off. [...] It is illuminating seeing business managers (some of whom still dictate memos for their secretaries to type) “slicing and dicing”, drilling down/through and generally interacting away merrily and stating that if all IT was this easy to use and informative, they might have taken to it earlier.

My view here is that you can make the tool as complicated or a simple as you choose. Going back to my first warehouse project, in our somewhat naive early attempts at prototype cubes, we had all available dimensions and all available measures included. I think our idea is that the users could help us sift out the ones that were most important. Instead this approach caused the negative reactions that the article refers to.

We subsequently adopted a rule of having as few dimensions and measures as possible in a cube, without compromising the business need that the cube was trying to address. The second part of this rule was that every cube had to be focussed on answering business questions in at least one area and at most two.

Rather than having a small number of monolithic cubes, we went with the option of a slightly larger number of significantly clearer and simpler ones. I think that this was a factor in our success in driving business adoption.
 
 
Should the fact that some BI projects fail dissuade you from BI?

I won’t attempt to dissect the rest of the article, the areas that I comment on above are representative. There are some good points and some less good ones – just like any article, including of course my own. Take a look yourself and see whether the findings and recommendations chime with your own experience of success and failure. What I did want to do was to return to the context of the aphorism that starts this post.

The thesis of the original LinkedIn.com post was that because a significant number of organisations had failed to get enormous benefit from BI, BI itself was therefore somehow flawed. I think this is wrong-headed reasoning. If 1,000 people write a book, how many are likely to become acknowledged as great authors? How many are likely to have the lesser accolade of commercial success? The answer in both cases is “not many”. This is because writing well is a very difficult thing to do (I prove this myself with every blog post!). Not everyone who tries it will be successful. BI is also difficult to do well and a major cause of problems is underestimating this difficulty.

Maybe this is too recherché and example, and maybe if the chances of success with BI are as slim as winning the Purlitzer Prize then it is not worth the effort. So I’ll instead I’ll resort to my favourite area of the sporting analogy. Let’s take the same 1,000 people and say that they all take up a new sport – it is mostly immaterial what the sport is, let’s say tennis. How many of them will go on to become proficient in it? By this I don’t mean that they are the next Roger Federer, just that they become competent enough to serve adequately, master the dark arts of the backhand and sustain a few rallies. My feeling is that the stats would look something like those in the EMC² report.
 
 
Is the prize worth it?

Given this, does it mean that some companies are just not cut out for BI and should ignore the area? Well the answer is “it depends”. Going back to tennis, if some one wants to be good, and has the determination to succeed, that is a necessary (though sadly not sufficient) condition. What may drive such a person on is the objective of achieving a goal, or maybe the pleasure of being able to perform at a certain level.

Focussing on business outcomes, I believe that BI can deliver substantial benefits. In fact I have argued elsewhere that BI can have the greatest payback of any IT project. Of course this presupposes that the BI project is done well. If the prize is potentially that great then maybe – like the aspiring tennis player who wants to become better – trying again makes sense. In recent recruitment I have heard frequent mention of organisations that were building their second warehouse as they didn’t get the first one quite right.

However the comparison with tennis breaks down in that business is a team game. If an organisation as a whole has struggled with BI, then this is not a question of simply accepting your genetic limitations. Companies can “evolve” capabilities by hiring people who have been successful in a field. They can also get benefit from targeted consultancy from practitioners who have a track record of success; this can help them to build an internal capability. This is an approach that I took advantage of myself in the initial six months of my first BI project [note: although I often seem to get mistaken for a BI consultant, I am not touting for business here!].

This means that if a company’s BI architecture is currently the equivalent of a Jeffrey Archer novel, it is still possible to transform it into Heart of Darkness. It will not be easy and will take time and effort, but there are people out there who have been successful and can act as guides.

In closing I should also mention that, if you take appropriate precautions, it is far from inevitable that the end of a BI journey will be finding your own version of Kurtz!
 

	Tweet this article on twitter.com

Bookmark this article with:
| Facebook	| del.icio.us	| digg	| Reddit	| Stumble

I have been a featured blogger on SmartDataCollective.com almost as long as I have been a blogger. SDC.com is Social Media Today’s community site, focussed on all aspects of Business Intelligence, Data Warehousing and Analytics, with a pinch of social media thrown in to the mix.

Brian Roger, the SDC.com editor, was recently kind enough to interview me about my career in BI, the challenges I have faced and what has helped to overcome these. This interview is now available to listen to as part of their Podcast series – click on the image below to visit their site.

SmartDataCollective.com Intervew

I would be interested in feedback about any aspect of this piece, which I am grateful to Brian for arranging.
 

Social Media Today LLC, is a battle-tested, nimble team of former journalists, online managers, and advertising professionals who have come together to make a new kind of media company. With their backgrounds, and passions for, business-to-business and public policy conversations, they have decided to focus their efforts in this area. To facilitate the types of convresations that they would like to see Social Media Today is assembling the world’s best bloggers and providing them with an independent “playground” to include their posts, to comment and rate posts, and to connect with each other. On their flagship site, SocialMediaToday.com, they have brought together many of the most intriguing and original bloggers on media and marketing, covering all aspects of what makes up the connective tissue of social media from a global perspective.
 

	Tweet this article on twitter.com

Bookmark this article with:
Technorati	| del.icio.us	| digg	| Reddit	| Stumble

As is frequently the case, I was moved to write this piece by a discussion on LinkedIn.com. This time round, the group involved was The Data Warehousing Institute (TDWI™) 2.0 and the thread, entitled Is one version of the truth attainable?, was started by J. Piscioneri. I should however make a nod in the direction of an article on Jim Harris’ excellent Obsessive-Compulsive Data Quality Blog called The Data Information Continuum; Jim also contributed to the LinkedIn.com thread.

Standard note: You need to be a member of both LinkedIn.com and the group mentioned to view the discussions.
 
 
Introduction

Here are a couple of sections from the original poster’s starting comments:

I’ve been thinking: is one version of the truth attainable or is it a bit of snake oil? Is it a helpful concept that powerfully communicates a way out of spreadmart purgatory? Or does the idea of one version of the truth gloss over the fact that context or point of view are an inherent part of any statement about data, which effectively makes truth relative? I’m leaning toward the latter position.

[...]

There can only be one version of the truth if everyone speaks the same language and has a common point of view. I’m not sure this is attainable. To the extent that it is, it’s definitely not a technology exercise. It’s organizational change management. It’s about changing the culture of an organization and potentially breaking down longstanding barriers.

Please join the group if you would like to read the whole post and the subsequent discussions, which were very lively. Here I am only going to refer to these tangentially and instead focus on the concept of a single version of the truth itself.

Readers who are not interested in the ellipitcal section of this article and who would instead like to cut to the chase are invited to click here (warning there are still some ellipses in the latter sections).
 
 
A [very] brief and occasionally accurate history of truth

I have discovered a truly marvellous proof of the nature of truth, which this column is too narrow to contain.

– Pierre de Tomas (1637)

Instead of trying to rediscover M. Tomas’ proof, I’ll simply catalogue some of the disciplines that have been associated (rightly or wrongly) with trying to grapple with the area:

• Various branches of Philosophy, including:
  • Metaphysics
  • Epistemology
  • Ethics
  • Logic
• History
• Religion (or more perhaps more generally spirituality)
• Natural Science
• Mathematics
• and of course Polygraphism

Given my background in Pure Mathematics the reader might expect me to trumpet the claims of this discipline to be the sole arbiter of truth; I would reply yes and no. Mathematics does indeed deal in absolute truth, but only of the type: if we assume A and B, it then follows that C is true. This is known as the axiomatic approach. Mathematics makes no claim for the veracity of axioms themselves (though clearly many axioms would be regarded as self-evidently true to the non-professional). I will also manfully resist the temptation to refer to the wrecking ball that Kurt Gödel’s took to axiomatic systems in 1931.

I have also made reference (admittedly often rather obliquely) to various branches of science on this blog, so perhaps this is another place to search for truth. However the Physical sciences do not really deal in anything as absolute as truth. Instead they develop models that approximate observations, these are called scientific theories. A good theory will both explain aspects of currently observed phenomena and offer predictions for yet-to-be-observed behaviour (what use is a model if it doesn’t tell us things that we don’t already know?). In this way scientific theories are rather like Business Analytics.

Unlike mathematical theories, the scientific versions are rather resistant to proof. Somewhat unfairly, while a mountain of experiments that are consistent with a scientific theory do not prove it, it takes only one incompatible data point to disprove it. When such an inconvenient fact rears its head, the theory will need to be revised to accommodate the new data, or entirely discarded and replaced by a new theory. This is of course an iterative process and precisely how our scientific learning increases. Warning bells generally start to ring when a scientist starts to talk about their theory being true, as opposed to a useful tool. The same observation could be made of those who begin to view their Business Analytics models as being true, but that is perhaps a story for another time.

I am going to come back to Physical science (or more specifically Physics) a little later, but for now let’s agree that this area is not going to result in defining truth either. Some people would argue that truth is the preserve of one of the other subjects listed above, either Philosophy or Religion. I’m not going to get into a debate on the merits of either of these views, but I will state that perhaps the latter is more concerned with personal truth than supra-individual truth (otherwise why do so many religious people disagree with each other?).

Discussing religion on a blog is also a sure-fire way to start a fire, so I’ll move quickly on. I’m a little more relaxed about criticising some aspects of Philosophy; to me this can all too easily descend into solipism (sometimes even quicker than artificial intelligence and cognitive science do). Although Philosophy could be described as the search for truth, I’m not convinced that this is the same as finding it. Maybe truth itself doesn’t really exist, so attempting to create a single version of it is doomed to failure. However, perhaps there is hope.
 
 
Trusting your GUT feeling

© xkcd.com

After the preceding divertimento, it is time to return to the more prosaic world of Business Intelligence. However there is first room for the promised reference to Physics. For me, the phrase “a single version of the truth” always has echoes of the search for a Grand Unified Theory (GUT). Analogous to our discussions about truth, there are some (minor) definitional issues with GUT as well.

Some hold that GUT applies to a unification of the electromagnetic, weak nuclear and strong nuclear forces at very high energy levels (the first two having already been paired in the electroweak force). Others that GUT refers to a merging of the particles and forces covered by the Standard Model of Quantum Mechanics (which works well for the very small) with General Relativity (which works well for the very big). People in the first camp might refer to this second unification as a ToE (Theory of Everything), but there is sometimes a limit to how much Douglas Adams’ esteemed work applies to reality.

For the purposes of this article, I’ll perform the standard scientific trick of a simplifying assumption and use GUT in the grander sense of the term.

Scientists have striven to find a GUT for decades, if not centuries, and several candidates have been proposed. GUT has proved to be something of a Holy Grail for Physicists. Work in this area, while not as yet having been successful (at least at the time of writing), has undeniably helped to shed a light on many other areas where our understanding was previously rather dim.

This is where the connection with a single version of the truth comes in. Not so much that such either concept is guaranteed to be achievable, but that a lot of good and useful things can be accomplished on a journey towards both of them. If, in a given organisation, the journey to a single version of the truth reaches its ultimate destination, then great. However if, in an another company, a single version of the truth remains eternally just over the next hill, or round the next corner, then this is hardly disastrous and maybe it is the journey itself (and the aspirations with which it is commenced on) that matters more than the destination.

Before I begin to sound too philosophical (cf. above) let me try to make this more concrete by going back to our starting point with some Mathematics and considering some Venn diagrams.
 
 
Ordo ab chao

In my experience the following is the type of situation that a good Business Intelligence programme should address:

The problems here are manifold:

1. Although the various report systems are shown as separate, the real situation is probably much worse. Each of the reporting and analysis systems will overlap, perhaps substantially, with one or more or the other ones. Indeed the overlapping may be so convoluted that it would be difficult to represent this in two dimensions and I am not going to try. This means that you can invariably ask the same question (how much have we sold this month) of different systems and get different answers. It may be difficult to tell which of these is correct, indeed none of them may be a true reflection of business reality.
2. There are a whole set of things that may be treated differently in the different ellipses. I’ll mention just two for now: date and currency. In one system a transaction may be recorded in a month when it is entered into the system. In another it may be allocated to the month when the event actually occurred (sometimes quite a while before it is entered). In a third perhaps the transaction is only dated once it has been authorised by a supervisor.

   In a multi-currency environment reports may be in the transactional currency, rolled-up to the currency of the country in which they occurred, or perhaps aggregated across many countries in a number of “corporate” currencies. Which rate to use (rate on the day, average for the month, rolling average for the last year, a rate tied to some earlier business transaction etc.) may be different in different systems, equally the rate may well vary according to the date of the transaction (making the last set of comments about which date is used even more pertinent).

3. A whole set of other issues arise when you begin to consider things such as taxation (are figures nett or gross), discounts, commissions to other parties, phased transactions and financial estimates. Some reports may totally ignore these, others my take account of some but not others. A mist of misunderstanding is likely to arise.
4. Something that is not drawn on the above diagram is the flow of data between systems. Typically there will be a spaghetti-like flow of bits and bytes between the different areas. What is also not that uncommon is that there is both bifurcation and merging in these flows. For example, some sorts of transactions from Business Unit A may end up in the Marketing database, whereas others do not. Perhaps transactions carried out on behalf of another company in the group appear in Business Unit B’s reports, but must be excluded from the local P&L. The combinations are almost limitless.

   Interfaces can also do interesting things to data, re-labelling it, correcting (or so their authors hope) errors in source data and generally twisting the input to form output that may be radically different. Also, when interfaces are anything other than real-time, they introduce a whole new arena in which dates can get muddled. For instance, what if a business transaction occurred in a front-end system on the last day of a year, but was not interfaced to a corporate database until the first day of the next one – which year does it get allocated to in the two places?

5. Finally, the above says nothing about the costs (staff and software) of maintaining a heterogeneous reporting landscape; or indeed the costs of wasted time arguing about which numbers are right, or attempting to perform tortuous (and ultimately fruitless) reconciliations.

Now the ideal situation is that we move to the following diagram:

This looks all very nice and tidy, but there are still two major problems.

1. A full realisation of this transformation may be prohibitively expensive, or time-consuming.
2. Having brought everything together into one place offers an opportunity to standardise terminology and to eliminate the confusion caused by redundancy. However, it doesn’t per se address the other points made from 2. onwards above.

The need to focus on what is possible in a reasonable time-frame and at a reasonable cost may lead to a more pragmatic approach where the number of reporting and analysis systems is reduced, but to a number greater than one. Good project management may indeed dictate a rolling programme of consolidation, with opportunities to review what has worked and what has not and to ascertain whether business value is indeed being generated by the programme.

Nevertheless, I would argue that it is beneficial to envisage a final state for the information architecture, even if there is a tacit acceptance that this may not be realised for years, if at all. Such a framework helps to guide work in a way that making it up as we go along does not. I cover this area in more detail in both Holistic vs Incremental approaches to BI and Tactical Meandering for those who are interested.

It is also inevitable that even in a single BI system data will need to be presented in different ways for different purposes. To take just one example, if you goal is to see how the make up of a book of business has varied over time, then it is eminently sensible to use a current exchange rate for all transactions; thereby removing any skewing of the figures caused by forex fluctuations. This is particularly the case when trying to assess the profitability of business where revenue occurs at a discrete point in the past, but costs may be spread out over time.

However, if it is necessary to look at how the organisation’s cash-flow is changing over time, then the impact of fluctuations in foreign exchange rates must be taken into account. Sadly if an American company wants to report how much revenue it has from its French subsidiary then the figures must reflect real-life euro / dollar rates (unrealised and realised foreign currency gains and losses notwithstanding).

What is important here is labelling. Ideally each report should show the assumptions under which it has been compiled at the top. This would include the exchange rate strategy used, the method by which transactions are allocated to dates, whether figures are nett or gross and which transactions (if any) have been excluded. Under this approach, while it is inevitable that the totals on some reports will not agree, at least the reports themselves will explain why this is the case.

So this is my take on a single version of the truth. It is both a) an aspirational description of the ideal situation and something that is worth striving for and b) a convenient marketing term – a sound-bite if you will – that presents a palatable way of describing a complex set of concepts. I tried to capture this essence in my reply to the LinkedIn.com thread, which was as follows:

To me, the (extremely hackneyed) phrase “a single version of the truth” means a few things:

1. One place to go to run reports and perform analysis (as opposed to several different, unreconciled, overlapping systems and local spreadsheets / Access DBs)
2. When something, say “growth” appears on a report, cube, or dashboard, it is always calculated the same way and means the same thing (e.g. if you have growth in dollar terms and growth excluding the impact of currency fluctuations, then these are two measures and should be clearly tagged as such).
3. More importantly, that the organisation buys into there being just one set of figures that will be used and self-polices attempts to subvert this with roll-your-own data.

Of course none of this equates to anything to do with truth in the normal sense of the word. However life is full of imprecise terminology, which nevertheless manages to convey meaning better than overly precise alternatives.

More’s Utopia was never intended to depict a realistic place or system of government. These facts have not stopped generations of thinkers and doers from aspiring to make the world a better place, while realising that the ultimate goal may remain out of reach. In my opinion neither should the unlikelihood of achieving a perfect single version of the truth deter Business Intelligence professionals from aspiring to this Utopian vision.

I have come pretty close to achieving a single version of the truth in a large, complex organisation. Pretty close is not 100%, but in Business Intelligence anything above 80% is certainly more than worth the effort.
 

	Tweet this article on twitter.com

Bookmark this article with:
Technorati	| del.icio.us	| digg	| Reddit	| Stumble

Back in February, Dorothy Miller wrote a piece at IT Business Edge entitled, Measuring the Return on Investment for Business Intelligence. I wrote a comment on this, which I subsequently expanded to create my article, Measuring the benefits of Business Intelligence.

This particular wheel has now come full circle with Ann All from the same web site recently interviewing me and several BI industry leaders about our thoughts on the best ways to generate returns from business intelligence projects. This new article is called, Big vs. Small BI: Which Set of Returns Is Right for Your Company? In it Ann weaves together an interesting range of (sometimes divergent) opinions about which BI model is most likely to lead to success. I would recommend you read her work.

The other people that Ann quotes are:

 
Some differences of opinion

As might be deduced from the title of Ann’s piece the opinions of the different interviewees were not 100% harmonious with each other. There was however a degree of alignment between a few people. As Ann says:

Corcoran, Colbert and Thomas believe pervasive use of BI yields the greatest benefits.

On this topic she quoted me as follows (I have slightly rearranged the text in order to shorten the quote):

If BI can trace all the way from the beginning of a sales process to how much money it made the company, and do it in a way that focuses on questions that matter at the different decision points, that’s where I’ve seen it be most effective.

By way of contrast Pendse favours:

smaller and more tactical BI projects, largely due to what his surveys show are a short life for BI applications at many companies. “The median age of all of the apps we looked at is less than 2.5 years. For one reason or another, within five years the typical BI app is no longer in use. The problem’s gone away, or people are unhappy with the vendor, or the users changed their minds, or you got acquired and the new owner wants you to do something different,” he says. “It’s not like an ERP system, where you really would expect to use it for many years. The whole idea here is go for quick, simple wins and quick payback. If you’re lucky, it’ll last for a long time. If you’re not lucky, at least you’ve got your payback.”

I’m sure that Nigel’s observations are accurate and his statistics impeccable. However I wonder whether what he is doing here is lumping bad BI projects with good ones. For a BI project a lifetime of 2.5 years seems extraordinarily short, given the time and effort that needs to be devoted to delivering good BI. For some projects the useful lifetime must be shorter than the development period!

Of course it may be that Nigel’s survey does not discriminate between tiny, tactical BI initiatives, failed larger ones and successful enterprise BI implementations. If this is the case, then I would not surprised if the first two categories drag down the median. Though you do occasionally hear horror stories of bad BI projects running for multiple years, consuming millions of dollars and not delivering, most bad BI projects will be killed off fairly soon. Equally, presumably tactical BI projects are intended to have a short lifetime. If both of these types of projects are included in Pendse’s calculations, then maybe the the 2.5 years statistic is more understandable. However, if my assumptions about the survey are indeed correct, then I think that this figure is rather misleading and I would hesitate to draw any major conclusions from it.

In order that I am not accused of hidden bias, I should state unequivocally that I am a strong proponent of Enterprise BI (or all-pervasive BI, call it what you will), indeed I have won an award for an Enterprise BI implementation. I should also stress that I have been responsible for developing BI tools that have been in continuous use (and continuously adding value) for in excess of six years. My opinions on Enterprise BI are firmly based in my experiences of successfully implementing it and seeing the value generated.

With that bit of disclosure out of the way, let’s return to the basis of Nigel’s recommendations by way of a sporting analogy (I have developed quite a taste for these, having recently penned artciles relating both rock climbing and mountain biking to themes in business, technology and change).
 
 
A case study

The [English] Premier League is the world’s most watched Association Football (Soccer) league and the most lucrative, attracting the top players from all over the globe. It has become evident in recent seasons that the demands for club success have become greater than ever. The owners of clubs (be those rich individuals or shareholders of publicly quoted companies) have accordingly become far less tolerant of failure by those primarily charged with bringing about such success; the club managers. This observation was supported by a recent study^[1] that found that the average tenure of a dismissed Premier League manager had declined from a historical average of over 3 years to 1.38 years in 2008.

As an aside, the demands for business intelligence to deliver have undeniably increased in recent years; maybe BI managers are not quite paid the same as Football managers, but some of the pressures are the same. Both Football managers and BI managers need to weave together a cohesive unit from disparate parts (the Football manager creating a team from players with different skills, the BI manager creating a system from different data sources). So given, these parallels, I suggest that my analogy is not unreasonable.

Returning to the remarkable statistic of the average tenure of a departing Premier League manger being only 1.38 years and applying Pendse’s logic we reach an interesting conclusion. Football clubs should be striving to have their managers in place for less than twelve months as they can then be booted out before they are obsolete. If this seems totally counter-intutitive, then maybe we could look at things the other way round. Maybe unsuccessful Football managers don’t last long and maybe neither do unsuccessful BI projects. By way of corollary, maybe there are a lot of unsuccessful BI projects out there – something that I would not dispute.

By way of an example that perhaps bears out this second way of thinking about things, the longest serving Premier League manager, Alex Ferguson of Manchester United, is also the most successful. Manchester United have just won their third successive Premier League and have a realistic chance of becoming the first team ever to retain the UEFA Champions League.

Similarly, I submit that the median age of successful BI projects is most likely significantly more than 2.5 years.
 
 
Final thoughts

I am not a slavish adherent to an inflexible credo of big BI; for me what counts is what works. Tactical BI initiatives can be very beneficial in their own right, as well as being indispensible to the successful conduct of larger BI projects; something that I refer to in my earlier article, Tactical Meandering. However, as explained in the same article, it is my firm belief that tactical BI works best when it is part of a strategic framework.

In closing, there may be some very valid reasons why a quick and tactical approach to BI is a good idea in some circumstances. Nevertheless, even if we accept that the median useful lifetime of a BI system is only 2.5 years, I do not believe that this is grounds for focusing on the tactical to the exclusion of the strategic. In my opinion, a balanced tactical / strategic approach that can be adapted to changing circumstances is more likely to yield sustained benefits than Nigel Pendse’s tactical recipe for BI success.
 

	Tweet this article on twitter.com

Bookmark this article with:
Technorati	| del.icio.us	| digg	| Reddit	| Stumble

On further reflection about this earlier article, I realised that I missed out one important point. This was perhaps implicit in the diagram that I posted (and which I repeat below), but I think that it makes sense for me to make things explicit.

An example of a multi-tier BI architecture with different tools

The point is that in this architecture with different BI tools in different layers, it remains paramount to have consistency in terminology and behaviour for dimensions and measures. So “Country” and “Profit” must mean the same things in your dashboard as it does in your OLAP cubes. The way that I have achieved this before is to have virtually all of the logic defined in the warehouse itself. Of course some things may need to be calculated “on-the-fly” within the BI tool, in this case care needs to be paid to ensuring consistency.

It has been pointed out that the approach of using the warehouse to drive consistency may circumscribe your ability to fully exploit the functionality of some BI tools. While this is sometimes true, I think it is not just a price worth paying, but a price that it is mandatory to pay. Inconsistency of any kind is the enemy of all BI implementations. If your systems do not have credibility with your users, then all is already lost and no amount of flashy functionality will save you.
 

	Tweet this article on twitter.com

Bookmark this article with:
Technorati	| del.icio.us	| digg	| Reddit	| Stumble

Introduction

This post follows on from a question that was asked on the LinkedIn.com Data Warehousing Institute (TDWI™) 2.0 group. Unfortunately the original thread is no longer available for whatever reason, but the gist of the question was whether anyone had experience with using a number of BI tools to cover different functions within an implementation. So the scenario might be: Tool A for dashboards, Tool B for OLAP, Tool C for Analytics, Tool D for formatted reports and even Tool E for visualisation.

In my initial response I admitted that I had not faced precisely this situation, but that I had worked with the set-up shown in the following diagram, which I felt was not that dissimilar:

An example of a multi-tier BI architecture with different tools

Here there is no analytics tool (in the statistical modelling sense – Excel played that role) and no true visualisation (unless you count graphs in PowerPlay that is), but each of dashboards, OLAP cubes, formatted reports and simple list reports are present. The reason that this arrangement might not at first sight appear pertinent to the question asked on LinkedIn.com is that two of the layers (and three of the report technologies) are from one vendor; Cognos at the time, IBM-Cognos now. The reason that I felt that there was some relevance was that the Cognos products were from different major releases. The dashboard tool being from their Version 8 architecture and the OLAP cubes and formatted reports from their Version 7 architecture.
 
 
A little history

London Bridge circa 1600

Maybe a note of explanation is necessary as clearly we did not plan to have this slight mismatch of technologies. We initially built out our BI infrastructure without a dashboard layer. Partly this was because dashboards weren’t as much of a hot topic for CEOs when we started. However, I also think it also makes sense to overlay dashboards on an established information architecture (something I cover in my earlier article, “All that glisters is not gold” – some thoughts on dashboards, which is also pertinent to these discussions).

When we started to think about adding icing to our BI cake, ReportStudio in Cognos 8 had just come out and we thought that it made sense to look at this; both to deliver dashboards and to assess its potential future role in our BI implementation. At that point, the initial Cognos 8 version of Analysis Studio wasn’t an attractive upgrade path for existing PowerPlay users and so we wanted to stay on PowerPlay 7.3 for a while longer.

The other thing that I should mention is that we had integrated an in-house developed web-based reporting tool with PowerPlay as the drill down tool. The reasons for this were a) we had already trained 750 users in this tool and it seemed sensible to leverage it and b) employing it meant that we didn’t have to buy an additional Cognos 7 product, such as Impromptu, to support this need. This hopefully explains the mild heterogeneity of our set up. I should probably also say that users could directly access any one of the BI tools to get at information and that they could navigate between them as shown by the arrows in the diagram.

I am sure that things have improved immensely in the Cognos toolset since back then, but at the time there was no truly seamless integration between ReportStudio and PowerPlay as they were on different architectures. This meant that we had to code the passing of parameters between the ReportStudio dashboard and PowerPlay cubes ourselves. Although there were some similarities between the two products, there were also some differences at the time and these, plus the custom integration we had to develop, meant that you could also view the two Cognos products as essentially separate tools. Add in here the additional custom integration of our in-house reporting application with PowerPlay and maybe you can begin to see why I felt that there were some similarities between our implementation and one using different vendors for each tool.

I am going to speak a bit about the benefits and disadvantages of having a single vendor approach later, but for now an obvious question is “did our set-up work?” The answer to this was a resounding yes. Though the IT work behind the scenes was maybe not the most elegant (though everything was eminently supportable), from the users’ perspective things were effectively seamless. To slightly pre-empt a later point, I think that the user experience is what really matters, more than what happens on the IT side of the house. Nevertheless let’s move on from some specifics to some general comments.
 
 
The advantages of a single vendor approach to BI

One-stop shopping

I think that it makes sense if I lay my cards on the table up-front. I am a paid up member of the BI standardisation club. I think that you only release the true potential of BI when you take a broad based approach and bring as many areas as you can into your warehouse (see my earlier article, Holistic vs Incremental approaches to BI, for my reasons for believing this).

Within the warehouse itself there should be a standardised approach to dimensions (business entities and the hierarchies they are built into should be the same everywhere – I’m sure this will please all my MDM friends out there) and to measures (what is the point if profitability is defined different ways in different reports?). It is almost clichéd nowadays to speak about “the single version of the truth”, but I have always been a proponent of this approach.

I also think that you should have the minimum number of BI tools. Here however the minimum is not necessarily always one. To misquote one of Württemberg’s most famous sons:

Everything should be made as simple as possible, but no simpler.

What he actually said was:

It can scarcely be denied that the supreme goal of all theory is to make the irreducible basic elements as simple and as few as possible without having to surrender the adequate representation of a single datum of experience.

but maybe the common rendition is itself paying tribute to the principle that he propounded. Let me pause to cover what are the main reasons quoted for adopting a single vendor approach in BI:

1. Consistent look-and-feel: The tools will have a common look-and-feel, making it easier for people to use them and simplifying training.
2. Better interoperability: Interoperability between the tools is out-of-the-box, saving on time and effort in developing and maintaining integration.
3. Clarity in problem resolution: If something goes wrong with your implementation, you don’t get different vendors blaming each other for the problem.
4. Simpler upgrades: You future proof your architecture, when one element has a new release, it is the vendor’s job to ensure it works with everything else, not yours.
5. Less people needed: You don’t need to hire an expert for each different vendor tool, thereby reducing the size and cost of your BI team.
6. Cheaper licensing: It should be cheaper to buy a bundled solution from one vendor and ongoing maintenance fees should also be less.

This all seems to make perfect sense and each of the above points can be seen to be reducing the complexity and cost of your BI solution. Surely it is a no-brainer to adopt this approach? Well maybe. Let me offer some alternative perspectives on each item – none of these wholly negates the point, but I think it is nevertheless worth considering a different perspective before deciding what is best for your organisation.

1. Consistent look-and-feel: It is not always 100% true that different tools from the same vendor have the same look-and-feel. This might be down to quality control at the vendor, it might be because the vendor has recently acquired part of their product set and not fully integrated it as yet, or – even more basically – it may be because different tools are intended to do different things. To pick one example from outside of BI that has frustrated me endlessly over the years: PowerPoint and Word seem to have very little in common, even in Office 2007. Hopefully different tools from the same vendor will be able to share the same metadata, but this is not always the case. Some research is probably required here before assuming this point is true. Also, picking up on the Bauhaus ethos of form dictating function, you probably don’t want to have your dashboard looking exactly like your OLAP cubes – it wouldn’t be a dashboard then would it? Additional user training will generally be required for each tier in your BI architecture and a single-vendor approach will at best reduce this somewhat.
2. Better interoperability: I mention an problem with interoperability of the Cognos toolset above. This is is hopefully now a historical oddity, but I would be amazed if similar issues do not arise at least from time to time with most BI vendors. Cognos itself has now been acquired by IBM and I am sure everyone in the new organisation is doing a fine job of consolidating the product lines, but it would be incredible if there were not some mismatches that occur in the process. Even without acquisitions it is likely that elements of a vendor’s product set get slightly out of alignment from time to time.
3. Clarity in problem resolution: This is hopefully a valid point, however it probably won’t stop your BI tool vendor from suggesting that it is your web-server software, or network topology, or database version that is causing the issue. Call me cynical if you wish, I prefer to think of myself as a seasoned IT professional!
4. Simpler upgrades: Again this is also most likely to be a plus point, but problems can occur when only parts of a product set have upgrades. Also you may need to upgrade Tool A to the latest version to address a bug or to deliver desired functionality, but have equally valid reasons for keeping Tool B at the previous release. This can cause problems in a single supplier scenario precisely because the elements are likely to be more tightly coupled with each other, something that you may have a chance of being insulated against if you use tools from different vendors.
5. Less people needed: While there might be half a point here, I think that this is mostly fallacious. The skills required to build an easy-to-use and impactful dashboard are not the same as building OLAP cubes. It may be that you have flexible and creative people who can do both (I have been thus blessed myself in the past in projects I ran), but this type of person would most likely be equally adept whatever tool they were using. Again there may be some efficiencies in sharing metadata, but it is important not to over-state these. You may well still need a dashboard person and an OLAP person, if you don’t then the person who can do both with probably not care about which vendor provides the tools.
6. Cheaper licensing: Let’s think about this. How many vendors give you Tool B free when you purchase Tool A? Not many is the answer in my experience, they are commercial entities after all. It may be more economical to purchase bundles of products from a vendor, but also having more than one in the game may be an even better way of ensuring that cost are kept down. This is another area that requires further close examination before deciding what to do.

 
A more important consideration

Overall it is still likely that a single-vendor solution is cheaper than a multi-vendor one, but I hope that I have raised enough points to make you think that this is not guaranteed. Also the cost differential may not be as substantial as might be thought initially. You should certainly explore both approaches and figure out what works best for you. However there is another overriding point to consider here, the one I alluded to earlier; your users. The most important thing is that your users have the best experience and that whatever tools you employ are the ones that will deliver this. If you can do this while sticking to a single vendor then great. However if your users will be better served by different tools in different tiers, then this should be your approach, regardless of whether it makes things a bit more complicated for your team.

Of course there may be some additional costs associated with such an approach, but I doubt that this issue is insuperable. One comparison that it may help to keep in mind is that the per user cost of many BI tools is similar to desktop productivity tools such as Office. The main expense of BI programmes is not the tools that you use to deliver information, but all the work that goes on behind the scenes to ensure that it is the right information, at the right time and with the appropriate degree of accuracy. The big chunks of BI project costs are located in the four pillars that I consistently refer to:

1. Understand the important business decisions and what figures are necessary to support these.
2. Understand the data available in the organisation, how it relates to other data and to business decisions.
3. Transform the data to provide information answering business questions.
4. Focus on embedding the use of information in the corporate DNA.

The cost of the BI tools themselves are only a minor part of the above (see also, BI implementations are like icebergs). Of course any savings made on tools may make funds available for other parts of the project. It is however important not to cut your nose off to spite your face here. Picking right tools for the job, be they from one vendor or two (or even three at a push) will be much more important to the overall payback of your project than saving a few nickels and dimes by sticking to a one-vendor strategy just for the sake of it.
 

	Tweet this article on twitter.com

Bookmark this article with:
Technorati	| del.icio.us	| digg	| Reddit	| Stumble

 
 
Introduction

I enjoy reading the thoughts of vastly experienced industry analyst Merv Adrian on his blog, Market Strategies for IT Suppliers, and also on twitter via @merv. Merv covers industry trends and a wide variety of emerging and established technologies and companies. I would encourage you to subscribe to his RSS feed.

In a recent artcile, Balanced Insight – Automating BI Design to Deployment, Merv reviews the Consensus tool and approach developed by Ohio-based outfit Balanced Insight. I suggest that you read Merv’s thoughts first as I won’t unnecessarily repeat a lot of what he says here. His article also has links to a couple of presentations featuring the use of Consensus to build both Cognos 8 and Proclarity prototypes, which are interesting viewing.
 
 
An overview of Balanced Insight

The company certainly sets expectations high with the strap line of their web site:

Promising to “deliver in half the time without compromising cross project alignment” is a major claim and something that I will try to pay close attention to later.

The presentations / demonstrations start with a set-up of a fictional company (different ones in different demos) who want to find out more about issues in their business: outstanding receivables, or profit margins [Disclosure: the fact that the second demo included margins on mountain bikes initially endeared me to the company]. In considering these challenges, Balanced Insight offers the following slide contrasting IT’s typical response with the, presumably superior, one taken by them:

IT's approach to information problems vs Balanced Insight's

I agree with Balanced Insight’s recommendation, but rather take issue with the assumption that IT always starts by looking exclusively at data when asked to partake in information-based initiatives. I have outlined what I see as the four main pillars of a business intelligence project at many places on this blog, most recently in the middle of my piece on Business Intelligence Competency Centres. While of course it is imperative to understand the available data (what would be the alternative?), the first step in any BI project is to understand the business issues and, in particular, the questions that the business wants an answer to. If you search the web for BI case studies or methodologies, I can’t imagine many of these suggesting anything other than Balanced Insight’s recommended approach.

Moving on, the next stage of both the demos introduces the company’s “information packages”. These are panes holding business entities and have two parts; the upper half contains “Topics and Categories” (things such as date or product), the bottom half contains measurements. The “Topics and Categories” can be organised into hierarchies, for example: day is within week, which is within month, quarter and year. At this point most BI professionals will realise that “Topics and Categories” are what we all call “Dimensions” – but maybe Balanced Insight have a point picking a less technical-sounding name. So what the “information package” consists of is a list of measures and dimensions pertaining to a particular subject area – it is essentially a loose specification for a data mart.

The interesting point is what happens next, the Consensus Integrator uses the “information package” to generate what the vendor claims is an optimised star-schema database (in a variety of databases). It then creates a pre-built prototype that references the schema; this can be in a selection of different BI tools. From what I can tell from the demos, the second stage appears to consist of creating an XML file that is then read by the BI tool. In the first example, the “Topics and Categories” become dimensions in Cognos AnalysisStudio and the measures remain measures. In both demos sample data is initially used, but in the ProClarity one a version with full data is also shown – it is unclear whether this was populated via Consensus or not. The “information package” can also be exported to data modelling tools such as ERwin.

One of the Balanced Insight presentations then mentions that “all that’s left to do is then to develop your ETL”. I appreciate that it is difficult to go into everything in detail in a short presentation, but this does rather seem to be glossing over a major area, indeed one of my four pillars of BI projects referred to above. Such rather off-hand comments do not exactly engender confidence. If there is a better story to tell here, then Balanced Insight’s presentations should try to tell it.
 
 
The main themes

There are a few ideas operating here. First that Balanced Insight’s tools can support a process which will promote best practice in defining and documenting the requirements of a BI project and allow a strong degree of user interaction. Second that the same tools can quickly and easily produce functioning prototypes that can be used to refine these same requirements and also make discussions with business stakeholders more concrete. Finally that the prototypes can employ a variety of database and BI tools – so maybe you prototype on a cheap / free database and BI tool, then implement on a more expensive, and industrial strength, combination later.

Balanced Insight suggest that their product helps to address “the communication gap between IT and the business”. I think it is interesting using the “information package” as a document repository, which may be helpful at other stages of the project. But there are other ways of achieving this as well. How business friendly these are probably depends on how the BI team set them up. I have seen Excel and small Access databases work well without even buying a specific tool. Also I think that if a BI team needs a tool to ensure it sticks to a good process, then there is probably a bigger problem to worry about.

Of course, the production of regular prototypes is a key technique to employ in any BI project and it seems that Balanced Insight may be on to something here, particularly if the way that their “information package” presents subject areas makes it easier for the BI team and business people to discuss things. However, it is not that arduous to develop prototypes directly in most BI tools. To put this in a context drawn from my own experience, building Cognos cubes to illustrate the latest iteration of business requirement gathering was often a matter of minutes, compared to business analysts putting in many days of hard work before this stage.

Having decided to use Consensus to capture information about measures and dimensions, the ability to then transfer these to a range of BI tools in interesting. This may offer the opportunity to change tools during the initial stages of the project and to try out different tools with the same schema and data to assess their effectiveness. This may also be something that is a useful tool when negotiating with BI vendors. However, again I am not sure exactly how big of a deal this is. I would be interested in better understanding how users have taken advantage of this feature.
 
 
A potential fly in the ointment

It would be easy to offer a couple of other criticisms of the approach laid out in the demos; namely that it seems to be targeted at developing point solutions rather than a pervasive BI architecture and that (presumably related to this) the examples shown are very basic. However, I’m willing to given them the benefit of the doubt, a sales pitch is probably not the place for a lengthy exploration of broad and complex issues. So I think my overall response to Balanced Insight’s Consensus product could be summed up as guardedly positive.

Nevertheless, there is one thing that rather worries me and this can best be seen by looking at the picture below. [As per the disclaimer above, the following diagram is based on my own understanding of the product and has not been provided by Balanced Insight.]

My perception of how Balanced Insight addresses needs for information

I think I understand the single black arrow on the right of the diagram, I’m struggling to work out what Consensus offers (aside from documentation) for the two black arrows on the left hand side. Despite the fact that Balanced Insight disparaged the approach of looking at available data in their presentation, there is no escaping the fact that some one will have to do this at some point. Connections will then have to be made between the available data and the business questions that need answering.

In both demos Consensus is pre-populated with dimensions, measures and linkages of these to sample data. How this happens is not covered, but this is a key area for any BI project. Unless Balanced Insight have some deus ex machina that helps to cut the length of this stage, then I begin to become a little sceptical about their claim to halve the duration of BI work.

Of course my concerns could be unfounded. It will be interesting to see how things develop for the company and whether their bold claims stand the test of time.
 

	Tweet this article on twitter.com

Bookmark this article with:
Technorati	| del.icio.us	| digg	| Reddit	| Stumble

Maureen Clarry begins her latest BeyeNETWORK article, Leading Change in Business Intelligence, by stating:

If there was a standard list of core competencies for leaders of business intelligence (BI) initiatives, the ability to manage complex change should be near the top of the list.

I strongly concur with Maureen’s observation and indeed the confluence of BI and change management is a major theme of this blog; as well as the title of one of my articles on the subject. Maureen clearly makes the case that “business intelligence is central to supporting [...] organizational changes” and then spends some time on Prosci’s ADKAR model for leading change; bringing this deftly back into the BI sphere. Her closing thoughts are that such a framework can help a lot in driving the success of a BI project.
 
 
My reflections

I find it immensely encouraging that an increasing number of BI professionals and consultants are acknowledging the major role that change plays in our industry and in the success of our projects. In fact it is hard to find some one who has run a truly successful BI project without paying a lot of attention to how better information will drive different behaviour – if it fails to do this, then “why bother?” as Maureen succinctly puts it.

Without describing it as anything so grand as a framework, I have put together a trilogy of articles on the subject of driving cultural transformation via BI. These are as follows:

However the good news about many BI professionals and consultants embracing change management as a necessary discipline does not seem to have filtered through to all quarters of the IT world. Many people in senior roles still seem to see BI as just another technology area. This observation is born out of the multitude of BI management roles that request an intimate knowledge of specific technology stacks. These tend to make only a passing reference to experience of the industry in question and only very infrequently mention the change management aspects of BI at all.

Of course there are counterexamples, but the main exceptions to this trend seem to be where BI is part of a more business focused area, maybe Strategic Change, or the Change Management Office. Here it would be surprising if change management skills were not stressed. When BI is part of IT it seems that the list of requirements tends to be very technology focussed.

In an earlier article, BI implementations are like icebergs I argued that, in BI projects, the technology – at least in the shape of front-end slice-and-dice tools – is not nearly as important as understanding the key business questions that need to be answered and the data available to answer them with. In “All that glisters is not gold” – some thoughts on dashboards, I made similar points about this aspect of BI technology.

I am not alone in holding these opinions, many of the BI consultants and experienced BI managers that I speak to feel the same way. Given this, why is there the disconnect that I refer to above? It is a reasonable assumption that when a company is looking to set up a new BI department within IT, it is the CIO who sets the tone. Does this lead us inescapably to the the conclusion that many CIOs just don’t get BI?

I hope that this is not the case, but I see increasing evidence that there may be a problem. I suppose the sliver lining to this cloud is that, while such attitudes exist, they will lead to opportunities for more enlightened outfits, such as the one fronted by Maureen Clarry. However it would be even better to see the ideas that Maureen espouses moving into the mainstream thinking of corporate IT.
 

	Tweet this article on twitter.com

Bookmark this article with:
Technorati	| del.icio.us	| digg	| Reddit	| Stumble

The subject of this article ought to be reasonably evident from its title. However there is perhaps some room for misinterpretation around even this. Despite the recent furore about definitions, most reasonable people should be comfortable with a definition of business intelligence. My take on this is that BI is simply using information to drive better business decisions. In this definition, the active verb “drive” and the subject “business decisions” are the key elements; something that is often forgotten in a rush for technological fripperies.
 
 
The central issue

Having hopefully addressed of the “BI” piece of the BICC acronym, let’s focus on the “CC” part. I’ll do this in reverse order, first of all considering what is meant by “centre”. As ever I will first refer to my trusted Oxford English Dictionary for help. In a discipline, such as IT, which is often accused of mangling language and even occasionally using it to obscure more than to clarify, a back-to-basics approach to words can sometimes yield unexpected insights.

Ignoring the rather inexcusable use of the derived adjective “central” in the definition of the noun “centre”, then it is probably the “main area for an activity” sense that is meant to be conveyed in the final “C” of BICC. However, there is also perhaps some illumination to be had in considering another meaning of the word:

As well as appealing to the mathematician in me, this meaning gives the sense that a BICC is physically central geographically, or metaphorically central with respect to business units. Of course this doesn’t meant than a BICC needs to be at the precise centre of gravity of an organisation, with each branch contributing a “weight” calculated by its number of staff, or revenue; but it does suggest that the competency centre is located at a specific point, not dispersed through the organisation.

Of course, not all organisations have multiple locations. The simplest may not have multiple business units either. However, there is a sense by which “centre” means that a BICC should straddle whatever diversity there is an organisation. If it is in multiple countries, then the BICC will be located in one of these, but serve the needs of the others. If a company has several different divisions, or business units, or product streams; then again the BICC should be a discrete area that supports all of them. Often what will make most sense is for the BICC to be located within an organisation’s Head Office function. There are a number of reasons for this:

1. Head Office similarly straddles geographies and business units and so is presumably located in a place that makes sense to do this from (maybe in an organisation’s major market, certainly close to a transport hub if the organisation is multinational, and so on).
2. If a BICC is to properly fulfil the first two letters of its abbreviation, then it will help if it is collocated with business decision-makers. Head Office is one place than many of these are found, including generally the CEO, the CFO, the Head of Marketing and Business Unit Managers. Of course key decision makers will also be spread throughout the organisation (think of Regional and Country Managers), but it is not possible to physically collocate with all of these.
3. Another key manager who is hopefully located in Head Office is the CIO (though this is dispiritingly not always the case, with some CIOs confined to IT ghettos, far from the rest of the executive team and with a corresponding level of influence). Whilst business issues are pre-eminent in BI, of course there is a major technological dimension and a need to collaborate closely with those charged with running the organisation’s IT infrastructure and those responsible for care and feeding of source data systems.
4. If a BI system is to truly achieve its potential, then it must become all pervasive; including a wide range of information from profitability, to sales, to human resources statistics, to expense numbers. This means that it needs to sit at the centre of a web of different systems: ERP, CRM, line of business systems, HR systems etc. Often the most convenient place to do this from will be Head Office.

Thusfar, I haven’t commented on the business benefits of a BICC. Instead I have confined myself to explaining what people mean by the second “C” in the name and why this might be convenient. Rather than making this an even longer piece, I am going to cover both the benefits and disadvantages of a BICC in a follow-on article. Instead let’s now move on to considering the first “C”: Competency.
 
 
Compos centris

Returning to our initial theme of generating insights via an examination of the meaning of words in a non-IT context, let’s start with another dictionary definition:

* People who are not fully conversant with the mysteries of cricket may substitute “batter” here.

To me the important thing to highlight here is that, while it is to be hoped that a BICC will continue to become more competent once it is up and running, in order to successfully establish such a centre, a high degree of existing competence is a prerequisite. It is not enough to simply designate some floor space and allocate a number of people to your BICC, what you need is at least a core of seasoned professionals who have experience of delivering transformational information and know how to set about doing it.

There are many skills that will be necessary in such a group. These match the four main pillars of a BI implementation (I cover these in more depth in several places on the blog, including BI implementations are like icebergs and the middle section of Is outsourcing business intelligence a good idea?):

1. Understand the important business decisions and what figures are necessary to support these.
2. Understand the data available in the organisation, how it relates to other data and to business decisions.
3. Transform the data to provide information answering business questions.
4. Focus on embedding the use of information in the corporate DNA.

So a successful BICC must include: people with strong analytical skills and an understanding of general business practices; high-calibre designers; reliable and conscientious ETL and general programmers; experts in the care, feeding and design of databases; excellent quality assurance professionals; resource conversant with both whatever front-end tools you are using to deliver information and general web programming; staff with skills in technical project management; people who can both design and deliver training programmes; help desk personnel; and last, but by no means least, change managers.

Of course if your BI project is big enough, then you may be able to afford to have people dedicated to each of these roles. If resources are tighter (and where is this not the case nowadays?) then it is better to have people who can wear more than one hat: business analysts who can also design; BI programmers who will also take support calls; project managers who will also run training classes; and so on. This approach saves money and also helps to deal with the inevitable peaks and troughs of resource requirements at different stages in a project. I would recommend setting things up this way (or looking to stretch your people’s abilities into new areas) even if you have the luxury of a budget that would allow a more discrete approach. The challenge of course is going to be finding and retaining such multi-faceted staff.

Also, it hopefully goes without saying that BI is a very business-focussed area and some BICCs will explicitly include business people in them. Even if you do not go this far, then the BICC will have to form a strong partnership with key business stakeholders, often spread across multiple territories. The skill to manage this effectively is in itself a major requirement of the leading personnel of the centre.

Given all of the above, the best way to staff a BICC is with members of a team who have already been successful with a BI project within your organisation; maybe one that was confined to a given geographic region or business unit. If you have no such team, then starting with a BICC is probably a bridge too far. Instead my recommendation would be to build up some competency via a smaller BI project. Alternatively, if you have more than one successful BI team (and, despite the manifold difficulties in getting BI right, such things are not entirely unheard of) then maybe blending these together makes sense. This is unless there is some overriding reason not to (e.g. vastly different team cultures or methodologies. In this case, picking a “winner” may be a better course of action.

Such a team will already have the skills outlined above in abundance (else they could never have been successful). It is also likely that whatever information was needed in their region or business unit will be at least part of what is needed at the broader level of a BICC. Given that there are many examples of BI projects not delivering or consuming vastly more resource than anticipated, then leveraging those exceptional people who have managed to swim against this tide is eminently sensible. Such battle-hardened professionals will know what pitfalls to avoid, which areas are most important to concentrate on and can use their existing products to advertise the benefits of a wider system. If you have such people at the core of your BICC, then it will be easier to integrate new joiners and quickly shepherd them up the learning curve (something that can be particularly long in BI due to the many different aspects of the work).

Of course having been successful in one business unit or region is not enough to guarantee success on a larger scale. I spoke about some of the challenges of doing this in an earlier article, Developing an international BI strategy. Another issue that is likely to raise its head is the political dimension, in particular where different business units or regions already have a management information strategy at some stage of development. This is another area that I will also cover in more detail in a forthcoming piece.
 
 
Conclusions

It seems that simply musing on the normal meanings of the words “competency” and “centre” has led us into some useful discussions. As mentioned above, at least two other blog postings will expand upon areas that have been highlighted in this piece. For now what I believe we have learned so far is:

• BICCs should (by definition) straddle multiple geographies and/or business units.
• There are sound reasons for collocating the BICC with Head Office.
• There is need for a wide range of skills in your BICC, both business-focussed and technical.
• At least the core of your BICC should be made up of competent (and experienced) BI professionals .

More thoughts on the benefits and disadvantages of business intelligence competency centres and also the politcs that they have to negotiate will appear on this blog in future weeks.
 

	Tweet this article on twitter.com

Bookmark this article with:
Technorati	| del.icio.us	| digg	| Reddit	| NewsVine

In a previous article, A more appropriate metaphor for business intelligence projects, I explained one complication of business intelligence projects. This is that the frequently applied IT metaphor of building is not very applicable to BI. Instead I suggested that BI projects had more in common with archaeological digs. I’m not going to revisit the reasons for the suitability of looking at BI this way here, take a look at the earlier piece if you need convincing, instead I’ll focus on what this means for project estimation.

When you are building up, estimation is easier because each new tier is dependent mostly on completion of the one below, something that the construction team has control over (note: for the sake of simplicity I’m going to ignore the general need to dig foundations for buildings). In this scenario, the initial design will take into account of facts such as the first tier needing to support all of the rest of the floors and that central shafts will be needed to provide access and deliver essential services such as water, electricity and of course network cables. A reductionist approach can be taken, with work broken into discrete tasks, each of which can be estimated with a certain degree of accuracy. The sum of each of these, plus some contingency, hopefully gives you a good feel for the overall project. It is however perhaps salutary to note that even when building up (both in construction and in IT) estimation can still sometimes go spectacularly awry.

When you are digging down, your speed is dependent on what you find. Your progress is dictated by things that are essentially hidden before work starts. If your path ahead (or downwards) is obscured until your have cleared enough earth to uncover the next layer, then each section may hold unexpected surprises and lead to unanticipated delays. While it may be possible to say things like, “well we need to dig down 20m and each metre should take us 10 days”, any given metre might actually take 20 days, or more. There are two issues here; first it is difficult to reduce the overall work into tasks, second it is harder to estimate each task accurately. The further below ground a phase of the dig is, the harder it will be to predict what will happen before ground is broken. Even with exploratory digs, or the use of scanning equipment, this can be very difficult to assess in advance. However it is to the concept of exploratory digs that this article is devoted.
 
 
Why a feasibility study is invaluable

At any point in the economic cycle, even more so in today’s circumstances, it is not ideal to tell your executive team that you have no idea how long a project will take, nor how much it might cost. Even with the most attractive of benefits to be potentially seized (and it is my firm belief that BI projects have a greater payback than many other types of IT projects), unless there is some overriding reason that work must commence, then your project is unlikely to gain a lot of support if it is thus characterised. So how to square the circle of providing estimates for BI projects that are accurate enough to present to project sponsors and will not subsequently leave you embarrassed by massive overruns?

It is in addressing this issue that BI feasibility studies have their greatest value. These can be thought of as analogous to the exploratory digs referred to above. Of course there are some questions to be answered here. By definition, a feasibility study cannot cover all of the ground that the real project needs to cover, choices will need to be made. For example, if there are likely to be 10 different data sources for your eventual warehouse, then should you pick one and look at it in some depth, or should you fleetingly examine all 10 areas? Extending our archaeological metaphor, should your exploratory dig be shallow and wide, or a deep and narrow borehole?
 
 
A centre-centric approach

In answering this question, it is probably worth considering the fact that not all data sources are alike. There is probably a hierarchy to them, both in terms of importance and in terms of architecture. No two organisations will be the same, but the following diagram may capture some of what I mean here:

Two ways of looking at a systems' hierarchy

The figure shows a couple of ways of looking at your data sources / systems. The one of the left is rather ERP-centric, the one on the right gives greater prominence to front-end systems supporting different lines of business, but wrapped by a common CRM system. There are many different diagrams that could be drawn in many different ways of course. My reason for using concentric circles is to stress that there is often a sense in which information flows from the outside systems (ones primarily focussed on customer interactions and capturing business transactions) to internal systems (focussed on either external or internal reporting, monitoring the effectiveness of processes, or delivering controls).

There may be several layers through which information percolates to the centre; indeed the bands of systems and databases might be as numerous as rings in an onion. The point is that there generally is such a logical centre. Data is often lost on its journey to this centre by either aggregation, or by elements simply not being transferred (e.g. the name of a salesperson is not often recorded on revenue entries in a General Ledger). Nevertheless the innermost segment of the onion is often the most complex, with sometimes arcane rules governing how data is consolidated and transformed on its way to its final destination.

The centre in both of the above diagrams is financial and this is not atypical if what we are considering is an all-pervasive BI system aimed at measuring most, if not all, elements of an organisation’s activity (the most valuable type of BI system in my opinion). Even if your BI project is not all-pervasive (or at least the first phase is more specific), then the argument that there is a centre will probably still hold, however the centre may not be financial in this case.

My suggestion is that this central source of data (of course there may be more than one) is what should be the greatest focus of your feasibility study. There are several reasons for this, some technical, some project marketing-related:

1. As mentioned above, the centre is often the toughest nut to crack. If you can gain at least some appreciation of how it works and how it may be related to other, more peripheral systems, then this is a big advance for the project. Many of the archaeological uncertainties referred to above will be located in the central data store. Other data sources are likely to be simpler and thus you can be more confident about approaching these and estimating the work required.
2. A partial understanding of the centre is often going to be totally insufficient. This is because your central analyses will often have to reconcile precisely to other reports, such as those generated by your ERP system. As managers are often measured by these financial scorecards, if you BI system does not give the same total, it will have no credibility and will not be used by these people.
3. Because of its very nature, an understanding of the centre will require at least passing acquaintance with the other systems that feed data to it. While you will not want to spend as much time on analysing these other systems during the feasibility study, working out some elements of how they interact will be helpful for the main project.
4. One output from your feasibility study should be a prototype. While this will not be very close to the finished article and may contain data that is both unreconciled and partial (e.g. for just one country or line of business), it should give project sponsors some idea of what they can expect from the eventual system. If this prototype deals with data from the centre then it is likely to be of pertinence to a wide range of managers.
5. Strongly related to the last point, and in particular if the centre consists of financial data, then providing tools to analyse this is likely to be something that you will want to do early on in the main project. This is both because this is likely to offer a lot of business value and because, if done well, this will be a great advert for the rest of your project. If this is a key project deliverable, then learning as much as possible about the centre during the feasibility study is very important.
6. Finally what you are looking to build with your BI system is an information architecture. If you are doing this, then it makes sense to start in the middle and work your way outwards. This will offer a framework off of which other elements of your BI system can be hung. The danger with starting on the outside and working inwards is that you can end up with the situation illustrated below.

A possible result of building from the outside in to the centre

So my recommendation is that your feasibility study is mostly a narrow, deep dig, focussed on the central data source. If time allows it would be beneficial to supplement this with a more cursory examination of some of the data sources that feed the centre, particularly as this may be necessary to better understand the centre and because it will help you to get a better idea about your overall information architecture. You do not need to figure out every single thing about the central data source, but whatever you can find out will improve the accuracy of your estimate and save you time later. If you include other data sources in a deep / wide hybrid, then these can initially be studied in much less detail as they are often simpler and the assumption is that they will support later deliveries.

The idea of a prototype was mentioned above. This is something that is very important to produce in a feasibility study. Even if we take to one side the undeniable PR value of a prototype, producing one will allow you to go through the entire build process. Even if you do this with hand-crafted transformation of data (rather than ETL) and only a simplistic and incomplete approach to the measures and dimensions you support, you will at least have gone through each of the technical stages required in the eventual live system. This will help to shake out any issues, highlight areas that will require further attention and assist in sizing databases. A prototype can also be used to begin to investigate system and network performance, things that will influence your system topology and thereby project costs. A better appreciation of all of these areas will help you greatly when it comes to making good estimates.

Having understood quite a lot about your most complex data source and a little about other ones and produced a prototype both as a sales tool and to get experience of the whole build process, you should have all the main ingredients for making a credible presentation to your project sponsors. In this it is very important to stress the uncertainties inherent in BI and manage expectations around these. However you should also be very confident in stating that you have done all that can be done to mitigate the impact of these. This approach, of course supported by a compelling business case, will position you very well to pitch your overall BI project.
 

	Tweet this article on twitter.com

Bookmark this article with:
Technorati	| del.icio.us	| digg	| Reddit	| NewsVine