A more appropriate metaphor for Business Intelligence projects

A traditional metaphor for IT projects
A traditional metaphor for IT projects

IT people are familiar with a number of metaphors for their projects. The most typical relates to building; IT projects are compared to erecting a skyscraper. The IT literature is suffused with language derived from this metaphor. We build systems. We develop blueprints for them. We design architectures (two-for-one there). This analogy has some strength and there are indeed superficial similarities between the two areas. However, as with most metaphors, if over-extended their applicability often breaks down. I recall one CEO in particular who was obsessed by the “building team” moving on to the next “site”; regardless of the current one requiring further work and dedicated maintenance. One of his predecessors often referred to wanting a “diesel submarine” built, as opposed to a “nuclear one”. Before I fall into the same trap of over-exploiting the metaphor, let’s move hurriedly on.

As I mention above, aside from the occasional misapplication, the building analogy works reasonably well for many IT projects; does it also work for business intelligence? I think that there are some problems in applying the metaphor. Building tends to follow a waterfall project plan (as do many IT projects). Of course there may be some iterations, even many of them, but the idea is that the project is made up of discrete base-level tasks whose duration can be estimated with a degree of accuracy. Examples of such a task might include writing a functional specification, developing a specific code module, or performing integration testing between two sub-systems. Adding up all the base-level tasks and the rates of the people involved gets you a cost estimate. Working out the dependencies between the base-level tasks gets you an overall duration estimate.

The problem with BI projects is that some of the base-level tasks are a bit different. An example might be: develop an understanding of a legacy data table, how it relates to other legacy data sets and to more modern systems (this sits under area two of my model of BI development – see BI implementations are like icebergs). This is not an exercise that is very easy to estimate in advance. Indeed it may not be possible to produce an adequate estimate until a substantial amount of work has been done. Even at a late stage in the task, something may be discovered which expands the work required dramatically; surprises may lurk round every corner.

Why is this? Well with legacy data, the people who developed the system may have done so many years ago. Since then, they may have left the company or moved on to other areas, taking their knowledge with them. Their place may have been taken by successive tranches of new staff. Perhaps poor initial documentation meant that later workers did not fully understand the full nature of the system, but nevertheless did their best to build upon it. Perhaps the documentation was good at first, but has not been kept up-to-date and now describes a system that no longer exists.

By definition, legacy systems will have been around for some time and layers of changes will have accumulated on top of each other. Maybe, as a company has expanded, new data has been interfaced to the system from different business units and territories; perhaps each of these cases has its own dedicated interface code, each subtly different from those of other systems. Even where people exist in an organisation who preserve an “oral tradition” about the system, handed down to them over generations; these people may not appreciate how their data interacts with other data – even if the person who looks after another legacy system sits in the adjacent cubicle.

Waterfall Project Plan (intentionally blurred somewhat)

Although these challenges can also occur when trying to understand the data in more modern systems, they are particularly acute with older ones. For a start, the people who designed these systems are more likely to be around. Also legacy systems often sit at the centre of a Byzantine web of inter-connections, batch-processes, over-night jobs and the occasional more modern service. It can be a real mess and this is a situation with which any data analyst with a reasonable amount of experience will be very familiar.

The difficulty of estimating the duration of tasks such as properly analysing legacy data makes overall estimation of BI projects more of an art than a science. Of course techniques such as time-boxing tasks can be applied, but these are not always 100% appropriate. A 75% analysed data source (even assuming that the estimate that only 25% work is left is accurate) is not an analysed data source. Leaving dark corners of knowledge is likely to be reflected in BI cubes and reports that do not reconcile back to their sources. Probably the best way to deal with this problem is to be extremely open about the challenges up-front with executive sponsors and when submitting estimates. It helps to also stress the level of uncertainty in progress reports. The more honest you are initially, the better you will be able to explain any overruns and the more likely it is that you will be believed.

These types of issues mean that the – hopefully more orderly – process of constructing a building is not a fully accurate way to describe a BI project. That is unless the metaphor is extended to include an occurance that is all too common during construction in The City of London. Given the age of Londinium, whenever ground is broken on a new project, it is more likely than not that a mediaeval, Anglo-Saxon or Roman site is unearthed (often all three). These finds, while of enormous interest to academics, can result in projects being put on hold (sometimes for years) while the dig is fully assessed, artefacts are carefully removed and catalogued by experts and so on. Sometimes the remains are of such importance that a structure preserving and protecting them (and even allowing public viewing) has to be made part of the design of the foundations of new building. Many office blocks in The City have such viewing galleries in their basements. Such eventualities can create massive and unexpected overruns in central London building projects.

So this leads me to suggest a different metaphor for BI projects. Major elements of them are much more like archaeological digs than traditional building. The extent and importance of a dig is very difficult to ascertain before work starts and both may change during the course of a project. It is not atypical that an older site is discovered underneath an initial dig, doubling the amount of work required.

So, my belief is that BI professionals should not be likened to architects or structural engineers. Instead the epithet of archaeologist is much more appropriate. And if the fedora fits, wear it!

Fortune and glory in BI?
Fortune and glory in BI?


Apologies for the initial typo’ in the article heading, which persists in the perma-link. Of course I have the odd error scattered around the place, but in a heading! Maybe I need to employ a proof-reader.

The Top Business Issues facing CIOs / IT Directors – Results

Back in January, in collaboration with Chase Zander, I started a process of consulting with senior IT managers to develop a list of the top business issues that they faced. This exercise was intended to shape the content of a IT Director Forum that we were planning. This will now be happening on 26th March in Birmingham (for further information see this post).

Questionaire Responses
The Top Business Issues faced by CIOs / IT Directors

Back then, I promised to share some of the findings from this study. These are summarised in the above diagram. The input was based on public comments made by a selection of senior people on the CIO group of LinkedIn.com, plus e-mails sent to me on the topic and feedback received by Chase Zander.

A textual version of the data appeas below (sample size ~60):

  Issue % of Votes  

  IT / Business Alignment 27%  
  Cost-saving 13%  
  Managing change 8%  
  Status of the IT Director 8%  
  Legacy Systems 5%  
  Customer focus 5%  
  Enterprise Architecture 5%  
  Business Intelligence 5%  
  Avoiding the latest and greatest 3%  
  Cloud Computing 3%  
  Only one response 17%  

  Total 100%  

I would like to thank all of the IT professionals who contributed to this survey.

Is outsourcing Business Intelligence a good idea?


The phrase IT outsourcing tends to provoke strong reactions. People either embrace it as a universal panacea capable of addressing any business problem, or recoil in horror at the very sound of it. Just for a change, I am somewhere in the middle; to me it is another tool at the disposal of businesses which can either be used wisely or poorly (much like IT itself you might say). As always the difference between the two extremes comes down to how well the project is led. Regardless of this, there are some benefits and some disbenefits associated with IT outsourcing and this article will explore the case for applying outsourcing to business intelligence.
Benefits of general IT outsourcing

Before I plunge into the world of BI, it is perhaps worth revisiting the general reasons for IT outsourcing, some of the most regularly quoted are as follows:

1. Reduction in costs

The provider of outsourcing (I’m just going to say “the provider” from now on to save typing) can carry out the same tasks at a cheaper cost to the client organisation (while still presumably turning a profit). There can be a number of bases for this; the one that generally comes to mind is wage arbitrage between different economies. However, it could also be that the provider has economies of scale; for instance, less people being required to run the consolidated data centres of several companies, than is required to run each separately. Also the provider may have staff who are more productive than at the client.

2. Ability to scale-up and scale down resource

The nature of business is such that sometimes all hands are required on the IT deck and at others there is spare capacity (this is something I address in my two articles on Problems associated with the IT cycle and Mitigating problems with the IT cycle). Now IT departments are normally quite good at finding (hopefully) useful things for people to do, but the issue remains. The promise of an outsourcing arrangement is that the tap of resource can be adjusted to meet demand without having to either fire and rehire staff, or rely on bringing in expensive contract resource. It is often hoped that this feature of outsourcing will also help to speed IT products to market.

3. Making IT provision a contractual relationship

An arrangement with a provider, depending on how the contract is drafted, can make the provision of IT services subject to penalties and claw-backs when service levels drop below those that have been agreed. While there are clearly some sanctions that can be applied to underperformance by internal IT departments, the financial benefit to the organisation is likely to be less (unless your CIO is a multi-billionaire of course). Companies are used to these contractual relationships, they are often the lifeblood of business, and it is a more familiar way of dealing with issues for them.

4. Access to skills

The nature of IT is that it does tend to evolve, sometimes quickly, sometimes slowly. For organisations this means keeping their IT people’s skills up to date though courses, or continually looking to bring people with new skills into an organisation (such people generally not being the cheapest). The idea with an outsourcing arrangement is that these issues become the headache of the provider, not the client. This area can be particularly pertinent when there is a technology change or a significant upgrade; these are times at which the prospect of being shot of IT worries may seem very attractive. The effort and cost of, as it were, upgrading your in-house IT staff may seem prohibitive in these circumstances.

5. Focus on core competencies

This has been a business mantra for many years, why should a company engaged in a wholly separate area of human endeavour want to become experts in building and supporting complex IT systems, when they can get a specialist organisation to do this for them? This moves towards the idea of a lean, or even virtual, organisation.

6. Failure of in-house IT

It is sad to have to add this item, but it is often the implicit (and sometimes even the explicit) driver of a desire to outsource. CEOs, COOs or CFOs may be so fed up with the performance of their IT people that they feel that surely someone else could not be worse. There is an adage that you don’t outsource a problem, but this is often honoured more in breech than observance.

I am sure that there are other advantages, claimed or real, for IT outsourcing, but the above list at least covers many of the normal arguments. At this stage a fully-balanced article would probably present arguments against IT outsourcing. However, my objective here is not to provide a critique of IT outsourcing in general, but to see whether the above benefits apply to business intelligence. Because of this, and I should stress purely for the purposes of this article, I am going to accept that all of the above gains are both realisable and desirable for general IT. There will therefore you will find no comments here about arbitrage (of its very nature) resulting in differentials of pricing closing over time.

The only benefit that I am going to rule out is the final one; addressing failed IT departments. Applying outsourcing in these cases is only likely to make things worse, and probably more expensive. Far better in my opinion to work out why IT is failing (most typically due to poor leadership it has to be said, see also my article: Some reasons why IT projects fail) and draw up plans for addressing this. If outsourcing is a strong element of this, then so be it, but thinking that it will resolve this type of issue is probably naive in most circumstances.

So, as always seems to be the case in these types of articles, we have five potential benefits against which to assess outsourcing BI. Before I look at each in turn, I wanted to make some general observations.
Things that are different about BI

The main fly in the ointment with respect to outsourcing business intelligence is the fact that good BI is reliant upon four things (see also BI implementations are like icebergs):

A. An in-depth understanding of business requirements, developed by close collaboration with a wide range of business managers. In particular, what is necessary is understanding what questions the business wants to ask and why (see Scaling-up Performance Management and Developing an international BI strategy)
B. An extensive appreciation of the data available in different business systems, its accuracy and how data in different places is related to each other.
C. Developing creative ways of transforming the available data into the required information and presenting this in an easy-to-understand and use manner.
D. A focus on change management that includes business-focussed marketing, training and follow-up to ensure that the work carried out in the first three areas results in actual business adoption and thereby the creation of value (see my collection of articles focussed on cultural transformation).

With the possible exception of item C., which is more technical, the above are best carried out in a symbiotic relationship with the business. Ideally what develops is a true IT / business hybrid team, where, though people have clear roles, the differences between these blur into each other. In turn, building thus type of team is predicated on developing strong relationships between the IT and business members and establishing high levels of trust and respect.

Also with item C., this is not precisely a stand-alone activity. It is one best carried out collaboratively by technically-aware business analysts and business-aware data analysts, ETL programmers and OLAP designers. Once again, distinctions blur somewhat during this work and a different type of hybrid team appears.

I have tried to illustrate the way that these tasks and teams should overlap in the following diagram.


Clearly it is not impossible to achieve what I have described above in an outsourced environment, but it seems that it might be rather tougher to do this. One key point is that the type of skills that are necessary for success in BI are cross-over business / IT skills and these are generally less easy to buy off the shelf. Another is that the type of intellectual property that a BI team will build up (basically extensive knowledge of what makes the organisation tick) is precisely the sort that you would want to retain within an organisation.

I would suggest that if an organisation wants to outsource BI, then they should start that way. Once a BI team has gone through tasks A. to D. above then I can’t see how it would be cost-effective to subsequently outsource. The transfer of knowledge would take too long and be too costly.

To provide some context to this let me share some non-confidential details of a study I performed recently comparing the efficiency of a well-established BI team in a developed country with a less mature BI team in a lower-cost location. Rather than considering relative costs, I looked at relative productivity. A simple way to do this is to get quotes for carrying out a certain type of work from both teams (though I also applied some other techniques, which I won’t go into here). My main finding was that the ostensibly high cost team was more than twice as productive as the allegedly low-cost team. Just to be clear, if the “high-cost” team quoted $X for a piece of work, the “low-cost” team quoted over $2X,because they required much more resource and/or time to carry out the same work.

So, in what follows, I will assume that a decision is taken to outsource at the inception of a project. With this assumption and the previous background, let’s go back and look at the five benefits of outsourcing from the beginning.
Matching the benefits to BI

1. Reduction in costs

It will take external BI resource at least as long as internal BI resource to understand business requirements and available data. In fact internal staff probably have something of an advantage as they should already have an appreciation of what the organisation does and how IT systems support this. The external resource also has the disadvantage of it probably being more difficult for them to build business relationships, this can be exacerbated if there are personnel changes during the project; something that is perhaps more likely to happen with an external provider. If the provider is located in another country, then this raises even more challenges and inefficiencies (and leads to travel expense).

It will take an external BI team at least as long as an internal one to dig into the available data and how the various systems inter-relate. Again, having some familiarity with the existing systems’ landscape would be an advantage for an in-house team.

If an external team can get to the position where they understand the business needs and the available data really well in a reasonable period of time, then they could possibly have an advantage in the arena of transforming data into information. Something that may mitigate this however is that fact that most BI development is iterative and that a rolling set of prototypes needs to be reviewed closely with the business. This element introduces the same challenges as were apparent with defining business requirements above.

Similar arguments as were made about the business requirements phase apply to deployment and follow-up.

2. Ability to scale-up and scale down resource

While it may be possible (subject to contract) to scale-down resource with a provider (though perhaps tougher to get them back when you need them), scaling-up is just as hard as it is in-house at it means more staff at the provider going through the learning curve about the organisations business needs and data.

4. Access to skills

This is the crux of the matter. The skills in question are not Java programming (or even Cobol), they are business knowledge. ETL and OLAP skills are important, but only if they are applied by people who understand what they are doing and to what purpose. These skills are not just lying around in the market place; they are acquired through hard work and dedication.

3. Making IT provision a contractual relationship

Clearly this is a benefit of outsourcing. However, given that the contract is there for when things go awry, it is worth asking the question “are things more or less likely to go wrong with a provider?”

5. Focus on core competencies

While it is quite easy to argue that building e-commerce systems is not necessarily a core competency, good BI is about understanding what is necessary to best run the business. If that is not a core competency of any organisation, then I struggle to think of what would be.

My main argument is that BI is different to general IT projects (an assertion to which I will return in a forthcoming article). Having successfully run both, I am confident in this statement. I also think that you need different types of people with different skills in BI projects. These facts, plus the closeness of business / IT relationships which are necessary in the area mean that outsourcing is less likely to be effective. I am sure that an outsourcing arrangement can work well for some organisations in some circumstances, but I would argue strongly against it being best practise for most organisations most of the time.

After penning this article, a further problem with outsourcing business intelligence came to my mind; security. On part of most BI systems is a facility to analyse the organisation’s results. Ideally the BI system will have these figures in place very soon after the end of a financial closing. Such data is market sensitive and there may be concerns with trusting an external provider with both producing this and ensuring that it remains confidential until market announcements are made. I am not suggesting that providers are unethical, just that companies may not wish to take a chance in this area.
I should also credit a thread on the LinkedIn.com EPM – Business Intelligence group, which got me thinking about this area (as ever, you need to be a member of LinkedIn.com and the group to view this)


“Gartner sees a big discrepancy between BI expectations and realities” – Intelligent Enterprise

Doug Henschen
Doug Henschen at Intelligent Enterprise reports from the Gartner Business Intelligence Summit in Washington D.C., explaining that Gartner analysts John Van Decker and Kurt Schlegel cited five reasons why BI projects do not live up to expectations (article link here):

  1. No IT-Business Partnership – “We have to get away from thinking of it as a vendor-customer relationship,” said Schlegel.
  2. No Link to Corporate Strategy – BI team have to listen to the executives and develop metrics and measures that are aligned with their goals.
  3. No connection to the Process – “BI has been overtly disconnected for too long,” Schlegel proclaimed. It’s not enough to deliver the right information to the right users. You have to insert those insights into the processes and interfaces that business users live in every day.”
  4. No Governance or Too Much – It has to be just the right amount of governance. BI grew up departmentally, so it’s all too common to have many silos of BI. At the other extreme, some companies are so uptight about data standards that companies can’t get their data into the warehouse.
  5. No Skills – Business users often lack skills, chimed in Van Decker, citing the one area where the business side was said to be falling short. “Most companies have very sophisticated capabilities available that folks just aren’t taking advantage of,” he said, pointing to corporate performance management (CPM) software as a leading example.

I come close to the situation of words failing me when I read a list like this (though not close enough to prevent me penning an article of course). I appreciate that maybe a public seminar is not the easiest place to provide deep insights (I present a lot myself), but the commentary against each of the problem areas seems odd to me. I’m not sure that these are really the five reasons for BI continuing to disappoint in some organisations, while it continues to delight in others, but here are my thoughts on each headline.

  1. No IT-Business Partnership – We have to stop talking about IT and Business as if they were two parties trying to negotiate a cease-fire. IT is a business department, it carries out business projects. It would be ridiculous to say that there needs to be a Sales-Business Partnership, it should be equally so with IT. I expand on this theme in the very first article on this blog.
  2. No Link to Corporate Strategy – There should not be a link to Corporate Strategy, BI does not exist as a separate entity that requires linkage. Instead BI work should be an expression of Corporate Strategy (as should any IT project), what else is it an expression of? This is not about listening to executives (though that is important) it is about IT being part of the senior management team of an organisation and not some semi-detached entity, focussed only on the beauty of its own navel. I give some indication of how to go about ensuring that this is the case in two articles, one focussed on a European environment and one spanning four continents.
  3. No connection to the Process – I agree that embedding good BI in a coporoate culture requires effort (indeed I have written a series of articles on the subject, starting with this one), however I struggle to see how any BI team could fail to realise this and pay the area due attention. If this is truly a reason for failure, then it is because of a lack of basic project and change management skills in BI teams.
  4. No Governance or Too Much – I’m not sure that achieving the Goldilocks approach to Governance is the issue here. BI’s impact on an organisation is directly proportional to how pervasive it is. This means that silos of BI reduce value and the walls need to be knocked in. Does this have to be all in one go? of course not, there are always benefits in a balance between incremental progress and paradigm shifts. Any organisation who has gatekeepers who refuse access to the warehouse because of and overly rigid approach to data standards is going to have problems. Equally where systems are developed with little thought to the information that they provide and data is simply thrown over the wall to the BI team, this is going to destroy value. As ever, there needs to be a sensible balance struck, one that yields the greatest business benefit for the lowest cost.
  5. No Skills – “Business users often lack skills”, this seems both incredibly patronising (are only IT people smart enough to get it?) and also a poor excuse for BI teams not paying enough attention to education (see point 3. above). If business people truly lack the skills to use good BI, then they are probably unfit to be in business as the tools are pretty intuitive (if not over-engineered by an approach that is too technology-focussed). More likely, training has been poor, or the BI deliveries have failed to be tailored to answering questions that the business wants to ask.

In summary, I don’t have five reasons for BI failing to live up to expectations, I have one. I firmly believe that BI done well is both the easiest of IT systems to sell to people and has one of the highest paybacks of any IT initiative. BI done badly (at the design, development, implementation or follow-up stages) will fail.

The issue is basically a simple one: just how good is your BI team? If a BI implementation fails to deliver significant business value, then instead of looking for scape-goats, the BI team should purchase a mirror and start using it.

Continue reading about this area in: Some reasons why IT projects fail
Doug Henschen joined Intelligent Enterprise as Editor in 2004 and was named Editor-in-Chief in January 2007. He has specialized in covering the intersection of business intelligence, performance management, business process management and rules management technologies within enterprise applications and architectures. He previously served as Editor-in-Chief of Transform Magazine, which covered content management and business process management challenges.

I comment on another Intelligent Enterprise article, this one by Cindi Howson, here.

Trends in Business Intelligence


This year, as in every year since the phrase “Business Intelligence” first came to prominence, there have been a rash of predictions about what will happen with the area in 2009. I have linked to and commented on some of these myself on this blog. This article is my take on the area. I should however explain that there is a twist. These are not the trends that I expect to see in 2009, just my thoughts on what ought to be trends in BI over the next few years. I cannot claim to have any prescience about whether they will come to pass, but I will be encouraged if they do.

  1. A more holistic approach to BI in organisations that have already invested
  2. Operational BI
  3. Consolidation of the number of BI platforms with organisations
  4. An increase in the prevalence of BI competency centres
  5. BI teams being jointly owned and managed by business divisions and IT
  6. Data from central warehouses being served to front-end applications
  7. BI data being used to automate some business decisions
  8. Further emphasis on predictive analytics
  9. BI having an increasing role in compliance and risk management
  10. Incorporation of external data into BI platforms
  11. Provision of BI to business partners and/or customers
  12. The most creative users of BI employing it to thrive, even in the current climate

1. A more holistic approach to BI in organisations that have already invested

Often BI solutions have started in a single area. Typically – given its direct link to better managing overall performance – this has been around analysing an organisation’s financial results, though implementations starting in the sales and even operational arenas are also not uncommon. BI systems tend to add more value as they bring in more information, particularly where a high-level phenomenon – e.g. a decrease in expenditure – can be attributed to lower level ones – e.g. greater repeat business (business acquisition being more expensive than repeat business in most industries), plus enhanced productivity, plus reduced staff turnover (resulting in lower recruitment and training costs). To do this, BI needs to widen its scope to take on new data sources and combine them in creative ways. Where BI platforms have already added value, there will be pressure from users to expand them to deliver even more utility and provide more sophisticated insights. This in turn will require BI practitioners to take a more holistic view of the area and to develop roadmaps explaining how new data sources will be brought on stream and what business value will result from this.
2. Operational BI

I think that there is a particular argument for BI to make a greater contribution in the area of operational management, particularly tying this to overall performance. Often BI implementations have focussed on the more stately world of monthly (or weekly) results and steered clear of the daily or hourly demands of operational information. The BI projects that I have seen in the Operational sphere have been more focussed at producing weekly status reports or year-to-date trend analyses. Related to BI platforms expanding to new areas, I see increasing demand for information about the internal effectiveness of an organisation; quickly identifying bottlenecks or areas of inefficiency and tracking efforts to address these. Supporting this type of reporting will often require BI practitioners to rethink their architectures which are often set up to refresh on longer timeframes. This may in turn require the warehouse to support multiple environments with different periodicities.
3. Consolidation of the number of BI platforms with organisations

It is all too common for organisations to have separate BI implementations. Maybe the ERP system comes with some shrink-wrapped cubes, as does the CRM system. Perhaps Finance have invested in some budget analysis tools and different business units have started dabbling in predictive modelling (see 8. below). Maybe certain power users or numerate departments have their own, much cherished databases. Clearly this approach is sub-optimal. If it was ever acceptable to have such an ad hoc approach to the area, the current economic climate means that there is no longer room for a multitude of platforms, each supported by their own ETL, servers, IT teams and (hopefully) training programmes. While transitional costs may make it impractical for some organisations to move to a single platform in the short term, the reductions in licensing, internal support, data centre and training costs that come from standardising BI tools are compelling. This is to say nothing about reducing the level of confusion in users faced with multiple reporting and analysis systems, each with their own terminology, vagaries of functionality and often un-reconciled to each other. Arguments about certain BI tools being best of breed for particular tasks were never wholly convincing, with the breadth of functionality offered by all of the major players today, any of them will be at least good enough for all but the most exacting of applications.
4. An increase in the prevalence of BI competency centres

If this is partly as a result of the previous three trends, it also has some drivers all of its own. Even in multinational organisations with highly devolved structures and local accountability, most of the management information that is needed to run businesses will be relatively consistent from country to country and business unit to business unit. Perhaps the sources will vary, but the business transactions that they support are surprisingly consistent. Many organisations are waking up to this fact and pulling BI provision into the centre. There are obvious benefits to this in terms of cost savings, not reinventing the wheel, increasing the consistency of reporting and enabling corporate roll-ups. However, my own feeling is that the best organisations will supplement a strong central resource with a (probably much smaller) virtual component located close to business needs who can better respond to these in a timely manner, but within an overall information architecture. This hybrid approach offers the best of both worlds.
5. BI teams being jointly owned and managed by business divisions and IT

It is an oft-repeated aphorism that all IT projects are business projects. While clearly true in theory, there are enough counterexamples to suggest that practise is rather different. However, BI projects have often stayed much closer to this maxim than other IT efforts. This is because BI systems serve no purpose unless they are closely entwined with business goals. Natural selection should weed out any non-business-focussed BI projects eventually; even in the sleepiest of organisations. It is likely that this close relationship will deepen. Whilst many aspects of BI are firmly in the IT arena (managing the regular refresh of multiple terabytes of data effectively clearly being one), I see a joint stewardship of the area developing. By this I mean something deeper than business oversight or steering committees. It is also different to business BI liaison managers being appointed – these roles often have the unintended consequence of isolating IT from its customers. What I see emerging in more enlightened organisations is true co-ownership of BI with business and IT management closely collaborating to run the area.
6. Data from central warehouses being served to front-end applications

Of course it is not uncommon for transaction processing systems to have buttons or links that call up a given report. What I am talking about here is a closer coupling, one that does not require users to leave their current system and where information from the warehouse is presented in a seamless manner to users. While such information will have all the BI benefits of being reconciled, consistent and accurate, its provenance will increasingly become invisible to the user (who shouldn’t really have to worry where figures come from so long as they are accurate). This is an area in which web-services have some real potential to leverage the investments that organisations have made in warehouses.
7. BI data being used to automate some business decisions

Stepping a little further along the path from the previous point, even before users of front-end systems have a need to review information about a transaction, it may well be that the information itself has been what determines whether a human is involved or not. Already some organisations perform triage on business transactions. Ones that meet all of a number of requirements get processed automatically. Ones that meet some of them, but not all, may get routed to junior staff, whose role is to determine whether they require further review or can proceed. Finally, those with a major variance to requirements may get sent straight to more senior staff. This approach means that a larger volume of business can be handled and that expensive senior resource is only applied where it is necessary. Of course a prerequisite to this type of approach is having reliable information on which to perform the triage.
8. Further emphasis on predictive analytics

While the best BI implementations encourage users to extrapolate from past figures to estimate future ones, the degree to which the analytical skills of the user plays a part varies from implementation to implementation. Here by analytics I mean the use of larger data sets to identify trends or exceptions, mostly at a portfolio level. Again, having invested in developing data warehouses, it makes sense to utilise the many and sophisticated tools that are available to carry out advanced statistical analysis on these; the aim being to deduce trends that would be beyond even the most skilled of human analysts. A by-product of successful BI implementations is that they often free the more numerate of people in organisations from the burden of repetitive number crunching and allow them to focus on more added-value work such as this.
9. BI having an increasing role in compliance and risk management

Sometimes BI projects may have had their genesis in these areas. However, even when this is not the case a pleasing result of having consolidated much of the organisation’s data in one place is that this then forms a valuable resource for compliance and risk management. Indeed, taking an external perspective, regulators tend to view the existence of an enterprise data warehouse as a sign that an organisation takes managing its risks seriously. Although business managers and risk managers may have slightly different (though hopefully complementary) perspectives and want to answer different types of questions, the data that they need to do this is not that dissimilar. Producing compliance suites from a well-designed warehouse is probably not one of the more taxing BI problems. Again expansion in this area is a further example of point 1. in action.
10. Incorporation of external data into BI platforms

I have spoken above about the remit of BI systems expanding internally and becoming more consistent geographically. A further trend in expansion is to meld internal information with that from external providers. The type of external information would range from industry to industry, but might include market data, information about specific companies or individuals (e.g. credit scores, where the use of these is admissible). For example, in insurance, where I have spent the last twelve years of my career, incorporating information from externally produced flood, or windstorm models is often a priority.
11. Provision of BI to business partners and/or customers

Sometimes one objective of BI implementations is to better understand the relationships with business partners and customers. I have seen this develop to the degree where output from BI systems is mailed to such counterparties. A logical extension of this is to allow such organisations direct access to “their information”. Of course as with any e-commerce initiative, there would have be to strict controls on what is viewed and who can view it, but these are problems that are regularly addressed in other areas of IT and the tools to do this are readily available.
12. The most creative users of BI employing it to thrive, even in the current climate

There have been many articles (including ones written by me) which have spoken about good BI being a great defence in times of economic stress. I would go beyond this and state that the real BI pioneers will take advantage of these capabilities to capture markets from their less well-informed competitors and to steer a course away from areas of business that may bring other less-foresighted organisations down. I look forward to seeing case studies bearing this out appear over the next few years.

“Businesses Are Still Crazy for BI After All These Years” – CIO.com

Thomas Wailgum at CIO.com

Thomas Wailgum has written an article at CIO.com in which he talks about continuing demand for BI, but adds that this, in turn, suggests that in many organisations BI has yet to deliver on its promise. As Thomas puts it:

“I see pent-up enterprise-wide frustration, aimed squarely at IT and CIOs for failing to give the business what it needs and deserves”

He sees the fundamental problem as being fragmented systems and stand-alone BI applications. This sounds like challenges that I have faced before. I agree that BI only realises it potential when a more strategic and wide-ranging approach is taken. Something I refer to in many places on this blog, but possibly most directly in Holistic vs Incremental approaches to BI.

My basic point is that while it is sensible to take a pragmatic, incremental approach to implementing BI (collecting successes as you go and building momentum), this needs to be within the framework of a more encompassing vision for what the eventual BI system will be like and do.

I don’t believe that you can do BI by halves and remain somewhat sceptical about the claims of some of the newer BI products to do away with the necessary hard work.

Recommended Business Intelligence sites

Recommended sites

I have had a short list of recommended Business Intelligence sites on my sidebar for a while. I have just added a page where you can provide me with details of a site that you would like me to add. I’ll review any submissions and decide whether or not to post a link, so please don’t expect an automated service. Also, I’m just as interested in small sites that are just starting out as in established ones with a large subscriber base; what counts is whether your ideas are interesting.

Measuring the benefits of Business Intelligence

This post is based on some comments that I made on an article by Dorothy Miller at IT Business Edge. The title of Dorothy’s piece is Measuring the Return on Investment for Business Intelligence.

Yes I am British!

In a nutshell, good BI is all about users taking better business decisions. There are other benefits, some of which I allude to in this article, but nothing outweighs the potentially massive impact of taking better decisions.

While you would think that this is a very positive scenario for BI projects, there is a fly in the ointment. BI doesn’t let users take better business decisions, only good BI does that. Which begs the question: how do you tell how good you BI is and whether it is providing the promised benefits? This difficult area is the subject of this short article.
Measuring BI payback directly

All of the various different ways of measuring return on investment, from the simplest to the most sophisticated, compare the amount of money you spend to the amount of money you gain. Establishing the first parameter for this equation should be relatively simple, determining the second one is anything but straightforward.

In its purest form, the observation that taking better decisions should result in increased profits is entirely sound. If a BI project is targeting areas at the core of a company’s business (and why would you be investing in BI if this is not the case) then there should be a positive impact on profitability. This might be through increased sales or other income (assuming that these are at a profitable price), through increased margins on sales, reduced expenses, or a combination of all three. Of course the ways that good BI can contribute to each of these areas are manifold, but each will come down essentially to one of these three things.

The problem is that many other externalities can impact each of these areas. Some industries are cyclical, the impact of competitors will vary over time, demand for products will be different at different times, the business mix will not stay constant, markets can appear and disappear very quickly sometimes. Also, often the impact of better business decisions will not be realised until some future date; this year’s decisions may impact next year’s profits, or a strategic investment decision may not bear fruit for many years. While good BI will help you to make sense of these trends, at the same time it is difficult to disentangle the impact of BI on results from them.

However, sometimes it is possible to discern the direct impact of BI and I will offer a few examples:

1. Improvements in what BI is measuring

Suppose an element of your BI system tracks the number of sales leads that an organisation has against the number of orders placed. Being able to slice and dice such numbers by territory, product, marketing campaign, channel, customer segment, customer and so on is valuable in managing the sales process. However the trending capabilities of BI will also allow you to compare before and after. If the BI tool is truly being effective, then it should be possible to show that proportion of orders to leads is greater after its implementation than before it. In fact there should be a steady upward curve post-implementation as the system beds in and also as users find more creative ways to employ the information it gathers. If this is not the case, then something is wrong.

2. What if analyses

Supposing that the BI system aims to better manage the portfolio of a business; i.e. the balance of products and/or services that it sells. Again the historical aspect of BI can help us to consider its impact. Consider the simplified example of an organisation that has just three products: A, B and C, with each making up a third of sales. A good BI system will be able to calculate profitability for each product. Inevitably, shifts in market conditions will cause these profitabilities to change over time. Our example company might react to an increase in the profitability of product A in such a way that its portfolio becomes A (50%), B (25%), C (25%) – the assumption here is that the information available in their BI system is the main catalyst for change. What is important again is that the BI system holds historical information. This should allow a what if calculation to be performed; in this case, what if the mix of products had been left at 33%/33%/33%? The difference between calculated profitability under this scenario and actual profitability is one measure of the impact of good BI.

[It is interesting to note that this approach can also be used to support the case for BI projects. One technique that I have successfully employed is to look at an organisation’s results over a five-year period, use the first three years to develop a simple hypothesis for decision making (e.g. we should sell a product if it meets a set of criteria and not otherwise) and then use the last two to validate this. This type of study says something like: if we had had better information in the past, we would have taken different decisions with the following – hopefully positive – results. To make this a little clearer, the study I carried out showed that greater availability of even very basic BI would have led to profit margins doubling. I will cover this approach in more detail in a forthcoming article.]

3. Measurable productivity increases

Aside from enabling better decisions to be taken, good BI can save people time and effort, thereby increasing productivity. It is possible to measure such things. By both general survey and a specific analysis of tasks carried out, I was able to demonstrate that pre-BI implementation it could take 5-7 days to assemble all the information needed for a key account review. Post-BI implementation, this became a matter of minutes. Care should be taken in scaling-up these figures to overall time saved, but such studies can give a good indication of increased productivity and, if accompanied by some conservative cost estimates, can be presented in terms of actual monetary impact.

4. Direct comparison of profitability

In some companies and markets it may be possible to directly measure changes in profitability relating to the use of BI. Alternatively, it may be possible to adjust results to remove the impact of some external issues (e.g. where there is publicly available information about overall trends in the market). Sometimes the relationship between the implementation of good BI and corporate results will be so striking, that there will be no argument about their correlation. This is a situation that I have experienced myself.
Measuring BI payback via proxies

Notwithstanding the above points, the most likely scenario is that it will be challenging to discern the precise impact of a BI project. Given this, instead of giving up, it is important to consider any available proxies. Some of these are discussed in this section, all rest upon the assumption that business people are rational and will only use systems that add value, making their work easier or improving decision-making.

Many of these proxies are self-evident, so I won’t offer too much commentary on them.

5. User adoption

How many people use the system and do more people want to use it? How do these numbers change over time? What is penetration like in different areas of the organisation?

6. Actual usage

Of course this relates to the previous point as well, but directly measuring usage and even using your BI tool to analyse how this changes over time is important. If there is a correlation between how much a part of the organisation uses the system and how good its results are, so much the better.

7. User retention

Of the people who are given access to the system and trained in its use, how many go on to become regular users? How does this change over time?

8. Demand for enhancements / extension to the system

If you have people wanting the system to do more, then they must be happy with how it is working in general terms.

9. Feedback from surveys

It always helps to get feedback on what you are doing. Excerpts from one such survey appear here.

10. Do business users mention the system in meetings?

When presenting figures to senior management, is the source quoted as a matter of course (hopefully to establish that the figures are reliable)?

This article has argued that establishing the benefits of BI can be difficult, but that it is by no means impossible. There are a range of techniques available to either directly or indirectly assess its impact. Of course there are probably other creative ways to do this that other organisations are employing and which I have not mentioned.

Business Intelligence practitioners should pay especial attention to this area, it is an opportunity to demonstrate that the yields from BI projects can be substantial. In fact, it is my opinion that in many industries no other type of IT project will have greater payback than BI. It should be a priority for those who tout the benefits of BI to show that there is real substance behind these claims. My experience suggests that it is definitely worth taking the time and effort to prove this convincingly.


“All that glisters is not gold” – some thoughts on dashboards

Fool's gold

Yesterday I was tweeting quotes from Poe and blogging lines attributed to Heraclitus. Today I’m moving on to Shakespeare. Kudos to anyone posting a comment pointing out the second quote that appears later in the text.

Dashboards are all the rage at present. The basic idea is that they provide a way to quickly see what is happening, without getting lost in a sea of numbers. There are lots of different technologies out there that can help with dashboards. These range from parts of the product suites of all the main BI vendors, through boutique products dedicated to the area, all the way to simply using Java to write your own.

A lot of effort needs to go into how a dashboard is presented. The information really does need to leap off the screen, it is important that it looks professional. People are used to seeing well-designed sites on the web and if your corporate dashboard looks like it is only one step removed from Excel charts, you may have a problem. While engaging a design firm to help craft a dashboard might be overkill, it helps to get some graphic design input. I have been lucky enough over the years to have had people on my teams with experience in this area. They have mostly been hobbyists, but they had enough flair and enough of an aesthetic taste to make a difference.

However, echoing my comments on BI tools in general, I think an attractive looking dashboard is really only the icing on the cake. The cake itself has two main other ingredients:

  1. The actual figures that it presents (and how well they have been chosen) and
  2. The Information Architecture that underpins them

I’ll now consider the importance of these two areas.
Choosing the KPIs

Filtering out the KPIs

The acronym KPI is bandied about with enormous vigour in the BI community. Sometimes what the ‘K’ stands for can get a bit lost in the cacophony. Stepping back from dashboards for a few minutes, I want to focus on the measures that you have in your general business intelligence applications such as analysis cubes. Things like: sales revenue, units sold, growth, head count, profit and so on.

[Note: If you don’t like BI buzzwords, please feel free to read “figures”, or “numbers” where ever you see “measures”. I may attempt to provide my own definitions of some of these terms in the future as the Wikipedia entries aren’t always that illuminating.]

When you have built a Data Mart for a particular subject area and are looking to develop one or more cubes based on this, you may well have a myriad of measures to select from. In some of the earliest prototype cubes that my teams built, we made the mistake of having too many measures. The same observation equally applied to the number of dimensions (things that you want to slice and dice the measures by, e.g. geography, line of business, product, customer etc.). Having too many measures and dimensions led to a cube that was cumbersome, difficult to navigate and where the business purpose was less that crystal clear. These are all cardinal sins, but the last is the worst as I have referred to elsewhere. The clear objective is to cut down on both the figures and the business attributes that you want to look at them by. We set a rule (which we did break a couple of times for specialist applications) of generally having no more than ten measures and ten dimensions in a cube and ideally having less.

Well this all sounds great, the problem – and the reason for this diversion away from dashboards – is which measures do you keep and which do you drop. Here there is no real alternative to lots of discussions with business partners, building multiple prototypes to test out different combinations and, ultimately, accepting that you might make some mis-steps in your first release and need to revisit the area after it has been “shaken down” by real business use. I won’t delve into this particular process any deeper now. Suffice it to say that choosing which measures to include in a cube it is both an area that is important to get right and one in which it is all to easy to make mistakes.

So, retuning to our main discussion, if picking measures at the level of an analysis cube is hard, just how hard is it to pick KPIs for a dashboard. I recall a conversation with the CEO of a large organisation in which he basically told me to just pick the six most important figure and put them on a dashboard (with the clear implication that sooner would be rather better than later). After I had explained that the view of the CEO in this area was of paramount importance and that his input on which figures to use would be very valuable, we began to talk about what should be in and what should be out. After a period of going round in circles, I at least managed to convey the fact that this was not a trivial decision.

What you want with the KPIs on a dashboard is that they are genuinely key and that you can actually tell something from graphing them. The exercise in determining which figures to use and how to present them was a lengthy one, but very worthwhile. You need to rigorously apply the “so what?” test – what action will people take based on the trends and indicators that are presented to them. In the end we went for simplicity, with a focus on growth.

There was a map showing how each country was doing against plan; colour-coded red, amber and green according to their results. There were graphs comparing revenue to budget by month and the cumulative position and there was a break-down by business unit. The only to elements of interaction were to filter for a region or country and a business unit or line of business. Any further analysis required pulling up an underlying cube (actually we integrated the cube with the dashboard so that context was maintained moving from one to the other – this was not so easy as the dashboard and cube tools, while from the same vendor, were on two different major release numbers).

There were many iterations of the dashboard, but the one we eventually went live with received general acclaim. I’m not sure what we could have done differently to shorten the process.
Where does the data come from?

A dashboard without an underlying Information Architecture
A dashboard without an underlying Information Architecture

The same range of dashboard tools that I mention in the introduction are of course mostly capable of sourcing their data from pretty much anywhere. If the goal is to build a dashboard, then maybe it is tempting to do this as quickly as possible, based on whatever data sources are to hand (as in the diagram above). This is probably the quickest way to produce a dashboard, but it is unlikely to produce something that is used much, tells people anything useful, or adds any value. Why do I say this?

Well the problem with this approach is that all you are doing is reflecting what is likely to be a somewhat fragmented (and maybe even chaotic) set of information tools. Out of your sources, is there a unique place to go to get a definitive value for measure A? Do the various different sources hold data in the same way and calculate values using the same formulae? Do sources overlap (either duplicating data, or function), if so, which ones do you use? Do different sources get refreshed with the same frequency and do they treat currency the same way? Are customers and products defined consistently everywhere?

A dashboad underpinned by a proper Information Architecture
A dashboad underpinned by a proper Information Architecture

Leaving issues like these unresolved is a sure way to perpetuate a poor state of information. They are best addressed by establishing a wider information architecture (a simplified diagram of which appears above). I am not going to go into all of the benefits of such an approach, if readers would like more information, then please browse through the rest of this blog and the links to other resources that it contains (maybe this post would be a good place to start). What I will state is that a dashboard will only add value if it is part of an overall consistent approach to information, something that best practice indicates requires an Information Architecture. Anything else is simply going to be a pretty picture, signifying nothing.

So my advice to those seeking to build their first dashboard has three parts. First of all, keep it simple and identify a small group of measures and dimensions, which are highly pertinent to the core of the business and susceptible to graphical presentation. Second, dashboards are not a short-cut to management information Nirvana, they only really work when they are the final layer in a proper approach to information that spans all areas of the organisation. Finally, and partly driven by the first two observations, if you are in charge of building a dashboard, make sure that the plans you draw up reflect the complexity of the task and that you manage expectations accordingly.

The confluence of BI and change management

y =x^3 + 2x^2 - x + 1

The tag-line of this blog brings together business intelligence and cultural transformation. While one driver for this is that I have led BI projects that had explicit goals of cultural transformation, I think that there is a deeper connection to be explored here.

In other articles (notably “Can You Really Manage What You Measure?” by Neil Raden and Actionable Information), I discuss my experience that BI only adds value if:

  1. The information it provides answers pertinent business questions, and
  2. The answers to these questions lead to people taking action.

This means that any successful BI implementation has to consider such messy and difficult things as changing how people behave. This is where the link with change management arises.

Now of course you can argue that change management is an indispensible discipline for any business project (my strong opinion is that any IT project is a type of business project) and this is clearly true. However the parameters within which a new transaction processing system has to operate are different. Here if a person does not use the system, then work does not get done. Either it is impossible to carry out your job without the system (maybe only the system generates the necessary documentation), or not using the system to record transactions is a breech in compliance (keeping paper copies in your drawer).

BI systems are not like this. People chose to use them because they judge that they either make their business life easier, or they help to improve their decision-making (hopefully both). If someone doesn’t want to use a BI system, then they won’t and can probably get on with other parts of their job. The reason that change management is even more important in BI projects is that the element of compliance (or even coercion) is absent. If you want people to use the system and behave differently as a result, then you need to think about how best to influence them in these directions.

I have written elsewhere about the importance of marketing, education and follow-up in these areas. It also is important to explicitly recognise that a BI practitioner needs to be fully engaged in change management if they are to be successful.

A final thought also worth considering is that, as the BI industry matures and focus turns more to making it work in a business context than the latest flashy dashboard technology, it is likely that one of the things that will differentiate the best users of BI is how well they manage the necessary and desirable change that it drives.

πυρὸς θάνατος ἀέρι γένεσις, καὶ ἀέρος θάνατος ὕδατι γένεσις