general

Some reasons why IT projects fail

Peter James Thomas business, change management, it business alignment, project management, technology , , , , , , ,
© Alex Messenger - http://www.alexmessenger.co.uk/
© Alex Messenger - http://www.alexmessenger.co.uk/

Having yesterday been somewhat disparaging about the efforts of others to delineate the reasons for BI projects failing, I realised that I had recently put together just such a list myself. By way of context, this was in response to being asked for some feedback in a subject area where I had little expertise and experience. Instead of bailing out of answering, I put together a more general response, a lightly edited and mildly expanded version of which appears below.

Please note that there is no claim on my part that this list is exhaustive; in common with all humans, us IT types can be very creative in finding new ways to fail, I am sure there are some out there that I have not come across yet.
 

  1. The objectives of the project not being clear. By this I mean the business objectives. There are two layers of problems, the actual business issues may not be understood well enough to form an effective response and, if the business knows what it needs to do in general terms, IT may not fully appreciate this for a number of reasons (mostly down to lack of communication) or may be unable to translate this into a suitable programme of work (possibly because of a lack of knowledge of how the business operates). Where IT is not part of the senior management team, or sees itself as a department apart, this issue is more likely to occur.
  2. Strategy formation being skipped. If you don’t understand what a project is meant to be about, it is difficult (to say the least) to form a strategy. However, if the test in point 1. is passed, then it may be tempting (or there may be pressure applied) to get to the end game as soon as possible without either forming a strategy for the project, or fitting this into both over-arching business and IT strategies (which one fervently hopes are complementary). As I know all too well, the strategy formation step can be tough one and people may sometimes be keen to skip it. The current economic climate may lead to this happening more frequently and my opinion is that this will be storing up trouble for the future.
  3. Fragmented systems’ landscapes. Related to the above, it is often very difficult to make progress when there is a patchwork of different systems and approaches throughout an organisation and little desire to address this short-coming. Often some sort of revolution (albeit sometimes a quiet one sustained over many years) is required to move forward. Sometimes this requires some crisis, internal or external, as virtually every organisation is inherently conservative; no matter what their marketing spiel may claim to the contrary.
  4. Lack of Change Management. Projects often also have an organisational change aspect (what else are they for?) and the problems here are: a) people do not like change and resist it; and b) many otherwise able managers are not experienced in change – indeed we tend to educate most managers to be efficient in a steady-state environment. Even when this aspect is recognised, it is often underestimated and work does not start until too late in the game.
  5. People. Aside from these, the main other problem is people. Projects, even small ones, are difficult and not everyone is up to running them. Simple errors in execution can derail projects that otherwise tick all of the boxes.

 
Of course any passing Gartner analyst is more than welcome to rip this to shreds if they see fit.
 

“Gartner sees a big discrepancy between BI expectations and realities” – Intelligent Enterprise

Peter James Thomas business, business intelligence, change management, education, it business alignment, management information, project management, technology , , , , , , , , , , , , ,

Doug Henschen
 
Doug Henschen at Intelligent Enterprise reports from the Gartner Business Intelligence Summit in Washington D.C., explaining that Gartner analysts John Van Decker and Kurt Schlegel cited five reasons why BI projects do not live up to expectations (article link here):
 

  1. No IT-Business Partnership – “We have to get away from thinking of it as a vendor-customer relationship,” said Schlegel.
  2. No Link to Corporate Strategy – BI team have to listen to the executives and develop metrics and measures that are aligned with their goals.
  3. No connection to the Process – “BI has been overtly disconnected for too long,” Schlegel proclaimed. It’s not enough to deliver the right information to the right users. You have to insert those insights into the processes and interfaces that business users live in every day.”
  4. No Governance or Too Much – It has to be just the right amount of governance. BI grew up departmentally, so it’s all too common to have many silos of BI. At the other extreme, some companies are so uptight about data standards that companies can’t get their data into the warehouse.
  5. No Skills – Business users often lack skills, chimed in Van Decker, citing the one area where the business side was said to be falling short. “Most companies have very sophisticated capabilities available that folks just aren’t taking advantage of,” he said, pointing to corporate performance management (CPM) software as a leading example.

 
I come close to the situation of words failing me when I read a list like this (though not close enough to prevent me penning an article of course). I appreciate that maybe a public seminar is not the easiest place to provide deep insights (I present a lot myself), but the commentary against each of the problem areas seems odd to me. I’m not sure that these are really the five reasons for BI continuing to disappoint in some organisations, while it continues to delight in others, but here are my thoughts on each headline.
 

  1. No IT-Business Partnership – We have to stop talking about IT and Business as if they were two parties trying to negotiate a cease-fire. IT is a business department, it carries out business projects. It would be ridiculous to say that there needs to be a Sales-Business Partnership, it should be equally so with IT. I expand on this theme in the very first article on this blog.
  2. No Link to Corporate Strategy – There should not be a link to Corporate Strategy, BI does not exist as a separate entity that requires linkage. Instead BI work should be an expression of Corporate Strategy (as should any IT project), what else is it an expression of? This is not about listening to executives (though that is important) it is about IT being part of the senior management team of an organisation and not some semi-detached entity, focussed only on the beauty of its own navel. I give some indication of how to go about ensuring that this is the case in two articles, one focussed on a European environment and one spanning four continents.
  3. No connection to the Process – I agree that embedding good BI in a coporoate culture requires effort (indeed I have written a series of articles on the subject, starting with this one), however I struggle to see how any BI team could fail to realise this and pay the area due attention. If this is truly a reason for failure, then it is because of a lack of basic project and change management skills in BI teams.
  4. No Governance or Too Much – I’m not sure that achieving the Goldilocks approach to Governance is the issue here. BI’s impact on an organisation is directly proportional to how pervasive it is. This means that silos of BI reduce value and the walls need to be knocked in. Does this have to be all in one go? of course not, there are always benefits in a balance between incremental progress and paradigm shifts. Any organisation who has gatekeepers who refuse access to the warehouse because of and overly rigid approach to data standards is going to have problems. Equally where systems are developed with little thought to the information that they provide and data is simply thrown over the wall to the BI team, this is going to destroy value. As ever, there needs to be a sensible balance struck, one that yields the greatest business benefit for the lowest cost.
  5. No Skills – “Business users often lack skills”, this seems both incredibly patronising (are only IT people smart enough to get it?) and also a poor excuse for BI teams not paying enough attention to education (see point 3. above). If business people truly lack the skills to use good BI, then they are probably unfit to be in business as the tools are pretty intuitive (if not over-engineered by an approach that is too technology-focussed). More likely, training has been poor, or the BI deliveries have failed to be tailored to answering questions that the business wants to ask.

 
In summary, I don’t have five reasons for BI failing to live up to expectations, I have one. I firmly believe that BI done well is both the easiest of IT systems to sell to people and has one of the highest paybacks of any IT initiative. BI done badly (at the design, development, implementation or follow-up stages) will fail.

The issue is basically a simple one: just how good is your BI team? If a BI implementation fails to deliver significant business value, then instead of looking for scape-goats, the BI team should purchase a mirror and start using it.
 

 
Continue reading about this area in: Some reasons why IT projects fail
 
 
Doug Henschen joined Intelligent Enterprise as Editor in 2004 and was named Editor-in-Chief in January 2007. He has specialized in covering the intersection of business intelligence, performance management, business process management and rules management technologies within enterprise applications and architectures. He previously served as Editor-in-Chief of Transform Magazine, which covered content management and business process management challenges.

I comment on another Intelligent Enterprise article, this one by Cindi Howson, here.
 

Keynote Articles

Peter James Thomas site update

Keynote Articles

A new section has today appeared at on the menu of this site, Keynote Articles. Rather than try to describe this in a novel way, I will simply quote part of the introduction that appears on the page.

There are many strengths of the blog format, however one weakness is its focus on the recent at the expense of the relevant. This section of the site seeks to remedy this problem by providing a grouped list of what I am going to call Keynote Articles.

There are a lot of different articles on this blog. Some are to do with new functionality that is available; some are brief notes about what I have been up to, or am planning to do; some link to news elsewhere on the Internet, most often with some commentary from me. Keynote Articles are what is left. That is original pieces written by me based on my own experience and expertise. These types of articles form the meat of this site.

I have tried to group these Keynote Articles into general areas, with some pieces appearing in more than one place. Inside each area, the entries appear in chronological order, i.e. the reverse of the main blog page.

This categorised list of major articles should provide another way of finding relevant content, supplementing the various tools and links that already appear in the right-hand sidebar.
 

Trends in Business Intelligence

Peter James Thomas business, business analytics, business intelligence, data warehousing, enterprise performance management, it business alignment, management information, technology , , , , ,

trends

This year, as in every year since the phrase “Business Intelligence” first came to prominence, there have been a rash of predictions about what will happen with the area in 2009. I have linked to and commented on some of these myself on this blog. This article is my take on the area. I should however explain that there is a twist. These are not the trends that I expect to see in 2009, just my thoughts on what ought to be trends in BI over the next few years. I cannot claim to have any prescience about whether they will come to pass, but I will be encouraged if they do.
 

  1. A more holistic approach to BI in organisations that have already invested
  2. Operational BI
  3. Consolidation of the number of BI platforms with organisations
  4. An increase in the prevalence of BI competency centres
  5. BI teams being jointly owned and managed by business divisions and IT
  6. Data from central warehouses being served to front-end applications
  7. BI data being used to automate some business decisions
  8. Further emphasis on predictive analytics
  9. BI having an increasing role in compliance and risk management
  10. Incorporation of external data into BI platforms
  11. Provision of BI to business partners and/or customers
  12. The most creative users of BI employing it to thrive, even in the current climate

 
1. A more holistic approach to BI in organisations that have already invested

Often BI solutions have started in a single area. Typically – given its direct link to better managing overall performance – this has been around analysing an organisation’s financial results, though implementations starting in the sales and even operational arenas are also not uncommon. BI systems tend to add more value as they bring in more information, particularly where a high-level phenomenon – e.g. a decrease in expenditure – can be attributed to lower level ones – e.g. greater repeat business (business acquisition being more expensive than repeat business in most industries), plus enhanced productivity, plus reduced staff turnover (resulting in lower recruitment and training costs). To do this, BI needs to widen its scope to take on new data sources and combine them in creative ways. Where BI platforms have already added value, there will be pressure from users to expand them to deliver even more utility and provide more sophisticated insights. This in turn will require BI practitioners to take a more holistic view of the area and to develop roadmaps explaining how new data sources will be brought on stream and what business value will result from this.
 
 
2. Operational BI

I think that there is a particular argument for BI to make a greater contribution in the area of operational management, particularly tying this to overall performance. Often BI implementations have focussed on the more stately world of monthly (or weekly) results and steered clear of the daily or hourly demands of operational information. The BI projects that I have seen in the Operational sphere have been more focussed at producing weekly status reports or year-to-date trend analyses. Related to BI platforms expanding to new areas, I see increasing demand for information about the internal effectiveness of an organisation; quickly identifying bottlenecks or areas of inefficiency and tracking efforts to address these. Supporting this type of reporting will often require BI practitioners to rethink their architectures which are often set up to refresh on longer timeframes. This may in turn require the warehouse to support multiple environments with different periodicities.
 
 
3. Consolidation of the number of BI platforms with organisations

It is all too common for organisations to have separate BI implementations. Maybe the ERP system comes with some shrink-wrapped cubes, as does the CRM system. Perhaps Finance have invested in some budget analysis tools and different business units have started dabbling in predictive modelling (see 8. below). Maybe certain power users or numerate departments have their own, much cherished databases. Clearly this approach is sub-optimal. If it was ever acceptable to have such an ad hoc approach to the area, the current economic climate means that there is no longer room for a multitude of platforms, each supported by their own ETL, servers, IT teams and (hopefully) training programmes. While transitional costs may make it impractical for some organisations to move to a single platform in the short term, the reductions in licensing, internal support, data centre and training costs that come from standardising BI tools are compelling. This is to say nothing about reducing the level of confusion in users faced with multiple reporting and analysis systems, each with their own terminology, vagaries of functionality and often un-reconciled to each other. Arguments about certain BI tools being best of breed for particular tasks were never wholly convincing, with the breadth of functionality offered by all of the major players today, any of them will be at least good enough for all but the most exacting of applications.
 
 
4. An increase in the prevalence of BI competency centres

If this is partly as a result of the previous three trends, it also has some drivers all of its own. Even in multinational organisations with highly devolved structures and local accountability, most of the management information that is needed to run businesses will be relatively consistent from country to country and business unit to business unit. Perhaps the sources will vary, but the business transactions that they support are surprisingly consistent. Many organisations are waking up to this fact and pulling BI provision into the centre. There are obvious benefits to this in terms of cost savings, not reinventing the wheel, increasing the consistency of reporting and enabling corporate roll-ups. However, my own feeling is that the best organisations will supplement a strong central resource with a (probably much smaller) virtual component located close to business needs who can better respond to these in a timely manner, but within an overall information architecture. This hybrid approach offers the best of both worlds.
 
 
5. BI teams being jointly owned and managed by business divisions and IT

It is an oft-repeated aphorism that all IT projects are business projects. While clearly true in theory, there are enough counterexamples to suggest that practise is rather different. However, BI projects have often stayed much closer to this maxim than other IT efforts. This is because BI systems serve no purpose unless they are closely entwined with business goals. Natural selection should weed out any non-business-focussed BI projects eventually; even in the sleepiest of organisations. It is likely that this close relationship will deepen. Whilst many aspects of BI are firmly in the IT arena (managing the regular refresh of multiple terabytes of data effectively clearly being one), I see a joint stewardship of the area developing. By this I mean something deeper than business oversight or steering committees. It is also different to business BI liaison managers being appointed – these roles often have the unintended consequence of isolating IT from its customers. What I see emerging in more enlightened organisations is true co-ownership of BI with business and IT management closely collaborating to run the area.
 
 
6. Data from central warehouses being served to front-end applications

Of course it is not uncommon for transaction processing systems to have buttons or links that call up a given report. What I am talking about here is a closer coupling, one that does not require users to leave their current system and where information from the warehouse is presented in a seamless manner to users. While such information will have all the BI benefits of being reconciled, consistent and accurate, its provenance will increasingly become invisible to the user (who shouldn’t really have to worry where figures come from so long as they are accurate). This is an area in which web-services have some real potential to leverage the investments that organisations have made in warehouses.
 
 
7. BI data being used to automate some business decisions

Stepping a little further along the path from the previous point, even before users of front-end systems have a need to review information about a transaction, it may well be that the information itself has been what determines whether a human is involved or not. Already some organisations perform triage on business transactions. Ones that meet all of a number of requirements get processed automatically. Ones that meet some of them, but not all, may get routed to junior staff, whose role is to determine whether they require further review or can proceed. Finally, those with a major variance to requirements may get sent straight to more senior staff. This approach means that a larger volume of business can be handled and that expensive senior resource is only applied where it is necessary. Of course a prerequisite to this type of approach is having reliable information on which to perform the triage.
 
 
8. Further emphasis on predictive analytics

While the best BI implementations encourage users to extrapolate from past figures to estimate future ones, the degree to which the analytical skills of the user plays a part varies from implementation to implementation. Here by analytics I mean the use of larger data sets to identify trends or exceptions, mostly at a portfolio level. Again, having invested in developing data warehouses, it makes sense to utilise the many and sophisticated tools that are available to carry out advanced statistical analysis on these; the aim being to deduce trends that would be beyond even the most skilled of human analysts. A by-product of successful BI implementations is that they often free the more numerate of people in organisations from the burden of repetitive number crunching and allow them to focus on more added-value work such as this.
 
 
9. BI having an increasing role in compliance and risk management

Sometimes BI projects may have had their genesis in these areas. However, even when this is not the case a pleasing result of having consolidated much of the organisation’s data in one place is that this then forms a valuable resource for compliance and risk management. Indeed, taking an external perspective, regulators tend to view the existence of an enterprise data warehouse as a sign that an organisation takes managing its risks seriously. Although business managers and risk managers may have slightly different (though hopefully complementary) perspectives and want to answer different types of questions, the data that they need to do this is not that dissimilar. Producing compliance suites from a well-designed warehouse is probably not one of the more taxing BI problems. Again expansion in this area is a further example of point 1. in action.
 
 
10. Incorporation of external data into BI platforms

I have spoken above about the remit of BI systems expanding internally and becoming more consistent geographically. A further trend in expansion is to meld internal information with that from external providers. The type of external information would range from industry to industry, but might include market data, information about specific companies or individuals (e.g. credit scores, where the use of these is admissible). For example, in insurance, where I have spent the last twelve years of my career, incorporating information from externally produced flood, or windstorm models is often a priority.
 
 
11. Provision of BI to business partners and/or customers

Sometimes one objective of BI implementations is to better understand the relationships with business partners and customers. I have seen this develop to the degree where output from BI systems is mailed to such counterparties. A logical extension of this is to allow such organisations direct access to “their information”. Of course as with any e-commerce initiative, there would have be to strict controls on what is viewed and who can view it, but these are problems that are regularly addressed in other areas of IT and the tools to do this are readily available.
 
 
12. The most creative users of BI employing it to thrive, even in the current climate

There have been many articles (including ones written by me) which have spoken about good BI being a great defence in times of economic stress. I would go beyond this and state that the real BI pioneers will take advantage of these capabilities to capture markets from their less well-informed competitors and to steer a course away from areas of business that may bring other less-foresighted organisations down. I look forward to seeing case studies bearing this out appear over the next few years.
 

Back safe and sound

Peter James Thomas site update
The horseshoe of Cwm Idwal with a dusting of March snow
The horseshoe of Cwm Idwal with a dusting of March snow

Further to my last post, I am back safe and well (if rather wet) from my climbing trip to snowy North Wales. Thank you to those who either wished me well or expressed concern for my safety. Refreshed by some physical activity, I’ll now return to more cerebral matters.

For anyone interested, a mini-report on our trip appears on my partner’s blog here.
 

There now follows a brief intermission…

Peter James Thomas site update

I will be off in North Wales for the rest of the week. The plan is to do some rock climbing, specifically bouldering. However the forecast is not exactly positive, so I may spend rather more time looking at the rain.

Normal service will be resumed next week.
 

Adding my headlines to your site

Peter James Thomas site update

Peter Thomas - BI Blog

↑ Grab this Headline Animator

 
Care of Feedburner I have just created a page where you can grab the animated widget above and easily add it to your site or blog. The widget cycles through the headlines of my five most recent articles. A link to this also appears in the RSS box at the top of the right hand side bar.
 

“Businesses Are Still Crazy for BI After All These Years” – CIO.com

Peter James Thomas bi and the economic crisis, business, business intelligence, enterprise performance management, it business alignment, management information, technology , , , , , , , , , ,

Thomas Wailgum at CIO.com

Thomas Wailgum has written an article at CIO.com in which he talks about continuing demand for BI, but adds that this, in turn, suggests that in many organisations BI has yet to deliver on its promise. As Thomas puts it:

“I see pent-up enterprise-wide frustration, aimed squarely at IT and CIOs for failing to give the business what it needs and deserves”

He sees the fundamental problem as being fragmented systems and stand-alone BI applications. This sounds like challenges that I have faced before. I agree that BI only realises it potential when a more strategic and wide-ranging approach is taken. Something I refer to in many places on this blog, but possibly most directly in Holistic vs Incremental approaches to BI.

My basic point is that while it is sensible to take a pragmatic, incremental approach to implementing BI (collecting successes as you go and building momentum), this needs to be within the framework of a more encompassing vision for what the eventual BI system will be like and do.

I don’t believe that you can do BI by halves and remain somewhat sceptical about the claims of some of the newer BI products to do away with the necessary hard work.
 

Recommended Business Intelligence sites

Peter James Thomas business, business intelligence, site update, technology , ,

Recommended sites

I have had a short list of recommended Business Intelligence sites on my sidebar for a while. I have just added a page where you can provide me with details of a site that you would like me to add. I’ll review any submissions and decide whether or not to post a link, so please don’t expect an automated service. Also, I’m just as interested in small sites that are just starting out as in established ones with a large subscriber base; what counts is whether your ideas are interesting.
 

Measuring the benefits of Business Intelligence

Peter James Thomas business, business intelligence, enterprise performance management, management information, technology , , ,

This post is based on some comments that I made on an article by Dorothy Miller at IT Business Edge. The title of Dorothy’s piece is Measuring the Return on Investment for Business Intelligence.
 
 
Introduction

Yes I am British!

In a nutshell, good BI is all about users taking better business decisions. There are other benefits, some of which I allude to in this article, but nothing outweighs the potentially massive impact of taking better decisions.

While you would think that this is a very positive scenario for BI projects, there is a fly in the ointment. BI doesn’t let users take better business decisions, only good BI does that. Which begs the question: how do you tell how good you BI is and whether it is providing the promised benefits? This difficult area is the subject of this short article.
 
 
Measuring BI payback directly

All of the various different ways of measuring return on investment, from the simplest to the most sophisticated, compare the amount of money you spend to the amount of money you gain. Establishing the first parameter for this equation should be relatively simple, determining the second one is anything but straightforward.

In its purest form, the observation that taking better decisions should result in increased profits is entirely sound. If a BI project is targeting areas at the core of a company’s business (and why would you be investing in BI if this is not the case) then there should be a positive impact on profitability. This might be through increased sales or other income (assuming that these are at a profitable price), through increased margins on sales, reduced expenses, or a combination of all three. Of course the ways that good BI can contribute to each of these areas are manifold, but each will come down essentially to one of these three things.

The problem is that many other externalities can impact each of these areas. Some industries are cyclical, the impact of competitors will vary over time, demand for products will be different at different times, the business mix will not stay constant, markets can appear and disappear very quickly sometimes. Also, often the impact of better business decisions will not be realised until some future date; this year’s decisions may impact next year’s profits, or a strategic investment decision may not bear fruit for many years. While good BI will help you to make sense of these trends, at the same time it is difficult to disentangle the impact of BI on results from them.

However, sometimes it is possible to discern the direct impact of BI and I will offer a few examples:

1. Improvements in what BI is measuring

Suppose an element of your BI system tracks the number of sales leads that an organisation has against the number of orders placed. Being able to slice and dice such numbers by territory, product, marketing campaign, channel, customer segment, customer and so on is valuable in managing the sales process. However the trending capabilities of BI will also allow you to compare before and after. If the BI tool is truly being effective, then it should be possible to show that proportion of orders to leads is greater after its implementation than before it. In fact there should be a steady upward curve post-implementation as the system beds in and also as users find more creative ways to employ the information it gathers. If this is not the case, then something is wrong.

2. What if analyses

Supposing that the BI system aims to better manage the portfolio of a business; i.e. the balance of products and/or services that it sells. Again the historical aspect of BI can help us to consider its impact. Consider the simplified example of an organisation that has just three products: A, B and C, with each making up a third of sales. A good BI system will be able to calculate profitability for each product. Inevitably, shifts in market conditions will cause these profitabilities to change over time. Our example company might react to an increase in the profitability of product A in such a way that its portfolio becomes A (50%), B (25%), C (25%) – the assumption here is that the information available in their BI system is the main catalyst for change. What is important again is that the BI system holds historical information. This should allow a what if calculation to be performed; in this case, what if the mix of products had been left at 33%/33%/33%? The difference between calculated profitability under this scenario and actual profitability is one measure of the impact of good BI.

[It is interesting to note that this approach can also be used to support the case for BI projects. One technique that I have successfully employed is to look at an organisation’s results over a five-year period, use the first three years to develop a simple hypothesis for decision making (e.g. we should sell a product if it meets a set of criteria and not otherwise) and then use the last two to validate this. This type of study says something like: if we had had better information in the past, we would have taken different decisions with the following – hopefully positive – results. To make this a little clearer, the study I carried out showed that greater availability of even very basic BI would have led to profit margins doubling. I will cover this approach in more detail in a forthcoming article.]

3. Measurable productivity increases

Aside from enabling better decisions to be taken, good BI can save people time and effort, thereby increasing productivity. It is possible to measure such things. By both general survey and a specific analysis of tasks carried out, I was able to demonstrate that pre-BI implementation it could take 5-7 days to assemble all the information needed for a key account review. Post-BI implementation, this became a matter of minutes. Care should be taken in scaling-up these figures to overall time saved, but such studies can give a good indication of increased productivity and, if accompanied by some conservative cost estimates, can be presented in terms of actual monetary impact.

4. Direct comparison of profitability

In some companies and markets it may be possible to directly measure changes in profitability relating to the use of BI. Alternatively, it may be possible to adjust results to remove the impact of some external issues (e.g. where there is publicly available information about overall trends in the market). Sometimes the relationship between the implementation of good BI and corporate results will be so striking, that there will be no argument about their correlation. This is a situation that I have experienced myself.
 
 
Measuring BI payback via proxies

Notwithstanding the above points, the most likely scenario is that it will be challenging to discern the precise impact of a BI project. Given this, instead of giving up, it is important to consider any available proxies. Some of these are discussed in this section, all rest upon the assumption that business people are rational and will only use systems that add value, making their work easier or improving decision-making.

Many of these proxies are self-evident, so I won’t offer too much commentary on them.

5. User adoption

How many people use the system and do more people want to use it? How do these numbers change over time? What is penetration like in different areas of the organisation?

6. Actual usage

Of course this relates to the previous point as well, but directly measuring usage and even using your BI tool to analyse how this changes over time is important. If there is a correlation between how much a part of the organisation uses the system and how good its results are, so much the better.

7. User retention

Of the people who are given access to the system and trained in its use, how many go on to become regular users? How does this change over time?

8. Demand for enhancements / extension to the system

If you have people wanting the system to do more, then they must be happy with how it is working in general terms.

9. Feedback from surveys

It always helps to get feedback on what you are doing. Excerpts from one such survey appear here.

10. Do business users mention the system in meetings?

When presenting figures to senior management, is the source quoted as a matter of course (hopefully to establish that the figures are reliable)?
 
 
Summary

This article has argued that establishing the benefits of BI can be difficult, but that it is by no means impossible. There are a range of techniques available to either directly or indirectly assess its impact. Of course there are probably other creative ways to do this that other organisations are employing and which I have not mentioned.

Business Intelligence practitioners should pay especial attention to this area, it is an opportunity to demonstrate that the yields from BI projects can be substantial. In fact, it is my opinion that in many industries no other type of IT project will have greater payback than BI. It should be a priority for those who tout the benefits of BI to show that there is real substance behind these claims. My experience suggests that it is definitely worth taking the time and effort to prove this convincingly.