Measuring the benefits of Business Intelligence

Peter James Thomas business, business intelligence, enterprise performance management, management information, technology, , ,

This post is based on some comments that I made on an article by Dorothy Miller at IT Business Edge. The title of Dorothy’s piece is Measuring the Return on Investment for Business Intelligence.
 
 
Introduction

Yes I am British!

In a nutshell, good BI is all about users taking better business decisions. There are other benefits, some of which I allude to in this article, but nothing outweighs the potentially massive impact of taking better decisions.

While you would think that this is a very positive scenario for BI projects, there is a fly in the ointment. BI doesn’t let users take better business decisions, only good BI does that. Which begs the question: how do you tell how good you BI is and whether it is providing the promised benefits? This difficult area is the subject of this short article.
 
 
Measuring BI payback directly

All of the various different ways of measuring return on investment, from the simplest to the most sophisticated, compare the amount of money you spend to the amount of money you gain. Establishing the first parameter for this equation should be relatively simple, determining the second one is anything but straightforward.

In its purest form, the observation that taking better decisions should result in increased profits is entirely sound. If a BI project is targeting areas at the core of a company’s business (and why would you be investing in BI if this is not the case) then there should be a positive impact on profitability. This might be through increased sales or other income (assuming that these are at a profitable price), through increased margins on sales, reduced expenses, or a combination of all three. Of course the ways that good BI can contribute to each of these areas are manifold, but each will come down essentially to one of these three things.

The problem is that many other externalities can impact each of these areas. Some industries are cyclical, the impact of competitors will vary over time, demand for products will be different at different times, the business mix will not stay constant, markets can appear and disappear very quickly sometimes. Also, often the impact of better business decisions will not be realised until some future date; this year’s decisions may impact next year’s profits, or a strategic investment decision may not bear fruit for many years. While good BI will help you to make sense of these trends, at the same time it is difficult to disentangle the impact of BI on results from them.

However, sometimes it is possible to discern the direct impact of BI and I will offer a few examples:

1. Improvements in what BI is measuring

Suppose an element of your BI system tracks the number of sales leads that an organisation has against the number of orders placed. Being able to slice and dice such numbers by territory, product, marketing campaign, channel, customer segment, customer and so on is valuable in managing the sales process. However the trending capabilities of BI will also allow you to compare before and after. If the BI tool is truly being effective, then it should be possible to show that proportion of orders to leads is greater after its implementation than before it. In fact there should be a steady upward curve post-implementation as the system beds in and also as users find more creative ways to employ the information it gathers. If this is not the case, then something is wrong.

2. What if analyses

Supposing that the BI system aims to better manage the portfolio of a business; i.e. the balance of products and/or services that it sells. Again the historical aspect of BI can help us to consider its impact. Consider the simplified example of an organisation that has just three products: A, B and C, with each making up a third of sales. A good BI system will be able to calculate profitability for each product. Inevitably, shifts in market conditions will cause these profitabilities to change over time. Our example company might react to an increase in the profitability of product A in such a way that its portfolio becomes A (50%), B (25%), C (25%) – the assumption here is that the information available in their BI system is the main catalyst for change. What is important again is that the BI system holds historical information. This should allow a what if calculation to be performed; in this case, what if the mix of products had been left at 33%/33%/33%? The difference between calculated profitability under this scenario and actual profitability is one measure of the impact of good BI.

[It is interesting to note that this approach can also be used to support the case for BI projects. One technique that I have successfully employed is to look at an organisation’s results over a five-year period, use the first three years to develop a simple hypothesis for decision making (e.g. we should sell a product if it meets a set of criteria and not otherwise) and then use the last two to validate this. This type of study says something like: if we had had better information in the past, we would have taken different decisions with the following – hopefully positive – results. To make this a little clearer, the study I carried out showed that greater availability of even very basic BI would have led to profit margins doubling. I will cover this approach in more detail in a forthcoming article.]

3. Measurable productivity increases

Aside from enabling better decisions to be taken, good BI can save people time and effort, thereby increasing productivity. It is possible to measure such things. By both general survey and a specific analysis of tasks carried out, I was able to demonstrate that pre-BI implementation it could take 5-7 days to assemble all the information needed for a key account review. Post-BI implementation, this became a matter of minutes. Care should be taken in scaling-up these figures to overall time saved, but such studies can give a good indication of increased productivity and, if accompanied by some conservative cost estimates, can be presented in terms of actual monetary impact.

4. Direct comparison of profitability

In some companies and markets it may be possible to directly measure changes in profitability relating to the use of BI. Alternatively, it may be possible to adjust results to remove the impact of some external issues (e.g. where there is publicly available information about overall trends in the market). Sometimes the relationship between the implementation of good BI and corporate results will be so striking, that there will be no argument about their correlation. This is a situation that I have experienced myself.
 
 
Measuring BI payback via proxies

Notwithstanding the above points, the most likely scenario is that it will be challenging to discern the precise impact of a BI project. Given this, instead of giving up, it is important to consider any available proxies. Some of these are discussed in this section, all rest upon the assumption that business people are rational and will only use systems that add value, making their work easier or improving decision-making.

Many of these proxies are self-evident, so I won’t offer too much commentary on them.

5. User adoption

How many people use the system and do more people want to use it? How do these numbers change over time? What is penetration like in different areas of the organisation?

6. Actual usage

Of course this relates to the previous point as well, but directly measuring usage and even using your BI tool to analyse how this changes over time is important. If there is a correlation between how much a part of the organisation uses the system and how good its results are, so much the better.

7. User retention

Of the people who are given access to the system and trained in its use, how many go on to become regular users? How does this change over time?

8. Demand for enhancements / extension to the system

If you have people wanting the system to do more, then they must be happy with how it is working in general terms.

9. Feedback from surveys

It always helps to get feedback on what you are doing. Excerpts from one such survey appear here.

10. Do business users mention the system in meetings?

When presenting figures to senior management, is the source quoted as a matter of course (hopefully to establish that the figures are reliable)?
 
 
Summary

This article has argued that establishing the benefits of BI can be difficult, but that it is by no means impossible. There are a range of techniques available to either directly or indirectly assess its impact. Of course there are probably other creative ways to do this that other organisations are employing and which I have not mentioned.

Business Intelligence practitioners should pay especial attention to this area, it is an opportunity to demonstrate that the yields from BI projects can be substantial. In fact, it is my opinion that in many industries no other type of IT project will have greater payback than BI. It should be a priority for those who tout the benefits of BI to show that there is real substance behind these claims. My experience suggests that it is definitely worth taking the time and effort to prove this convincingly.
 

 

33 thoughts on “Measuring the benefits of Business Intelligence”

  2. Peter:
    Posted this comment at LinkedIn, repeated here as well.

    Terrific post, beginning to think we should all take a deep breath and maybe begin to frame up or organize discussions like this around a theme our business partners understand – e.g. translating your findings for example across the five (and only five) essentials in business that matter; cash flow, profitability, velocity, growth, and customer intimacy. This theme can be extended to the non-profit or government world by simply substituting the word outcome for profitability. I guess what I’m driving at is a way to simplify our thinking and share what we are really doing in terms (lexicon) most of our customers and management can understand.

    For more on this see an early post at http://pragmaticarchitect.wordpress.com/2009/02/11/where-is-enterprise-architecture-when-we-really-need-it/. And this related topic (a little off BI, but gives you an idea of what my EA peers are struggling with now – essentially the same issue in a different but related discipline) at http://pragmaticarchitect.wordpress.com/2009/02/13/the-business-case-for-enterprise-architecture/

    Keep up the good work always enjoy reading your writing.

    -jdp (Parnitzke)

    Reply

  7. Peter, great article.

    I take a bit of a different look on the value of BI. I work with only with software vendors looking to embed BI into their solutions. I have posted 3 parts to a 4 part series out at http://perf-mgt.blogspot.com/ entitled “Beyond the Checkbox”.

    Software vendors typically think of BI as reporting. Like you mention above and in several things I see you’ve written, BI should be more about effectiveness of processes, not just dumping out counts and amounts. The true value of BI to a software vendor should be in the ability to prove they are delivering on the promise they made to their customers.

    Chris Tyler

    Reply

  12. Examples of Business Blogs…

    Here are some business blogs that catched my attention in the recent times. They are not perfect, however, they falls into a category of good blogging. Please, look at them carefully and see what you can use for your own blog. Examples of Business Blog…

    Reply

  23. That BI is about “users making better decisions” is a common misnomer and project mistake I regularly see. Ideally, BI is directed at optimizing *business processes* not informing people. Considering only the latter allows organizations to fall into the trap of delivering only interesting unused analysis rather than important actionable analytics. Similarly, considering only users (humans) as the recipients of analytics ignores the larger opportunity for optimizing operational processes.

    Reply

    • Hi Doug,

      Thanks for taking the time to comment.

      Don’t want to get into analysis / analytics semantics – I spent long enough on that area in the series of posts starting with Business Analytics vs Business Intelligence to last a lifetime.

      I don’t view the phrase that you pick on as a misnomer (for a start it is not a noun-phrase, but I digress), however maybe this is again semantics as you later refer to the outcome being “actionable”, which I 100% endorse. Indeed my current presentation deck includes the statements:

      Business Intelligence only adds value if:

      1. It answers pertinent business questions
      2. These answers lead people to take action

      So I live in hope that maybe we are not as far apart in our opinions as your comments suggest.

      A lot of my BI work has been in the field of optimising processes, but that hasn’t excluded helping the people responsible for them to take decisions. Not 100% sure that I am getting the essence of what you are saying.

      All the best

      Peter

      Reply

  28. I think the what if analyses are the most important benefit of BI. Being able to project what’s next for your business is invaluable. You want to create a plan and stick to it, not just hop from idea to idea.

    Reply

Leave a Reply to James ParnitzkeCancel reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.