This post is based on some comments that I made on an article by Dorothy Miller at IT Business Edge. The title of Dorothy’s piece is Measuring the Return on Investment for Business Intelligence.
In a nutshell, good BI is all about users taking better business decisions. There are other benefits, some of which I allude to in this article, but nothing outweighs the potentially massive impact of taking better decisions.
While you would think that this is a very positive scenario for BI projects, there is a fly in the ointment. BI doesn’t let users take better business decisions, only good BI does that. Which begs the question: how do you tell how good you BI is and whether it is providing the promised benefits? This difficult area is the subject of this short article.
Measuring BI payback directly
All of the various different ways of measuring return on investment, from the simplest to the most sophisticated, compare the amount of money you spend to the amount of money you gain. Establishing the first parameter for this equation should be relatively simple, determining the second one is anything but straightforward.
In its purest form, the observation that taking better decisions should result in increased profits is entirely sound. If a BI project is targeting areas at the core of a company’s business (and why would you be investing in BI if this is not the case) then there should be a positive impact on profitability. This might be through increased sales or other income (assuming that these are at a profitable price), through increased margins on sales, reduced expenses, or a combination of all three. Of course the ways that good BI can contribute to each of these areas are manifold, but each will come down essentially to one of these three things.
The problem is that many other externalities can impact each of these areas. Some industries are cyclical, the impact of competitors will vary over time, demand for products will be different at different times, the business mix will not stay constant, markets can appear and disappear very quickly sometimes. Also, often the impact of better business decisions will not be realised until some future date; this year’s decisions may impact next year’s profits, or a strategic investment decision may not bear fruit for many years. While good BI will help you to make sense of these trends, at the same time it is difficult to disentangle the impact of BI on results from them.
However, sometimes it is possible to discern the direct impact of BI and I will offer a few examples:
1. Improvements in what BI is measuring
Suppose an element of your BI system tracks the number of sales leads that an organisation has against the number of orders placed. Being able to slice and dice such numbers by territory, product, marketing campaign, channel, customer segment, customer and so on is valuable in managing the sales process. However the trending capabilities of BI will also allow you to compare before and after. If the BI tool is truly being effective, then it should be possible to show that proportion of orders to leads is greater after its implementation than before it. In fact there should be a steady upward curve post-implementation as the system beds in and also as users find more creative ways to employ the information it gathers. If this is not the case, then something is wrong.
2. What if analyses
Supposing that the BI system aims to better manage the portfolio of a business; i.e. the balance of products and/or services that it sells. Again the historical aspect of BI can help us to consider its impact. Consider the simplified example of an organisation that has just three products: A, B and C, with each making up a third of sales. A good BI system will be able to calculate profitability for each product. Inevitably, shifts in market conditions will cause these profitabilities to change over time. Our example company might react to an increase in the profitability of product A in such a way that its portfolio becomes A (50%), B (25%), C (25%) – the assumption here is that the information available in their BI system is the main catalyst for change. What is important again is that the BI system holds historical information. This should allow a what if calculation to be performed; in this case, what if the mix of products had been left at 33%/33%/33%? The difference between calculated profitability under this scenario and actual profitability is one measure of the impact of good BI.
[It is interesting to note that this approach can also be used to support the case for BI projects. One technique that I have successfully employed is to look at an organisation’s results over a five-year period, use the first three years to develop a simple hypothesis for decision making (e.g. we should sell a product if it meets a set of criteria and not otherwise) and then use the last two to validate this. This type of study says something like: if we had had better information in the past, we would have taken different decisions with the following – hopefully positive – results. To make this a little clearer, the study I carried out showed that greater availability of even very basic BI would have led to profit margins doubling. I will cover this approach in more detail in a forthcoming article.]
3. Measurable productivity increases
Aside from enabling better decisions to be taken, good BI can save people time and effort, thereby increasing productivity. It is possible to measure such things. By both general survey and a specific analysis of tasks carried out, I was able to demonstrate that pre-BI implementation it could take 5-7 days to assemble all the information needed for a key account review. Post-BI implementation, this became a matter of minutes. Care should be taken in scaling-up these figures to overall time saved, but such studies can give a good indication of increased productivity and, if accompanied by some conservative cost estimates, can be presented in terms of actual monetary impact.
4. Direct comparison of profitability
In some companies and markets it may be possible to directly measure changes in profitability relating to the use of BI. Alternatively, it may be possible to adjust results to remove the impact of some external issues (e.g. where there is publicly available information about overall trends in the market). Sometimes the relationship between the implementation of good BI and corporate results will be so striking, that there will be no argument about their correlation. This is a situation that I have experienced myself.
Measuring BI payback via proxies
Notwithstanding the above points, the most likely scenario is that it will be challenging to discern the precise impact of a BI project. Given this, instead of giving up, it is important to consider any available proxies. Some of these are discussed in this section, all rest upon the assumption that business people are rational and will only use systems that add value, making their work easier or improving decision-making.
Many of these proxies are self-evident, so I won’t offer too much commentary on them.
5. User adoption
How many people use the system and do more people want to use it? How do these numbers change over time? What is penetration like in different areas of the organisation?
6. Actual usage
Of course this relates to the previous point as well, but directly measuring usage and even using your BI tool to analyse how this changes over time is important. If there is a correlation between how much a part of the organisation uses the system and how good its results are, so much the better.
7. User retention
Of the people who are given access to the system and trained in its use, how many go on to become regular users? How does this change over time?
8. Demand for enhancements / extension to the system
If you have people wanting the system to do more, then they must be happy with how it is working in general terms.
9. Feedback from surveys
It always helps to get feedback on what you are doing. Excerpts from one such survey appear here.
10. Do business users mention the system in meetings?
When presenting figures to senior management, is the source quoted as a matter of course (hopefully to establish that the figures are reliable)?
This article has argued that establishing the benefits of BI can be difficult, but that it is by no means impossible. There are a range of techniques available to either directly or indirectly assess its impact. Of course there are probably other creative ways to do this that other organisations are employing and which I have not mentioned.
Business Intelligence practitioners should pay especial attention to this area, it is an opportunity to demonstrate that the yields from BI projects can be substantial. In fact, it is my opinion that in many industries no other type of IT project will have greater payback than BI. It should be a priority for those who tout the benefits of BI to show that there is real substance behind these claims. My experience suggests that it is definitely worth taking the time and effort to prove this convincingly.
33 thoughts on “Measuring the benefits of Business Intelligence”
[…] one. I firmly believe that BI done well is both the easiest of IT systems to sell to people and has one of the highest paybacks of any IT initiative. BI done badly (at the design, development, implementation or follow-up stages) will […]
Posted this comment at LinkedIn, repeated here as well.
Terrific post, beginning to think we should all take a deep breath and maybe begin to frame up or organize discussions like this around a theme our business partners understand – e.g. translating your findings for example across the five (and only five) essentials in business that matter; cash flow, profitability, velocity, growth, and customer intimacy. This theme can be extended to the non-profit or government world by simply substituting the word outcome for profitability. I guess what I’m driving at is a way to simplify our thinking and share what we are really doing in terms (lexicon) most of our customers and management can understand.
For more on this see an early post at http://pragmaticarchitect.wordpress.com/2009/02/11/where-is-enterprise-architecture-when-we-really-need-it/. And this related topic (a little off BI, but gives you an idea of what my EA peers are struggling with now – essentially the same issue in a different but related discipline) at http://pragmaticarchitect.wordpress.com/2009/02/13/the-business-case-for-enterprise-architecture/
Keep up the good work always enjoy reading your writing.
[…] 11, 2009 by James Parnitzke Peter Thomas has written a terrific post about “Measuring the benefits of Business Intelligence”. I encourage you to go see this quickly and then return before continuing on with the rest of […]
[…] EMIR Project and my collection of articles on Cultural Transformation. I have also written about the general benefits of good Business Intelligence for any organisation. This article focuses on the business benefits of BI that are specific to the […]
[…] Measuring the benefits of Business Intelligence […]
[…] is something I have heard from a number of business people when they have begun to appreciate both the power of well-designed business intelligence (BI) and, more importantly, the way that it lays bare what is happening in different parts of an […]
Peter, great article.
I take a bit of a different look on the value of BI. I work with only with software vendors looking to embed BI into their solutions. I have posted 3 parts to a 4 part series out at http://perf-mgt.blogspot.com/ entitled “Beyond the Checkbox”.
Software vendors typically think of BI as reporting. Like you mention above and in several things I see you’ve written, BI should be more about effectiveness of processes, not just dumping out counts and amounts. The true value of BI to a software vendor should be in the ability to prove they are delivering on the promise they made to their customers.
Thanks for the comment, I will check out your blog next week.
[…] Even with the most attractive of benefits to be potentially seized (and it is my firm belief that BI projects have a greater payback than many other types of IT projects), unless there is some overriding reason that work must commence, then your project is unlikely to […]
[…] Back in February, Dorothy Miller wrote a piece at IT Business Edge entitled, Measuring the Return on Investment for Business Intelligence. I wrote a comment on this, which I subsequently expanded to create my article, Measuring the benefits of Business Intelligence. […]
[…] I firmly believe that BI done well is both the easiest of IT systems to sell to people and has one of the highest paybacks of any IT initiative. BI done badly (at the design, development, implementation or follow-up stages) will […]
Examples of Business Blogs…
Here are some business blogs that catched my attention in the recent times. They are not perfect, however, they falls into a category of good blogging. Please, look at them carefully and see what you can use for your own blog. Examples of Business Blog…
[…] Measuring the benefits of Business Intelligence […]
[…] as part of broad-based BI programmes that drive business value. Though BI has the potential of releasing massive benefits for organisations, the track record has been somewhat patchy. Hopefully as people start to worry […]
[…] on business outcomes, I believe that BI can deliver substantial benefits. In fact I have argued elsewhere that BI can have the greatest payback of any IT project. Of course this presupposes that the BI […]
[…] ball either. They are extremely useful tools – indeed I have argued many times before that BI projects can have the largest payback of any IT project – but they are not universal panaceas. An inflation prediction from The Bank of […]
[…] via this facility that I came across an article on Microsoft’s site linking back to my piece Measuring the benefits of Business Intelligence. The title, sub-title and authorship of the Microsoft post is as follows: How to Measure BI […]
[…] business intelligence programmes is something that I both speak and blog about regularly. In fact Measuring the benefits of Business Intelligence remains the most frequently read article on this site (with over 4,000 hits at the time of […]
Found this really interesting thanks. If you are interested to read our similar take on business intelligence performance measurement, please do so here http://bit.ly/9rgp0u
Thanks for the note – will take a look.
[…] the meat of the presentation expanded on ideas I briefly touched on in my earlier article Measuring the Benefits of Business Intelligence. This included highlighting some of the reasons why measuring the impact of BI on, say, […]
[…] experience is that enhanced information can have enormous payback. But it is risky to embark on an information programme without this being explicitly recognised by […]
[…] some time ago, when I wrote Measuring the Benefits of Business Intelligence, I mentioned that, in some circumstances, I had been able to leverage historical data (is there any […]
That BI is about “users making better decisions” is a common misnomer and project mistake I regularly see. Ideally, BI is directed at optimizing *business processes* not informing people. Considering only the latter allows organizations to fall into the trap of delivering only interesting unused analysis rather than important actionable analytics. Similarly, considering only users (humans) as the recipients of analytics ignores the larger opportunity for optimizing operational processes.
Thanks for taking the time to comment.
Don’t want to get into analysis / analytics semantics – I spent long enough on that area in the series of posts starting with Business Analytics vs Business Intelligence to last a lifetime.
I don’t view the phrase that you pick on as a misnomer (for a start it is not a noun-phrase, but I digress), however maybe this is again semantics as you later refer to the outcome being “actionable”, which I 100% endorse. Indeed my current presentation deck includes the statements:
Business Intelligence only adds value if:
1. It answers pertinent business questions
2. These answers lead people to take action
So I live in hope that maybe we are not as far apart in our opinions as your comments suggest.
A lot of my BI work has been in the field of optimising processes, but that hasn’t excluded helping the people responsible for them to take decisions. Not 100% sure that I am getting the essence of what you are saying.
All the best
[…] o que isso tem haver com BI?Hoje já existem estudos significativos quanto a mensurar o ROI do BI, (Aqui e Aqui) é importante que se crie e use estratégias para mensurar os ganhos com investimentos […]
how many years you have to take for the bussiness intelligence
Not sure I understand the question – how long to develop a BI platform?
Interesting post Peter, I really enjoyed reading it. Will be sharing this with my colleagues!
Thank you – glad that you appreciate it – Peter
[…] See Measuring the benefits of Business Intelligence […]
I think the what if analyses are the most important benefit of BI. Being able to project what’s next for your business is invaluable. You want to create a plan and stick to it, not just hop from idea to idea.
[…] Measuring the benefits of Business Intelligence – this is one of the most frequently read articles on this site […]
You must log in to post a comment.