This article completes the three-part series which started with Using historical data to justify BI investments – Part I and continued (somewhat inevitably) with Using historical data to justify BI investments – Part II. Having presented a worked example, which focused on using historical data both to develop a profit-enhancing rule and then to test its efficacy, this final section considers the implications for justifying Business Intelligence / Data Warehouse programmes and touches on some more general issues.
 
 
The Business Intelligence angle

In my experience when talking to people about the example I have just shared, there can be an initial “so what?” reaction. It can maybe seem that we have simply adopted the all-too-frequently-employed business ruse of accentuating the good and down-playing the bad. Who has not heard colleagues say “this was a great month excluding the impact of X, Y and Z”? Of course the implication is that when you include X, Y and Z, it would probably be a much less great month; but this is not what we have done.

One goal of business intelligence is to help in estimating what is likely to happen in the future and guiding users in taking decisions today that will influence this. What we have really done in the above example is as follows:

1. shift “now” back two years in time
2. pretend we know nothing about what has happened in these most recent two years
3. develop a predictive rule based solely on the three years preceding our back-shifted “now”
4. then use the most recent two years (the ones we have metaphorically been covering with our hand) to see whether our proposed rule would have been efficacious

For the avoidance of doubt, in the previously attached example, the losses incurred in 2009 – 2010 have absolutely no influence on the rule we adopt, this is based solely on 2006 – 2008 losses. All the 2009 – 2010 losses are used for is to validate our rule.

We have therefore achieved two things:

1. Established that better decisions could have been taken historically at the juncture of 2008 and 2009
2. Devised a rule that would have been more effective and displayed at least some indication that this could work going forward in 2011 and beyond

From a Business Intelligence / Data Warehousing perspective, the general pitch is then something like:

1. if we can mechanically take such decisions, based on a very non-sophisticated analysis of data, then if we make even simple information available to the humans taking decisions (i.e. basic BI), then surely the quality of their decision-making will improve
2. If we go beyond this to provide more sophisticated analyses (e.g. including industry segmentation, analysis of insured attributes, specific products sold etc., i.e. regular BI) then we can – by extrapolation from the example – better shape the evolution of the performance of whole books of business
3. We can also monitor the decisions taken to determine the relative effectiveness of individuals and teams and compare these to their peers – ideally these comparisons would also be made available to the individuals and teams themselves, allowing them to assess their relative performance (again regular BI)
4. Finally, we can also use more sophisticated approaches, such as statistical modelling to tease out trends and artefacts that would not be easily apparent when using a standard numeric or graphical approach (i.e. sophisticated BI, though others might use the terms “data mining”, “pattern recognition” or the now ubiquitous marketing term “analytics”)

The example also says something else – although we may already have reporting tools, analysis capabilities and even people dabbling in statistical modelling, it appears that there is room for improvement in our approach. The 2009 – 2010 loss ratio was 54% and it could have been closer to 40%. Thus what we are doing now is demonstrably not as good as it could be and the monetary value of making a stepped change in information capabilities can be estimated.

In the example, we are talking about £1m of biannual premium and £88k of increased profit. What would be the impact of better information on an annual book of £1bn premium? Assuming a linear relationship and using some advanced Mathematics, we might suggest £44m. What is more, these gains would not be one-off, but repeatable every year. Even if we moderate our projected payback to a more conservative figure, our exercise implies that we would be not out of line to suggest say an ongoing annual payback of £10m. These are numbers and concepts which are likely to resonate with Executive decision-makers.

To put it even more directly an increase of £10m a year in profits would quickly swamp the cost of a BI/DW programme in very substantial benefits. These are payback ratios that most IT managers can only dream of.

 
Correlation

© xkcd.com

Let’s pause for a moment and consider the Insurance example a little more closely. What has actually happened? Well we seem to have established that performance of policies in 2006 – 2008 is at least a reasonable predictor of performance of the same policies in 2009 – 2010. Taking the mutual fund vendors’ constant reminder that past performance does not indicate future performance to one side, what does this actually mean?

What we have done is to establish a loose correlation between 2006 – 2008 and 2009 – 2010 loss ratios. But I also mentioned a while back that I had fabricated the figures, so how does that work? In the same section, I also said that the figures contained an intentional bias. I didn’t adjust my figures to make the year-on-year comparison work out. However, at the policy level, I was guilty of making the numbers look like the type of results that I have seen with real policies (albeit of a specific type). Hopefully I was reasonably realistic about this. If every policy that was bad in 2006 – 2008 continued in exactly the same vein in 2009 – 2010 (and vice versa) then my good segment would have dropped from an overall loss ratio of 54% to considerably more than 40%. The actual distribution of losses is representative of real Insurance portfolios that I have analysed. It is worth noting that only a small bias towards policies that start bad continuing to be bad is enough for our rule to work and profits to be improved. Close scrutiny of the list of policies will reveal that I intentionally introduced several counter-examples to our rule; good business going bad and vice versa. This is just as it would be in a real book of business.

Rather than continuing to justify my methodology, I’ll make two statements:

1. I have carried out the above sort of analysis on multiple books of Insurance business and come up with comparable results; sometimes the implied benefit is greater, sometimes it is less, but it has been there without exception (of course statistics being what it is, if I did the analysis frequently enough I would find just such an exception!).
2. More mathematically speaking, the actual figure for the correlation between the two sets of years is a less than stellar 0.44. Of course a figure of 1 (or indeed -1) would imply total correlation, and one of 0 would imply a complete lack of correlation, so I am not working with doctored figures. Even a very mild correlation in data sets (one much less than the threshold for establishing statistical dependence) can still yield a significant impact on profit.

 
Closing thoughts

Having gone into a lot of detail over the course of these three articles, I wanted to step back and assess what we have covered. Although the worked-example was drawn from my experience in Insurance, there are some generic learnings to be made.

Broadly I hope that I have shown that – at least in Insurance, but I would argue with wider applicability – it is possible to use the past to infer what actions we should take in the future. By a slight tweak of timeframes, we can even take some steps to validate approaches suggested by our information. It is important that we remember that the type of basic analysis I have carried out is not guaranteed to work. The same can be said of the most advanced statistical models; both will give you some indication of what may happen and how likely this is to occur, but neither of them is foolproof. However, either of these approaches has more chance of being valuable than, for example, solely applying instinct, or making decisions at random.

In Patterns, patterns everywhere, I wrote about the dangers associated with making predictions about events are essentially unpredictable. This is another caveat to be born in mind. However, to balance this it is worth reiterating that even partial correlation can lead to establishing rules (or more sophisticated models) that can have a very positive impact.

While any approach based on analysis or statistics will have challenges and need careful treatment, I hope that my example shows that the option of doing nothing, of continuing to do things how they have been done before, is often fraught with even more problems. In the case of Insurance at least – and I suspect in many other industries – the risks associated with using historical data to make predictions about the future are, in my opinion, outweighed by the risks of not doing this; on average of course!

This article is the second in what has now expanded from a two-part series to a three-part one. This started with Using historical data to justify BI investments – Part I and finishes with Using historical data to justify BI investments – Part III (once again exhibiting my talent for selecting buzzy blog post titles).
 
 
Introduction and some belated acknowledgements

The intent of these three pieces is to present a fairly simple technique by which existing, historical data can be used to provide one element of the justification for a Business Intelligence / Data Warehousing programme. Although the specific example I will cover applies to Insurance (and indeed I spent much of the previous, introductory segment discussing some Insurance-specific concepts which are referred to below), my hope is that readers from other sectors (or whose work crosses multiple sectors) will be able to gain something from what I write. My learnings from this period of my career have certainly informed my subsequent work and I will touch on more general issues in the third and final section.

This second piece will focus on the actual insurance example. The third will relate the example to justifying BI/DW programmes and, as mentioned above, also consider the area more generally.

Before starting on this second instalment in earnest, I wanted to pause and mention a couple of things. At the beginning of the last article, I referenced one reason for me choosing to put fingertip to keyboard now, namely me briefly referring to my work in this area in my interview with Microsoft’s Bruno Aziza (@brunoaziza). There were a couple of other drivers, which I feel rather remiss to have not mentioned earlier.

First, James Taylor (@jamet123) recently published his own series of articles about the use of BI in Insurance. I have browsed these and fully intend to go back and read them more carefully in the near future. I respect James and his thoughts brought some of my own Insurance experiences to the fore of my mind.

Second, I recently posted some reflections on my presentation at the IRM MDM / Data Governance seminar. These focussed on one issue that was highlighted in the post-presentation discussion. The approach to justifying BI/DW investments that I will outline shortly also came up during these conversations and this fact provided additional impetus for me to share my ideas more widely.
 
 
Winners and losers

The main concept that I will look to explain is based on dividing sheep from goats. The idea is to look at a set of policies that make up a book of insurance business and determine whether there is some simple factor that can be used to predict their performance and split them into good and bad segments.

In order to do this, it is necessary to select policies that have the following characteristics:

1. Having been continuously renewed so that they at least cover a contiguous five-year period (policies that have been “in force” for five years in Insurance parlance).

   The reason for this is that we are going to divide this five-year term into two pieces (the first three and the final two years) and treat these differently.

2. Ideally with the above mentioned five-year period terminating in the most recent complete year – at the time of writing 2010.

   This is so that the associated loss ratios better reflect current market conditions.

3. Being short-tail policies.

   I explained this concept last time round. Short-tail policies (or lines or business) are ones in which any claims are highly likely to be reported as soon as they occur (for example property or accident insurance).

   These policies tend to have a low contribution from IBNR (again see the previous piece for a definition). In practice this means that we can use the simplest of the Insurance ratios, paid loss-ratio (i.e. simply Claims divided by Premium), with some confidence that it will capture most of the losses that will be attached to the policy, even if we are talking about say 2010.

   Another way of looking at this is that (borrowing an idea discussed last time round) for this type of policy the Underwriting Year and Calendar Year treatments are closer than in areas where claims may be reported many years after the policy was in force.

Before proceeding further, it perhaps helps to make things more concrete. To achieve this, you can download a spreadsheet containing a sample set of Insurance policies, together with their premiums and losses over a five-year period from 2006 to 2010 by clicking here (this is in Office 97-2003 format – if you would prefer, there is also a PDF version available here). Hopefully you will be able to follow my logic from the text alone, but the figures may help.

A few comments about the spreadsheet. First these are entirely fabricated policies and are not even loosely based on any data set that I have worked with before. Second I have also adopted a number of simplifications:

1. There are only 50 policies, normally many thousand would be examined.
2. Each policy has the same annual premium – £10,000 (I am British!) – and this premium does not change over the five years being considered. In reality these would vary immensely according to changes in cover and the insurer’s pricing strategy.
3. I have entirely omitted dates. In practice not every policy will fit neatly into a year and account will normally need to be taken of this fact.
4. Given that this is a fabricated dataset, the claims activity has not been generated randomly. Instead I have simply selected values (though I did perform a retrospective sense check as to their distribution). While this example is not meant to 100% reflect reality, there is an intentional bias in the figures; one that I will come back to later.

The sheet also calculates the policy paid loss ratio for each year and figures for the whole portfolio appear at the bottom. While the in-year performance of any particular policy can gyrate considerably, it may be seen from the aggregate figures that overall performance of this rather small book of business is relatively consistent:

Above I mentioned looking at the five years in two parts. At least metaphorically we are going to use our right hand t cover the results from years 2009 and 2010 and focus on the first three years on the left. Later – after we have established a hypothesis based on 2006 to 2008 results – we can lift our hand and check how we did against the “real” figures.

For the purposes of this illustration, I want to choose a rather mechanistic way to differentiate business that has performed well and badly. In doing this I have to remember that a policy may have a single major loss one year and then run free of losses for the next 20. If I was simply to say any policy with a large loss is bad, I am potentially drastically and unnecessarily culling my book (and also closing the stable door after the horse has bolted). Instead we need to develop a rule that takes this into account.

In thinking about overall profitability, while we have greatly reduced the impact of both reported but unpaid claims and IBNR by virtue of picking a short-tail business, it might be prudent to make say a 5% allowance for these. If we also assume an expense ratio of 35%, then we have a total of non-underwriting-related outgoings of 40%. This means that we can afford to have a paid loss ratio of up to 60% (100% – 40%) and still turn a profit.

Using this insight, my simple rule is as follows:

This rule roughly splits the book 75 / 25; with 74% of policies being good. Other choices of parameters may result in other splits and it would be advisable spending a little time optimising things. Perhaps 26% of policies being flagged as bad is too aggressive for example (though this rather depends on what you do about them – see below). However in the simpler world of this example, I’ll press on to the next stage with my first pick.

Well all we have done so far is to tag policies that have performed badly – in the parlance of Analytics zealots we are being backward-looking. Now it is time to lift our hand on 2009 to 2010 and try to be forward-looking. While these figures are obviously also backward looking (the day that someone comes up with future data I will eat my hat), from the frame of reference of our experimental perspective (sitting at the close of 2008), they can be thought of as “the future back then”. We will use the actual performance of the policies in 2009 – 2010 to validate our choice of good and bad that was based on 2006 – 2008 results.

Overall the 50 policies had a loss ratio of 54% in 2009 – 2010. However those flagged as bad in our above exercise had a subsequent loss ratio of 92%. Those flagged as good had a subsequent loss ratio of 40%. The latter is a 14 point improvement on the overall performance of the book.

So we can say with some certainly that our rule, though simplistic, has produced some interesting results. The third part of this series will focus more closely on why this has worked. For now, let’s consider what actions the split we have established could drive.
 
 
What to do with the bad?

We were running a 54% paid ratio in 2009-2010. Using the same assumptions as above, this might have equated to a 94% combined ratio. Our book of business had an annual premium of £0.5m so we received £1m over the two years. The 94% combined would have implied making a £60k profit if we had done nothing different. So what might have happened if we had done something?

There are a number of options. The most radical of these would have been to not renew any of the bad policies; to have carried out a cull. Let us consider what would have been the impact of such an approach. Well our book of business would have shrunk to £740k over the two years at a combined of 40% (the ratio of the good book) + 40% (other outgoing) = 80%, which implies a profit of £148k, up £88k. However there are reasons why we might not have wanted to so drastically shrink our business. A smaller pot of money for investment purposes might have been one. Also we might have had customers with policies in both the good and bad segments and it might have been tricky to cancel the bad while retaining the good. And so on…

Another option would have been to have refined our rule to catch fewer policies. Inevitably, however, this would have reduced the positive impact on profits.

At the other extreme, we might have chosen to take less drastic action relating to the bad policies. This could have included increasing the premium we charged (which of course could also have resulted in us losing the business but via the insured’s choice), raising the deductible payable on any losses, or looking to work with insureds to put in place better risk management processes. Let’s be conservative and say that if the bad book was running at 92% and the overall book at 54% then perhaps it would have been feasible to improve the bad book’s performance to a neutral figure of say 60% (implying a break-even combined of 100%). This would have enabled the insurance organisation to maintain its investment base, to have not lost good business as a result of culling related bad and to have preserved the profit increase generated by the cull.

In practice of course it is likely that some sort of mixed approach would have been taken. The general point is that we have been able to come up with a simple strategy to separate good and bad business and then been able to validate how accurate our choices were. If, in the future, we possessed similar information, then there is ample scope for better decisions to be taken, with potentially positive impact on profits.
 
 
Next time…

In the final part of what is now a trilogy, I will look more deeply at what we have learnt from the above example, tie these learnings into how to pitch a BI/DW programme in Insurance and make some more general observations.
 

This is the first of what was originally a two part piece that has now expanded into three. In the initial chapter, I provide some background on Insurance industry concepts and practices. These are built on in the second chapter (Using historical data to justify BI investments – Part II), in which I offer an Insurance-based worked example. In the final piece, which is cunningly named Part III, I will explain how such an approach to analysing historical data can be used to justify BI investments.

Readers who are already au fait with insurance may choose to wait for the next instalment.
 
 
Introduction

Quite some time ago, when I wrote Measuring the Benefits of Business Intelligence, I mentioned that, in some circumstances, I had been able to leverage historical data (is there any other kind?) to justify Business Intelligence investments. I briefly touched on this area in my recent interview with Microsoft’s Bruno Aziza (@brunoaziza) and thought that it was well past time me writing more fully on the topic.

My general approach applies where there are periodic decisions to be made about a business relationship and where how that relationship has performed in the past informs these decisions. These criteria particularly pertain to the industry in which I ran my first BI / DW project; commercial property and casualty insurance. While I hope that users from other sectors may be able to extrapolate my example to apply to them, it is to insurance that I will turn to explain what I did.
 
 
An insurance primer

My previous article, The Specific Benefits of Business Intelligence in Insurance, starts with a widely used and pig-related (no typo) explanation of how insurance works, both for the insurer and the insured. I won’t repeat this here, but if you are unfamiliar with the area I recommend you taking a look first.

Although of course there are exceptions (event related insurance for example), many commercial insurance policies – just like those that most of us purchase in our personal lives to cover cars and property – have an annual term after which either party can decide whether or not to renew the cover. At renewal, as in the pig example, the insurer will first of all want to assess whether or not they have received more money than they have paid out over the past year. However, the entire point of insurance is that sometimes an event occurs which requires the insurer to give the insured a sum in excess of the premium that they have paid in a given year (or indeed over many years). The insurer is therefore less interested in whether a particular year has been bad – from their perspective – than whether the overall relationship has been, or will become, bad. Perhaps I am over simplifying, but if in most years the insurer pays out less in settling claims than they receive in premium (or ideally there are no claims at all) and if one bad year’s claims are unlikely to negate the benefits accrued in the normal years, then this is good business for the insurer.
 
 
Some rational comments

I have bandied about a number of rather woolly concepts in the previous section which include: how much money the insured has paid out and how much they have taken in. Of course these things tend to be more complicated. On the simpler side of the equation, broadly speaking, money coming in is from the insurance premiums paid by customers (but see also the box appearing below).

Equally broadly speaking, money going out is normally in six categories:

1. settlement of claims – often referred to as case payments
2. claims adjusters’ estimates for the settlement of specific claims that have been notified to the insurer, but not as yet paid – often referred to as case reserves
3. actuarial estimates of insurance events that have occurred, but which have not yet been reported to the insurer – generally known as incurred but not reported losses, or IBNR (more on this later)
4. fees paid to insurance intermediaries for placing their clients’ business with the carrier – commission
5. premiums paid to other organisations to transfer some of the risk associated with specific policies, or baskets of types of policies – facultative or treaty reinsurance
6. the general expense of being in business (staff, premises, consumables, equipment, IT, advertising, uncollectable premiums etc.)

In the cause of clarity, I will lump commission, reinsurance and the general expense of being in business into Other Expenses for what follows. However please bear in mind that, as is often the case in life, things are not as simple as I will make them out to be.

Rather than dealing in monetary units, insurance companies like percentages; though they then insist on referring to these as ratios. Taking the above categories of money flowing in and out of an insurance company, the main ratios that they consider are then:

 
 
Incurred but not reported

This concept requires a short diversion as later on I will exclude it from our discussions and will need to explain why. There are some interesting time lags in insurance. Take the sad case of asbestosis (also mentioned in my previous article). Here those unfortunately exposed developed symptoms of the disease in some cases many years later. However if their exposure was in say 1972, they would be covered by whatever Employers Liability policy their organisation held or whatever personal policy they held in the case of the self-employed. An asbestosis sufferer may have changed insurance company ten times since their exposure, but it is the insurance company who provided cover at the time who is liable for any claims.

Rather than waiting for such claims to emerge, insurance companies follow the best practise of recognising liabilities at the earliest point. Because of this, they set up estimated reserves for claims that they may receive in future years (or decades) and apply these to the year in which the policy was in force. Of course in some lines of business, say Property cover, most claims are reported as soon as they occur and so IBNR reserves are low. However in others, say Directors and Officers Liability, or the Employers Liability mentioned above, claims may arise many years hence and IBNR can be a big factor in results.

It should be stressed that IBNR is seldom calculated for a single policy (though it is conceivable that this would happen on a very large risk). Instead it is estimated for classes of policies, often grouped into lines of business, and the same “rate” of IBNR is applied across the board. Of course IBNR is calculated based on experience of losses in the same baskets of policies in previous years, adjusted to take account of current differences (e.g. more or less favourable economic conditions for Directors and Officers Liability, or maybe rising or falling property indeces for Property).

For reasons that are probably obvious, lines of business where most claims are promptly reported (i.e. low IBNR) are called short-tail lines. Those where claims may emerge some time after the period covered by the policy (i.e. high IBNR) are called long-tail lines. Later on I will be focussing just on short-tail business.

[Incidentally, improving this process of estimation is one of the specific benefits of Business Intelligence in insurance that I highlighted in my previous article.]
 
 
Underwriting Year

Something else may have occurred to readers when considering the time lags that I reference in the previous section, namely that while a policy may last from say 1^st January 2006 to 31^st December 2006, claims against this may occur either during this period, or after it. The financial statements of an insurance company will place claims in the period that they are notified or settled. So in the above example, a claim paid on 23^rd April 2008 (assuming the financial and calendar years coincide) will be reflected in the 2008 report and accounts.

However it is often useful for analysis purposes to lump together all of the claims relating to a policy and associate these with the year in which it was written. Again in our example this would mean our 23^rd April 2008 claim would be recorded in the Underwriting Year of 2006. So an Underwriting Year report comparing 2006 and 2007 say would have the premium for all policies written in 2006 and all the claims against these policies – regardless of when they occur – compared to the premium for 2007 and all the claims against these policies, whenever they occur.

Because of this, Underwriting Year reports provide a good measure of the performance of policies (or books of business) over time, regardless of how associated losses are dispersed. By contrast Calendar Year (i.e. financial) reports will often have premium from policies written in say 2010 combined with losses from policies written in say 2000 – 2010.
 
 
Tune in next time…

Having laid some foundations, in the next article, I will draw on the various concepts that I have introduced above to offer a worked example. In the closing chapter, I will explain how I such an example to justify a major, multi-year Business Intelligence / Data Warehousing programme within the insurance industry.
 

Serendipity had led to the following cartoon appearing on the web-site of that doyen of scientific humour Randall Munroe, namely xkcd.com.

A lesson for us all - © xkcd.com

I’m sure this drawing must have appeared on some other BI blogs, but what the hell, it merits posting again in my opinion.
 

	Tweet this article on twitter.com

Bookmark this article with:
| Facebook	| del.icio.us	| digg	| Reddit	| Stumble

I have been following DecisionStat’s excellent series of interviews with leading figures in the IT industry who have a focus on Business Intelligence, Analytics and Data Management. So I was delighted when I received the invitation to be interviewed by Ajay myself.

This turned into a wide-ranging discussion on a number of areas including the perception of science in society, but most of the content refers to Business Intelligence, analytics, cloud computing, data quality and related areas. You can read the interview in full by clicking on the image or text below.

DecisionStats.com Interview

Thanks to Ajay for taking the time to talk to me.
 

	Tweet this article on twitter.com

Bookmark this article with:
Technorati	| del.icio.us	| digg	| Reddit	| Stumble

I have been a featured blogger on SmartDataCollective.com almost as long as I have been a blogger. SDC.com is Social Media Today’s community site, focussed on all aspects of Business Intelligence, Data Warehousing and Analytics, with a pinch of social media thrown in to the mix.

Brian Roger, the SDC.com editor, was recently kind enough to interview me about my career in BI, the challenges I have faced and what has helped to overcome these. This interview is now available to listen to as part of their Podcast series – click on the image below to visit their site.

SmartDataCollective.com Intervew

I would be interested in feedback about any aspect of this piece, which I am grateful to Brian for arranging.
 

Social Media Today LLC, is a battle-tested, nimble team of former journalists, online managers, and advertising professionals who have come together to make a new kind of media company. With their backgrounds, and passions for, business-to-business and public policy conversations, they have decided to focus their efforts in this area. To facilitate the types of convresations that they would like to see Social Media Today is assembling the world’s best bloggers and providing them with an independent “playground” to include their posts, to comment and rate posts, and to connect with each other. On their flagship site, SocialMediaToday.com, they have brought together many of the most intriguing and original bloggers on media and marketing, covering all aspects of what makes up the connective tissue of social media from a global perspective.
 

	Tweet this article on twitter.com

Bookmark this article with:
Technorati	| del.icio.us	| digg	| Reddit	| Stumble

I have featured Neil Raden’s thoughts quite a few times on this blog. It is always valuable to learn from the perspectives and insights of people like Neil who have been in the industry a long time and to whom there is little new under the sun.

In his latest post, IBM System S: Not for Everyone (which appears on his Intelligent Enterprise blog), Neil raises concerns about some commentators’ expectations of this technology. If business intelligence is seen as having democratised information, then some people appear to feel that System S will do the same for real-time analysis of massive data sets.

While intrigued by the technology and particular opportunities that System S may open up, Neil is sceptical about some of the more eye-catching claims. One of these, quoted in The New York Times, relates to real-time analysis in a hospital context, with IBM’s wizardry potentially alerting medical staff to problems before they get out of hand and maybe even playing a role in diagnosis. On the prospects for this universal panacea becoming reality, Neil adroitly observes:

How many organizations have both the skill and organizational alignment to implement something so complex and controversial?

Neil says that he is less fond of sporting analogies than many bloggers (having recently posted articles relating to cricket, football [soccer], mountain biking and rock climbing, I find myself blushing somewhat at this point), but nevertheless goes on to make a very apposite comparison between professional sportsmen and women and carrying out real-time analysis professionally. Every day sports fans can appreciate the skill, commitment and talent of the professionals, but these people operate on a different plane from mere mortals. With System S Neil suggests that:

The vendor projects the image of Tiger Woods to a bunch of duffers.

I think once again we arrive at the verity that there is no silver bullet in any element of information generation (see my earlier article, Automating the business intelligence process?). Many aspects of the technology used in business intelligence are improving every year and I am sure that there are many wonderful aspects to System S. However, this doubting Thomas is as sceptical as Neil about certain of the suggested benefits of this technology. Hopefully some concrete and useful examples of its benefits will soon replace the current hype and provide bloggers with some more tangible fare to write about.
 

Other articles featuring Intelligent Enterprise blog posts include: “Gartner sees a big discrepancy between BI expectations and realities” – Intelligent Enterprise and Cindi Howson at Intelligent Enterprise on using BI to beat the downturn.
 

	Tweet this article on twitter.com

Bookmark this article with:
Technorati	| del.icio.us	| digg	| Reddit	| Stumble

Introduction

This post follows on from a question that was asked on the LinkedIn.com Data Warehousing Institute (TDWI™) 2.0 group. Unfortunately the original thread is no longer available for whatever reason, but the gist of the question was whether anyone had experience with using a number of BI tools to cover different functions within an implementation. So the scenario might be: Tool A for dashboards, Tool B for OLAP, Tool C for Analytics, Tool D for formatted reports and even Tool E for visualisation.

In my initial response I admitted that I had not faced precisely this situation, but that I had worked with the set-up shown in the following diagram, which I felt was not that dissimilar:

An example of a multi-tier BI architecture with different tools

Here there is no analytics tool (in the statistical modelling sense – Excel played that role) and no true visualisation (unless you count graphs in PowerPlay that is), but each of dashboards, OLAP cubes, formatted reports and simple list reports are present. The reason that this arrangement might not at first sight appear pertinent to the question asked on LinkedIn.com is that two of the layers (and three of the report technologies) are from one vendor; Cognos at the time, IBM-Cognos now. The reason that I felt that there was some relevance was that the Cognos products were from different major releases. The dashboard tool being from their Version 8 architecture and the OLAP cubes and formatted reports from their Version 7 architecture.
 
 
A little history

London Bridge circa 1600

Maybe a note of explanation is necessary as clearly we did not plan to have this slight mismatch of technologies. We initially built out our BI infrastructure without a dashboard layer. Partly this was because dashboards weren’t as much of a hot topic for CEOs when we started. However, I also think it also makes sense to overlay dashboards on an established information architecture (something I cover in my earlier article, “All that glisters is not gold” – some thoughts on dashboards, which is also pertinent to these discussions).

When we started to think about adding icing to our BI cake, ReportStudio in Cognos 8 had just come out and we thought that it made sense to look at this; both to deliver dashboards and to assess its potential future role in our BI implementation. At that point, the initial Cognos 8 version of Analysis Studio wasn’t an attractive upgrade path for existing PowerPlay users and so we wanted to stay on PowerPlay 7.3 for a while longer.

The other thing that I should mention is that we had integrated an in-house developed web-based reporting tool with PowerPlay as the drill down tool. The reasons for this were a) we had already trained 750 users in this tool and it seemed sensible to leverage it and b) employing it meant that we didn’t have to buy an additional Cognos 7 product, such as Impromptu, to support this need. This hopefully explains the mild heterogeneity of our set up. I should probably also say that users could directly access any one of the BI tools to get at information and that they could navigate between them as shown by the arrows in the diagram.

I am sure that things have improved immensely in the Cognos toolset since back then, but at the time there was no truly seamless integration between ReportStudio and PowerPlay as they were on different architectures. This meant that we had to code the passing of parameters between the ReportStudio dashboard and PowerPlay cubes ourselves. Although there were some similarities between the two products, there were also some differences at the time and these, plus the custom integration we had to develop, meant that you could also view the two Cognos products as essentially separate tools. Add in here the additional custom integration of our in-house reporting application with PowerPlay and maybe you can begin to see why I felt that there were some similarities between our implementation and one using different vendors for each tool.

I am going to speak a bit about the benefits and disadvantages of having a single vendor approach later, but for now an obvious question is “did our set-up work?” The answer to this was a resounding yes. Though the IT work behind the scenes was maybe not the most elegant (though everything was eminently supportable), from the users’ perspective things were effectively seamless. To slightly pre-empt a later point, I think that the user experience is what really matters, more than what happens on the IT side of the house. Nevertheless let’s move on from some specifics to some general comments.
 
 
The advantages of a single vendor approach to BI

One-stop shopping

I think that it makes sense if I lay my cards on the table up-front. I am a paid up member of the BI standardisation club. I think that you only release the true potential of BI when you take a broad based approach and bring as many areas as you can into your warehouse (see my earlier article, Holistic vs Incremental approaches to BI, for my reasons for believing this).

Within the warehouse itself there should be a standardised approach to dimensions (business entities and the hierarchies they are built into should be the same everywhere – I’m sure this will please all my MDM friends out there) and to measures (what is the point if profitability is defined different ways in different reports?). It is almost clichéd nowadays to speak about “the single version of the truth”, but I have always been a proponent of this approach.

I also think that you should have the minimum number of BI tools. Here however the minimum is not necessarily always one. To misquote one of Württemberg’s most famous sons:

Everything should be made as simple as possible, but no simpler.

What he actually said was:

It can scarcely be denied that the supreme goal of all theory is to make the irreducible basic elements as simple and as few as possible without having to surrender the adequate representation of a single datum of experience.

but maybe the common rendition is itself paying tribute to the principle that he propounded. Let me pause to cover what are the main reasons quoted for adopting a single vendor approach in BI:

1. Consistent look-and-feel: The tools will have a common look-and-feel, making it easier for people to use them and simplifying training.
2. Better interoperability: Interoperability between the tools is out-of-the-box, saving on time and effort in developing and maintaining integration.
3. Clarity in problem resolution: If something goes wrong with your implementation, you don’t get different vendors blaming each other for the problem.
4. Simpler upgrades: You future proof your architecture, when one element has a new release, it is the vendor’s job to ensure it works with everything else, not yours.
5. Less people needed: You don’t need to hire an expert for each different vendor tool, thereby reducing the size and cost of your BI team.
6. Cheaper licensing: It should be cheaper to buy a bundled solution from one vendor and ongoing maintenance fees should also be less.

This all seems to make perfect sense and each of the above points can be seen to be reducing the complexity and cost of your BI solution. Surely it is a no-brainer to adopt this approach? Well maybe. Let me offer some alternative perspectives on each item – none of these wholly negates the point, but I think it is nevertheless worth considering a different perspective before deciding what is best for your organisation.

1. Consistent look-and-feel: It is not always 100% true that different tools from the same vendor have the same look-and-feel. This might be down to quality control at the vendor, it might be because the vendor has recently acquired part of their product set and not fully integrated it as yet, or – even more basically – it may be because different tools are intended to do different things. To pick one example from outside of BI that has frustrated me endlessly over the years: PowerPoint and Word seem to have very little in common, even in Office 2007. Hopefully different tools from the same vendor will be able to share the same metadata, but this is not always the case. Some research is probably required here before assuming this point is true. Also, picking up on the Bauhaus ethos of form dictating function, you probably don’t want to have your dashboard looking exactly like your OLAP cubes – it wouldn’t be a dashboard then would it? Additional user training will generally be required for each tier in your BI architecture and a single-vendor approach will at best reduce this somewhat.
2. Better interoperability: I mention an problem with interoperability of the Cognos toolset above. This is is hopefully now a historical oddity, but I would be amazed if similar issues do not arise at least from time to time with most BI vendors. Cognos itself has now been acquired by IBM and I am sure everyone in the new organisation is doing a fine job of consolidating the product lines, but it would be incredible if there were not some mismatches that occur in the process. Even without acquisitions it is likely that elements of a vendor’s product set get slightly out of alignment from time to time.
3. Clarity in problem resolution: This is hopefully a valid point, however it probably won’t stop your BI tool vendor from suggesting that it is your web-server software, or network topology, or database version that is causing the issue. Call me cynical if you wish, I prefer to think of myself as a seasoned IT professional!
4. Simpler upgrades: Again this is also most likely to be a plus point, but problems can occur when only parts of a product set have upgrades. Also you may need to upgrade Tool A to the latest version to address a bug or to deliver desired functionality, but have equally valid reasons for keeping Tool B at the previous release. This can cause problems in a single supplier scenario precisely because the elements are likely to be more tightly coupled with each other, something that you may have a chance of being insulated against if you use tools from different vendors.
5. Less people needed: While there might be half a point here, I think that this is mostly fallacious. The skills required to build an easy-to-use and impactful dashboard are not the same as building OLAP cubes. It may be that you have flexible and creative people who can do both (I have been thus blessed myself in the past in projects I ran), but this type of person would most likely be equally adept whatever tool they were using. Again there may be some efficiencies in sharing metadata, but it is important not to over-state these. You may well still need a dashboard person and an OLAP person, if you don’t then the person who can do both with probably not care about which vendor provides the tools.
6. Cheaper licensing: Let’s think about this. How many vendors give you Tool B free when you purchase Tool A? Not many is the answer in my experience, they are commercial entities after all. It may be more economical to purchase bundles of products from a vendor, but also having more than one in the game may be an even better way of ensuring that cost are kept down. This is another area that requires further close examination before deciding what to do.

 
A more important consideration

Overall it is still likely that a single-vendor solution is cheaper than a multi-vendor one, but I hope that I have raised enough points to make you think that this is not guaranteed. Also the cost differential may not be as substantial as might be thought initially. You should certainly explore both approaches and figure out what works best for you. However there is another overriding point to consider here, the one I alluded to earlier; your users. The most important thing is that your users have the best experience and that whatever tools you employ are the ones that will deliver this. If you can do this while sticking to a single vendor then great. However if your users will be better served by different tools in different tiers, then this should be your approach, regardless of whether it makes things a bit more complicated for your team.

Of course there may be some additional costs associated with such an approach, but I doubt that this issue is insuperable. One comparison that it may help to keep in mind is that the per user cost of many BI tools is similar to desktop productivity tools such as Office. The main expense of BI programmes is not the tools that you use to deliver information, but all the work that goes on behind the scenes to ensure that it is the right information, at the right time and with the appropriate degree of accuracy. The big chunks of BI project costs are located in the four pillars that I consistently refer to:

1. Understand the important business decisions and what figures are necessary to support these.
2. Understand the data available in the organisation, how it relates to other data and to business decisions.
3. Transform the data to provide information answering business questions.
4. Focus on embedding the use of information in the corporate DNA.

The cost of the BI tools themselves are only a minor part of the above (see also, BI implementations are like icebergs). Of course any savings made on tools may make funds available for other parts of the project. It is however important not to cut your nose off to spite your face here. Picking right tools for the job, be they from one vendor or two (or even three at a push) will be much more important to the overall payback of your project than saving a few nickels and dimes by sticking to a one-vendor strategy just for the sake of it.
 

	Tweet this article on twitter.com

Bookmark this article with:
Technorati	| del.icio.us	| digg	| Reddit	| Stumble

This article is another relating to a discussion on LinkedIn.com. As with my earlier piece, Short-term “Trouble for Big Business Intelligence Vendors” may lead to longer-term advantage, this was posted on the Chief Information Officer (CIO) Network group

The thread was initiated by Patrick Gray and was entitled: Is IT partially to blame for the financial crisis? (as ever you need to be a member of LinkedIn.com and the group to view this).

Patrick asked:

Information is one of the key components of any IT organization (I would personally argue it’s more important than the technology aspect). Two facts disturb me when one looks at IT’s role in the financial crisis:

1) We in IT have been pushing data warehouse and business intelligence technology for years, saying these technologies should allow for “proactive” decision making at all levels of an organization, and an ability to spot trends and changes in a business’ underlying financial health.

2) The finance industry is usually spends more on IT than any other industry.

This being the case, if BI actually does what we’ve pitched it to do, shouldn’t one of these fancy analytical tools spotted the underlying roots of the financial crisis in at least one major bank? Is IT partially culpable for either not looking at the right data, or selling a bill of goods in terms of the “intelligence” aspect of BI?

I have written elsewhere on LinkedIn.com about business intelligence’s role in the financial crisis. My general take is that if the people who were committing organisations to collateralised debt obligations and other even more esoteric assent-backed securities were unable (or unwilling) to understand precisely the nature of the exposure that they were taking on, then how could this be reflected in BI systems. Good BI systems reflect business realities and risk is one of those realities. However if risk is as ill-understood as it appears to have been in many financial organisations, then it is difficult to see how BI (or indeed it’s sister area of business analytics) could have shed light where the layers of cobwebs were so dense.

So far, so orthodox, but Patrick’s question got me thinking along a different line, one that is more closely related to the ideas that I propounded in Business is from Mars and IT is from Venus last year. I started wondering, ‘is it just too easy for IT to say, “the business people did not understand the risks, so how were we expected to?”?’ (I think I have that punctuation right, but would welcome corrections from any experts reading this). This rather amorphous feeling was given some substance when I read some of the other responses.

However, I don’t want to focus too much on any one comment. My approach will be instead to take a more personal angle and describe some of the thoughts that the comments provoked in me (I am using “provoked” here in a positive sense, maybe “inspired” would have been a better choice of word). If you want to read my comments with the full context, then please click on the link above. What I am going to do here is to present some excerpts from each of my two lengthier contributions. The first of these is as follows (please note that I have also corrected a couple of typos and grammatical infelicities):

Rather than being defensive, and as a BI professional I would probably have every right to be so, I think that Patrick has at least half a point. If some organisations had avoided problems (or mitigated their impact) through the use of good BI (note the adjective) in the current climate, then BI people (me included) would rush to say how much we had contributed. I have certainly done this when the BI systems that I have implemented helped an organisation to swing from record losses to record profits.

Well if we are happy to do this, then we have to take some responsibility when things don’t go so well. It worries me when IT people say that non-IT managers are accountable for the business and IT is just accountable for IT. Surely in a well-functioning organisation, IT is one department that shares responsibility for business success with all the other front-line and service departments.

I have seen it argued with respect to failed financial institutions that IT can only provide information and that other executives take decisions. Well if this is the case, then I question how well the information has been designed to meet business needs and to drive decisions. To me this is evidence of bad BI (note the adjective again).

There are some specific mitigating factors for IT within the current climate, including poor internal (non-IT) governance and the fact that even the people who were writing some financial instruments did not understand the potential liabilities that the we taking on. If this is the case, then how can such risk be rolled up meaningfully? However these factors do not fully exculpate IT in my opinion. I am not suggesting for a second that IT take prime responsibility, but to claim no responsibility whatsoever is invidious.

So yes either poor information, or a lack of information (both of which are IT’s fault – as well as that of non-IT business folk) are a contributory factors to the current problems.

Also, while IT managers see themselves as responsible only for some collateral department, semi-detached from the rest of the business, we will see poor IT and poor information continuing to contribute to business failure.

This is the second passage:

[...]

I just wonder how it is that IT people at such firms can say that any failures are 100% nothing to do with them, as opposed to say 1% responsibility, or something of that nature.

Part of the role of professionals working in BI is to change the organisation so that numerical decision making (backed up of course by many other things, including experience and judgement) becomes part of the DNA. We are to blame for this not being the case in many organisations and can’t simply throw our hands up and say “wasn’t me”.

[...]

I will freely admit that there was a large dose of Devil’s Advocate in my two responses. As I have stated at the beginning of this piece, I am not so masochistic to believe that IT caused the current financial crisis, however I do not think that IT can be fully absolved of all blame.

My concerns about IT’s role relate to the situation that I see in some companies where IT is a department set apart, rather than being a central part of the overall business. In this type of circumstance (which is perhaps more common than anyone would like to think), the success of the IT and the non-IT parts of the business are decoupled.

Under these arrangements, it would be feasible for IT to be successful and the business to suffer major losses, or for the business to post record profits while IT fails to deliver projects. Of couse such decoupling can happen in other areas; for example Product A could have a stellar year, while Product B fails miserably – the same could happen with countries or regions. However there is something else here, a sense that IT can sometimes be an organisation within an organisation, in a way that other service departments generally are not.

Rather than expanding further on this concept here, I recommend you read Jim Anderson’s excellent article Here’s What’s Really Wrong With IT And How To Fix It on his blog, The Business of IT. I think that there is a good deal of alignment between Jim and I on this issue; indeed I was very encouraged to find his blog and see that his views were not a million miles from my own.

I would also like to thank Patrick for posting his initial question. It’s good when on-line forums lead you to take an alternative perspective on things.
 

Also check out Jill Dyché’s article: Dear IT: A Letter from Your Business Users
 

	Tweet this article on twitter.com

Bookmark this article with:
Technorati	| del.icio.us	| digg	| Reddit	| NewsVine

There is clear water between the top five and the chasing pack. The next most popular article, The Top Business Issues facing CIOs / IT Directors – Results, is 176 hits back on 694. Of course one might assume that an “all-time” list would favour the earliest posts on this site. It is therefore interesting to note that the list instead features a number of more recent pieces and that the only really “old” piece is my first article plagiarising John Gray’s famous book.

I should also note that I have removed Keynote Articles (1,073 hits) from the list as this page aggregates all of my other posts, rather than being an article in its own right.

Of course this is merely a snap-shot of today’s figures and the list is already out of date as I write. I may look to update the figures occasionally, perhaps every three to six months.

	Tweet this article on twitter.com

Bookmark this article with:
Technorati	| del.icio.us	| digg	| Reddit	| NewsVine