Using historical data to justify BI investments – Part II

12 May 2011

The earliest recorded surd

This article is the second in what has now expanded from a two-part series to a three-part one. This started with Using historical data to justify BI investments – Part I and finishes with Using historical data to justify BI investments – Part III (once again exhibiting my talent for selecting buzzy blog post titles).
 
 
Introduction and some belated acknowledgements

The intent of these three pieces is to present a fairly simple technique by which existing, historical data can be used to provide one element of the justification for a Business Intelligence / Data Warehousing programme. Although the specific example I will cover applies to Insurance (and indeed I spent much of the previous, introductory segment discussing some Insurance-specific concepts which are referred to below), my hope is that readers from other sectors (or whose work crosses multiple sectors) will be able to gain something from what I write. My learnings from this period of my career have certainly informed my subsequent work and I will touch on more general issues in the third and final section.

This second piece will focus on the actual insurance example. The third will relate the example to justifying BI/DW programmes and, as mentioned above, also consider the area more generally.

Before starting on this second instalment in earnest, I wanted to pause and mention a couple of things. At the beginning of the last article, I referenced one reason for me choosing to put fingertip to keyboard now, namely me briefly referring to my work in this area in my interview with Microsoft’s Bruno Aziza (@brunoaziza). There were a couple of other drivers, which I feel rather remiss to have not mentioned earlier.

First, James Taylor (@jamet123) recently published his own series of articles about the use of BI in Insurance. I have browsed these and fully intend to go back and read them more carefully in the near future. I respect James and his thoughts brought some of my own Insurance experiences to the fore of my mind.

Second, I recently posted some reflections on my presentation at the IRM MDM / Data Governance seminar. These focussed on one issue that was highlighted in the post-presentation discussion. The approach to justifying BI/DW investments that I will outline shortly also came up during these conversations and this fact provided additional impetus for me to share my ideas more widely.
 
 
Winners and losers

Before him all the nations will be gathered, and he will separate them one from another, as a shepherd separates the sheep from the goats

The main concept that I will look to explain is based on dividing sheep from goats. The idea is to look at a set of policies that make up a book of insurance business and determine whether there is some simple factor that can be used to predict their performance and split them into good and bad segments.

In order to do this, it is necessary to select policies that have the following characteristics:

  1. Having been continuously renewed so that they at least cover a contiguous five-year period (policies that have been “in force” for five years in Insurance parlance).

    The reason for this is that we are going to divide this five-year term into two pieces (the first three and the final two years) and treat these differently.

  2. Ideally with the above mentioned five-year period terminating in the most recent complete year – at the time of writing 2010.

    This is so that the associated loss ratios better reflect current market conditions.

  3. Being short-tail policies.

    I explained this concept last time round. Short-tail policies (or lines or business) are ones in which any claims are highly likely to be reported as soon as they occur (for example property or accident insurance).

    These policies tend to have a low contribution from IBNR (again see the previous piece for a definition). In practice this means that we can use the simplest of the Insurance ratios, paid loss-ratio (i.e. simply Claims divided by Premium), with some confidence that it will capture most of the losses that will be attached to the policy, even if we are talking about say 2010.

    Another way of looking at this is that (borrowing an idea discussed last time round) for this type of policy the Underwriting Year and Calendar Year treatments are closer than in areas where claims may be reported many years after the policy was in force.

Before proceeding further, it perhaps helps to make things more concrete. To achieve this, you can download a spreadsheet containing a sample set of Insurance policies, together with their premiums and losses over a five-year period from 2006 to 2010 by clicking here (this is in Office 97-2003 format – if you would prefer, there is also a PDF version available here). Hopefully you will be able to follow my logic from the text alone, but the figures may help.

A few comments about the spreadsheet. First these are entirely fabricated policies and are not even loosely based on any data set that I have worked with before. Second I have also adopted a number of simplifications:

  1. There are only 50 policies, normally many thousand would be examined.
  2. Each policy has the same annual premium – £10,000 (I am British!) – and this premium does not change over the five years being considered. In reality these would vary immensely according to changes in cover and the insurer’s pricing strategy.
  3. I have entirely omitted dates. In practice not every policy will fit neatly into a year and account will normally need to be taken of this fact.
  4. Given that this is a fabricated dataset, the claims activity has not been generated randomly. Instead I have simply selected values (though I did perform a retrospective sense check as to their distribution). While this example is not meant to 100% reflect reality, there is an intentional bias in the figures; one that I will come back to later.

The sheet also calculates the policy paid loss ratio for each year and figures for the whole portfolio appear at the bottom. While the in-year performance of any particular policy can gyrate considerably, it may be seen from the aggregate figures that overall performance of this rather small book of business is relatively consistent:

Year Paid Loss Ratio
2006 53%
2007 59%
2008 54%
2009 53%
2010 54%
Total 54%

Above I mentioned looking at the five years in two parts. At least metaphorically we are going to use our right hand t cover the results from years 2009 and 2010 and focus on the first three years on the left. Later – after we have established a hypothesis based on 2006 to 2008 results – we can lift our hand and check how we did against the “real” figures.

For the purposes of this illustration, I want to choose a rather mechanistic way to differentiate business that has performed well and badly. In doing this I have to remember that a policy may have a single major loss one year and then run free of losses for the next 20. If I was simply to say any policy with a large loss is bad, I am potentially drastically and unnecessarily culling my book (and also closing the stable door after the horse has bolted). Instead we need to develop a rule that takes this into account.

In thinking about overall profitability, while we have greatly reduced the impact of both reported but unpaid claims and IBNR by virtue of picking a short-tail business, it might be prudent to make say a 5% allowance for these. If we also assume an expense ratio of 35%, then we have a total of non-underwriting-related outgoings of 40%. This means that we can afford to have a paid loss ratio of up to 60% (100% – 40%) and still turn a profit.

Using this insight, my simple rule is as follows:

A policy will be tagged as “bad” if two things occur:

  1. The overall three-year loss ratio is in excess of 60%

    i.e. is has been unprofitable over this period; and

  2. The loss ratio is in excess of 30% in at least two of the three years

    i.e. there is a sustained element to the poor performance and not just the one-off bad luck that can hit the best underwritten of policies

This rule roughly splits the book 75 / 25; with 74% of policies being good. Other choices of parameters may result in other splits and it would be advisable spending a little time optimising things. Perhaps 26% of policies being flagged as bad is too aggressive for example (though this rather depends on what you do about them – see below). However in the simpler world of this example, I’ll press on to the next stage with my first pick.

The ultimate sense of perspective

Well all we have done so far is to tag policies that have performed badly – in the parlance of Analytics zealots we are being backward-looking. Now it is time to lift our hand on 2009 to 2010 and try to be forward-looking. While these figures are obviously also backward looking (the day that someone comes up with future data I will eat my hat), from the frame of reference of our experimental perspective (sitting at the close of 2008), they can be thought of as “the future back then”. We will use the actual performance of the policies in 2009 – 2010 to validate our choice of good and bad that was based on 2006 – 2008 results.

Overall the 50 policies had a loss ratio of 54% in 2009 – 2010. However those flagged as bad in our above exercise had a subsequent loss ratio of 92%. Those flagged as good had a subsequent loss ratio of 40%. The latter is a 14 point improvement on the overall performance of the book.

So we can say with some certainly that our rule, though simplistic, has produced some interesting results. The third part of this series will focus more closely on why this has worked. For now, let’s consider what actions the split we have established could drive.
 
 
What to do with the bad?

You shall be taken to the place from whence you came...

We were running a 54% paid ratio in 2009-2010. Using the same assumptions as above, this might have equated to a 94% combined ratio. Our book of business had an annual premium of £0.5m so we received £1m over the two years. The 94% combined would have implied making a £60k profit if we had done nothing different. So what might have happened if we had done something?

There are a number of options. The most radical of these would have been to not renew any of the bad policies; to have carried out a cull. Let us consider what would have been the impact of such an approach. Well our book of business would have shrunk to £740k over the two years at a combined of 40% (the ratio of the good book) + 40% (other outgoing) = 80%, which implies a profit of £148k, up £88k. However there are reasons why we might not have wanted to so drastically shrink our business. A smaller pot of money for investment purposes might have been one. Also we might have had customers with policies in both the good and bad segments and it might have been tricky to cancel the bad while retaining the good. And so on…

Another option would have been to have refined our rule to catch fewer policies. Inevitably, however, this would have reduced the positive impact on profits.

At the other extreme, we might have chosen to take less drastic action relating to the bad policies. This could have included increasing the premium we charged (which of course could also have resulted in us losing the business but via the insured’s choice), raising the deductible payable on any losses, or looking to work with insureds to put in place better risk management processes. Let’s be conservative and say that if the bad book was running at 92% and the overall book at 54% then perhaps it would have been feasible to improve the bad book’s performance to a neutral figure of say 60% (implying a break-even combined of 100%). This would have enabled the insurance organisation to maintain its investment base, to have not lost good business as a result of culling related bad and to have preserved the profit increase generated by the cull.

In practice of course it is likely that some sort of mixed approach would have been taken. The general point is that we have been able to come up with a simple strategy to separate good and bad business and then been able to validate how accurate our choices were. If, in the future, we possessed similar information, then there is ample scope for better decisions to be taken, with potentially positive impact on profits.
 
 
Next time…

In the final part of what is now a trilogy, I will look more deeply at what we have learnt from the above example, tie these learnings into how to pitch a BI/DW programme in Insurance and make some more general observations.
 


A bad workman blames his [Business Intelligence] tools

29 July 2009

Tools
 
Introduction

This is a proverb with quite some history to it. Indeed its lineage has been traced to 13th Century France in: mauvés ovriers ne trovera ja bon hostill (les mauvais ouvriers ne trouveront jamais un bon outil being a rendition in more contemporary French). To me this timeless observation is applicable to present-day Business Intelligence projects. Browsing through on-line forums, it is all too typical to see discussions that start “What is the best BI software available on the market?”, “Who are the leaders in SaaS BI?” and (rather poignantly in my opinion) “Please help me to pick the best technology for a dashboard.” I feel that these are all rather missing the point. Before I explain why, I am going to offer another of my sporting analogies, which I believe is pertinent. Indeed sporting performace is an area to which the aphorism appearing in the title is frequently applied.

If you would like to skip the sporting analogy and cut to the chase, please click here.
 
 
The importance of having the right shoes

Rock climbing is a sport that certainly has its share of machismo; any climbing magazine or web-site will feature images of testosterone-infused youths whose improbable physiques (often displayed to full advantage by the de rigueur absence of any torso-encumbering clothing) propel them to the top of equally improbable climbs.

Given this, many commentators have noted the irony of climbing being conducted by people wearing the equivalent of rubber-covered ballet slippers. The fact that one of the most iconic rock climbing shoes of all time was a fetching shade of pink merely adds piquancy to this observation. Examples of these, the classic FiveTen Anasazi Lace-ups, are featured in the following photo of top British climber, Steve McClure (yes it is the right way up).

The UK's finest sport climer - Steve McClure - sports the Pink'uns

When I started rock climbing, my first pair of shoes were Zephyrs from Spanish climbing firm Boreal. They looked something like this:

The Zephyr by Boreal - $87 - £67

The Zephyr by Boreal - $87 - £67

Although it might not be apparent from the above image, these are intended to be comfortable shoes. Ones to be worn by more experienced climbers on long mountain days, or suitable for beginners, like myself at the time, on shorter climbs. Although not exactly cheap, they are not prohibitively expensive and the rubber on the soles is quite hard-wearing as well.

The Zephyrs worked well for me, but inevitably over time you begin to notice the shoes worn by better climbers at the crag or at the wall. You also cannot fail to miss the much sexier shoes worn by professional climbers in films, climbing magazine articles and (no coincidence here) advertisements. These other shoes also cost more (again no coincidence) and promise better performance. When you are looking to get better at something, it is tempting to take any advantage that you can get. Also, perhaps especially when you are looking to break into a new area, there is some pressure to conform, to look like the “in-crowd”, maybe even simply to distance yourself from the beginner that you were only a few months previously.

This is very shallow behaviour of course, but it is also the rock on which the advertising industry is founded. I wanted to get better as a climber, but would have to admit that other, less noble, motives also drove me to wanting to purchase new rock shoes.

The Galileo by FiveTen - $130 - £85

The Galileo by FiveTen - $130 - £85

The Galileos shown above are made by US company FiveTen and are representative of the type of shoes that I have worn for most my climbing career. FiveTen shoes have been worn by many top climbers over the years (though there have recently been some quite high-profile defections to start-up brand Evolv, who can never seem to decide whether to append a final ‘e’ to their name or not).

Amongst other things, FiveTens are noted for the stickiness of their rubber, which is provided by an organisation called Stealth Rubber and appears on no other rock climbing shoes. Generally the greater the adhesion between your foot and the rock, the greater the force that you can bring to bear on it to drive yourself upwards. Also it helps to have confidence that your foot has a good chance of staying in place, no matter how glassy the rock may be (and no matter how long the fall may be should this not happen). I have worn FiveTen shoes on all of my hardest climbs (none of which have actually been very hard in the grand scheme of things sad to say).

The Solution by La Sportiva - $155 - £120 (link goes to the Sportiva site)

The Solution by La Sportiva - $155 - £120
(the link loads a Flash page on the Sportiva site)

Nevertheless, with what I admit was rather a sense of guilt, I have recently embarked on a dalliance with another rock shoe manufacturer, La Sportiva of Italy. The Sportiva Solutions which are shown above are both the most expensive rock shoes I have ever owned and the most technical. If NASA made a rock shoe, they would probably not be a million miles away from the Solutions. The radical nature of their design can perhaps best be appreciated in three dimensions and you can do this by clicking on the above image.

The Solutions are very, very good rock shoes. I recently had the opportunity to carry out a before and after comparison on the following climb, A Miller’s Tale:

A Miller's Tale (V4/Font 6b+) - Rubicon Wall, Derbyshire, England

A Miller's Tale (V4/Font 6b+)
Rubicon Wall, Derbyshire, England
© http://77jenn.blogspot.com

My partner, who appears in the photo (incidentally sporting FiveTen shoes), climbed this on her second go. By contrast, I had many fruitless attempts wearing my own pair of FiveTens (that, to be fair to FiveTen, were much less technical than the Galileo’s above and were also probably past the end of their useful life). I frequently found my feet skittering off of the highly polished limestone, which resulted in me rapidly returning to terra firma.

A couple of weeks later, equipped with my shiny new Sportivas, my feet did not slip once. Of course the perfect end to this story would have been to say that I then climbed the problem (for an explanation of why some types of climbs are called problems see my earlier article Perseverance). Sadly, though I made much more progress during my second session, I need to go back to finally tick it off of my list.

So here surely is an example of the tool making a difference, or is it? My partner had climbed A Miller’s Tale quite happily without having the advantage of my new footwear. She is 5’3″ (160cm) compared to my 5’11” (180cm) and the taller you are the easier it is to reach the next hold. Strength is a factor in climbing and I am also stronger in absolute terms than she is. The reason that she succeeded where I failed is simply that she is a better climber than I am. It is an oft-repeated truism in the climbing world that many females have better techniques than men. This, together with the “unfair” advantage of smaller fingers, is the excuse often offered by muscle-bound men who fail to complete a climb that a female then dances her way up. However in my partner’s case, she is also very strong, with her power-to-weight ratio being the key factor. You don’t need to lift massive weights in climbing, just your own body.

So I didn’t really need better rock shoes to prevent my feet from slipping. If I got my body into a better balanced position, then this would have had the same impact. Equally, if my abdominal muscles were stronger, I could have squeezed my feet harder onto the rock, increasing their adhesion (this type of strength, known as core strength for obvious reasons, is crucial to progressing in many types of climbing). What the Solutions did was not to make me a better climber, but to make up for some of my inadequacies. In this way, by allowing me the luxury of not focussing on increasing my strength or improving my technique, you could even argue that they might be bad for my climbing in the long run. I probably protest too much in this last comment, but hopefully the reader can appreciate the point that I am trying to make.

Campus board training

In order to become a better climber I need to do lots of things. I need to strengthen the tendons in my fingers (or at least in nine of them as I ruptured the tendon in my right ring finger playing rugby years ago) so that I can hold on to smaller edges and grasp larger ones for longer. I need to develop my abdominal muscles to hold me onto the rock face better and put more pressure on my feet; particularly when the climb is overhanging. I need to build up muscles in my back, shoulders and arms to be able to move more assuredly between holds that are widely spaced. I must work on my endurance, so that I do not fail climbs because I am worn out by a long series of lower moves. Finally I need to improve my technique: making my footwork more precise; paying more attention to the shape of my body and how this affects my centre of gravity and the purchase I have on holds; getting more comfortable with the tricks of the trade such as heel- and toe-hooks; learning when to be aggressive in my climbing and when to be slow and deliberate; and finally better visualising how my body fits against the rock and the best way to flow economically from one position to the next.

If I can get better in all of these areas, then maybe I will have earned my new technical rock shoes and I will be able to take advantage of the benefits that they offer. Having the right shoes can undoubtedly improve your climbing, but it is no substitute for focussing on the long list in the previous paragraph. There is no real short-cut to becoming a better climber, it just takes an awful lot of work.

A final thing to add in this section is that the Solutions offer advantages to the climber on certain types of climbs. On any overhanging, pocketed rock, they are brilliant. But the way that they shape your foot into a down-turned claw would be a positive disadvantage when trying to pad up a slab. In this second scenario, something like my worn out FiveTens (now sadly consigned to the rubbish tip) would be the tool of choice. It is important to realise that the right tool is often dictated by the task in hand and one that excels in area A may be an also-ran in area B.

Notes:

  1. Lest it be thought that the above manufacturers play only in narrow niches, I should explain that each of Boreal, FiveTen and La Sportiva produce a wide range of rock shoes catering to virtuially every type of climber from the neophyte to the world’s best.
  2. If you think that the pound dollar rates are rather strange in the above exhibits, then a few things are at play. Some are genuine differences, but others are because they are historical rates. for example, I struggled to find a US web-site that still sells Boreal Zephyrs.
  3. If you are interested in finding out more about my adventures in rock climbing, then take a look at my partner’s blog.

 
 
The role of technology in Business Intelligence

I hope that I have established that at least in the world of rock climbing, the technology that you have at your disposal is only one of many factors necessary for success; indeed it is some way from being the most important factor.

Having really poor, or worn out, rock shoes can dent your confidence and even get you into bad habits (such as not using your feet enough). Having really good rock shoes can bring some incremental benefits, but these are not as great as those to be gained by training and experience. Most of the technologically-related benefits will be realised by having reasonably good and reasonably new shoes.

While the level of a professional rock climber’s performance will be undoubtedly be improved by using the best equipment available, a bad climber with $150 rock shoes will still be a bad climber (note this is not intended to be a self-referential comment).

Requirements - Data Analysis - Information - Manage Change

Requirements - Data Analysis - Information - Manage Change

Returning to another of my passions, Business Intelligence, I see some pertinent parallels. In a series of previous articles (including BI implementations are like icebergs, “All that glisters is not gold” – some thoughts on dashboards and Short-term “Trouble for Big Business Intelligence Vendors” may lead to longer-term advantage) , I have laid out my framework for BI success and explained why I feel that technology is not the most important part of a BI programme.

My recommended approach is based on four pillars:

  1. Determine what information is necessary to drive key business decisions.
  2. Understand the various data sources that are available and how they relate to each other.
  3. Transform the data to meet the information needs.
  4. Manage the embedding of BI in the corporate culture.

Obviously good BI technology has a role to play across all of these areas, but it is not the primary concern in any of them. Let us consider what is often one of the most difficult areas to get right, embedding BI in an organisation’s DNA. What is the role of the BI tool here?

Well if you want people to actually use the BI system, it helps if the way that the BI technology operates is not a hindrance to this. Ideally the ease-of-use and intuitiveness of the BI technology deployed should be a plus point for you. However, if you have the ultimate in BI technology, but your BI system does not highlight areas that business people are interested in, does not provide information that influences actual decision-making, or contains numbers that are inaccurate, out-of-date, or unreconciled, then it will not be used. I put this a little more succinctly in a recent article: Using multiple business intelligence tools in an implementation – Part II (an inspired title I realise), which I finished by saying:

If your systems do not have credibility with your users, then all is already lost and no amount of flashy functionality will save you.

Similar points can be made about all of the other pillars. Great BI technology should be the icing on your BI cake, not one of the main ingredients.
 
 
The historical perspective

What Car?

Ajay Ohri from the DecisionStats web-site recently interviewed me in some depth about a range of issues. He specifically asked me about what differentiated the various BI tools and I reproduce my reply here:

The really important question in BI is not which tool is best, but how to make BI projects successful. While many an unsuccessful BI manager may blame the tool or its vendor, this is not where the real issues lie. I firmly believe that successful BI rests on four mutually reinforcing pillars: understand the questions the business needs to answer, understand the data available, transform the data to meet the business needs and embed the use of BI in the organisation’s culture. If you get these things right then you can be successful with almost any of the excellent BI tools available in the marketplace. If you get any one of them wrong, then using the paragon of BI tools is not going to offer you salvation.

I think about BI tools in the same way as I do the car market. Not so many years ago there were major differences between manufacturers. The Japanese offered ultimate reliability, but maybe didn’t often engage the spirit. The Germans prided themselves on engineering excellence, slanted either in the direction of performance or luxury, but were not quite as dependable as the Japanese. The Italians offered out-and-out romance and theatre, with mechanical integrity an afterthought. The French seemed to think that bizarrely shaped cars with wheels as thin as dinner plates were the way forward, but at least they were distinctive. The Swedes majored on a mixture of safety and aerospace cachet, but sometimes struggled to shift their image of being boring. The Americans were still in the middle of their love affair with the large and the rugged, at the expense of convenience and value-for-money. Stereotypically, my fellow-countrymen majored on agricultural charm, or wooden-panelled nostalgia, but struggled with the demands of electronics.

Nowadays, the quality and reliability of cars are much closer to each other. Most manufacturers have products with similar features and performance and economy ratings. If we take financial issues to one side, differences are more likely to related to design, or how people perceive a brand. Today the quality of a Ford is not far behind that of a Toyota. The styling of a Honda can be as dramatic as an Alfa Romeo. Lexus and Audi are playing in areas previously the preserve of BMW and Mercedes and so on. To me this is also where the market for BI tools is at present. It is relatively mature and the differences between product sets are less than before.

Of course this doesn’t mean that the BI field will not be shaken up by some new technology or approach (in-memory BI or SaaS come to mind). This would be the equivalent of the impact that the first hybrid cars had on the auto market. However, from the point of view of implementations, most BI tools will do at least an adequate job and picking one should not be your primary concern in a BI project.

If you are interested, you can read the full interview here.
 
 
The current reality

IBM to acquire SPSS

As my comments to Ajay suggest, maybe in past times there were greater differences between BI vendors and the tools that they supplied. One benefit of the massive consolidation that has occurred in recent years is that the five biggest players: IBM/Cognos, Oracle/Hyperion, SAP/BusinessObjects, Microsoft and (the as yet still independent) SAS all have product portfolios that are both wide and deep. If there is something that you want your BI tool to do, it is likely that any of these organisations can provide you with the software; assuming that your wallet allows it. Both the functionality and scope of offerings from smaller vendors operating in the BI arena have also increased greatly in recent times. Finding a technology that fits your specific needs for functionality, ease-of-use, scalability and reliability should not be a problem.

This general landscape is one against which it is interesting to view the recent acquisition of business analytics firm SPSS by IBM. According to Reuters, IBM’s motivations are as follows:

IBM plans to buy business analytics company SPSS Inc for $1.2 billion in cash to better compete with Oracle Corp and SAP AG in the growing field of business intelligence

Full story here.

As an aside, should both Microsoft and SAS be worried that they are omitted from this list?

Whatever the corporate logic for IBM, to me this is simply more evidence that BI technology is becoming a utility (it should however be noted that this is not the same as BI itself becoming a utility). I believe that this trend will lead to a greater focus on the use of BI technology as part of broad-based BI programmes that drive business value. Though BI has the potential of releasing massive benefits for organisations, the track record has been somewhat patchy. Hopefully as people start to worry less about BI technology and more about the factors that really drive success in BI programmes, this will begin to change.

A precursor to Business Intelligence

As with any technical innovation over the centuries, it is only when the technology itself becomes invisible that the real benefits flow.
 

tweet this Tweet this article on twitter.com
Bookmark this article with:
| Facebook | del.icio.us | digg | Reddit | Stumble

 


New Social Media Section

26 July 2009

My Keynote Articles page gathers together links to my more substantial blog posts in one place and groups them into broad areas of interest. When I first created this, the four areas were as follows:

1. Business Intelligence
2. Cultural Transformation
3. IT / Business Alignment and IT Strategy
4. General Articles

Back in April 2009, at the height of the Oracle / Sun fervour, I added a new section to reflect the number of articles I was writing about general technology issues such as these and also BI industry news. This was as follows:

5. Industry Commentary

In a similar vein, I have recently been writing more about issues in Social Media (very self-referential for a blog of course). I have also had my artciles syndicated on SmartDataCollective.com for many months.

SocialMediaToday

The thing that finally decided me to have a section dedicated to Social Media was joining SmartDataCollective’s parent site, SocialMediaToday.com, who were recently kind enough to syndicate the first of my series of articles on Social Media: New Adventures in Wi-Fi – Track 1: Blogging (the original article may be viewed here).

My thinking was, if I am having my articles posted on a site devoted to Social Media, then the time has come to recognise this by creating a new section. This is what I have done with:

6. Social Media

As with the other sections, I will keep this list up-to-date as I add new content. In particular my forthcoming pieces on micro-blogging with Twitter.com and professional networking on LinkedIn.com will have a shiny new home.
 

tweet this Tweet this article on twitter.com
Bookmark this article with:
| Facebook | del.icio.us | digg | Reddit | Stumble

 


New Adventures in Wi-Fi – Track 1: Blogging

24 July 2009

New Adventures in Wi-Fi (with apologies to R.E.M.)
 
Introduction

I established this blog back in November 2008 – shortly after this I joined twitter.com in December 2008 – I had already been a member of LinkedIn.com since July 2005. However, my involvement with what is now collectively called social media goes back a lot further than this. Back then we tended to use the phrase on-line communities to describe what we were engaged in.

My first foray into this new world was in 1998/99 when I joined a, now defunct, discussion forum (then known as a Bulletin Board). This was focused on computer games. I wasn’t terribly in to such games at the time, I didn’t own a console and my PC was used for more prosaic purposes. Nevertheless, for reasons that I will not bore the reader with, I signed up. Since then I have been a member of a number of on-line forums, mostly with some sporting element, for example rock climbing.

Yahoo! Geocities

In May 1999, my forum activities led me to creating my first web-site (again now also defunct). I started on Geocities (another chance to use the word “defunct”) and then moved to having my own domain and an agreement with a hosting company. I even ended up jointly running a very successful forum with an on-line friend from Australia. Back then the men were real men, the women were real women and the HTML was real HTML. However this article is not about ancient history, but rather about my more recent experiences in social media.

Nowadays, nobody seems to think of it as being odd that you regularly “speak” to people you have never met and who inhabit countries on the other side of the world. People do not slowly back away from you at parties if you drop the fact that you have your own web-site into a conversation (though maybe one reason that the portmanteau of web-log became socially acceptable is that its abridgement to blog sounds the opposite of technological). It was not always thus and maybe I retain something of the spirit of those pioneering days. For example, I am currently typing these words into the HTML pane of WordPress.com. Old habits die hard and WYSIYWG is for softies!

Social media is now mainstream – in fact you could argue that it is real life that has become a minority activity – and things are a lot easier. Although I doggedly insist on still cutting HTML, you can be up and running with a fairly professional-looking blog on WordPress in minutes and without having to know much about any of the technical underpinnings. Software as a Service certainly works really well as an approach to blogging.

Over a number of articles, I am going to touch upon my recent experience of Social Media in the three areas that I first mentioned at the beginning: blogging, micro-blogging and professional networking. Without fully revealing the denouement of this series, I will state now that one of the most interesting things is how well these three areas work in combination and how mutually reinforcing they have become for me. The sequence starts with my thoughts on blogging.
 
 
WordPress and Motivation

WordPress.com

I suppose I have to thank my partner for getting me in to this area as she started her blog long before any of mine. However, having suffered a couple of climbing-related injuries I started my own training blog, both to chart my recovery and to act as a motivational tool.

I started out using Blogger as that was what my partner had used, but got rather frustrated with its lack of support for some basic HTML constructs (e.g. tables). A friend suggested WordPress instead and this became the venue for my training blog. Somewhat amazingly this is not defunct. However, after a period when I religiously posted at least once or twice every week, I haven’t updated it in a long while.

When I wanted to start a professional blog, WordPress seemed the way to go and I have been mostly happy with my choice. But what were my motivations for blogging about business-related issues? I guess that there were a few of these, in no particular order:

  1. I wanted to build upon the public profile that appearing in press articles and speaking at seminars had afforded me.
  2. I like writing and the idea of doing this in a more general context than internal strategy papers and memoranda seemed appealing.
  3. Based on the feedback I had received from my public speaking, I believed that I had quite a lot of relevant experience to draw on which might make interesting reading; at least for a niche audience.
  4. Although it would be fair to say that I started writing mostly for myself, over time the idea of building a blog following seemed like a challenge and I like challenges.
  5. In this same category of emergent motivation, after a short while the notion of establishing a corpus of work, spanning my ideas about a range of issues also became a factor. Maybe some element of Narcissism is present in most blogging.
  6. There was a big slice of simple curiosity about the area, how it worked and how I could be a part of it. You get some interaction in public speaking, but I was intrigued by the idea of getting the benefit of the input of a wider range of people.

So I leapt in with both feet and my first article was based on some reflections on attending a Change Management seminar. It was entitled Business is from Mars and IT is from Venus and dealt with what I see as an artificial divide between IT and business groups. I suppose it makes sense to start as you mean to go on and IT / Business alignment has been a theme running through much of what I have written.
 
 
Things that I have learnt so far

In a subsequent piece, Recipes for success?, I expressed my scepticism about articles of the type “My Top Ten Tips for Successful Blogging”, so the following is not meant to be a set of precepts to be followed to the last letter. Instead, with the benefit of over 60,000 page views (small beer compared to many blogs), here are some things that have worked for me. If some of these chime with your own experience, then great. If others are not pertinent to you, then this is only to be expected.

Finally I should also stress that these observations relate mostly to professional blogs, for personal blogs there are essentially no constraints on your creativity (assuming that the results of this are legal of course).

  1. Write about areas that you know something about. You don’t have to be a world authority, but on a professional blog, no one is going to be that interested in your fevered speculations on something that you know nothing about. This is one of many reasons that you will never see me blogging about IT Infrastructure!
  2. When you blog about an area of personal expertise, then you can be pretty free in expressing your opinions, though [note to self] a dose of humility never did anyone any harm.

    If you know as much as him, then knock yourself out. Else proceed with caution!

    When the subject is one in which your own knowledge is less well-developed (for me something like text analytics would fall into this category), then seek out the opinions of experts in the field and quote these (even if you disagree with them). Linking to the places that experts have expressed their thoughts also expands you network and increases the utility of your blog, which becomes part of a wider world.

  3. It helps if you are interested in the majority of the topics that you cover. If you are unmotivated about something, them why write about it? If you decide to do so for some reason (maybe because you haven’t written anything else this week, or because a piece of news is “hot” at present) then your personal ennui will seep into your words and be evident to your readers. No doubt it will generate similar feelings in them.
  4. Beyond the previous point, I would go further and say that it is crucial that you are truly passionate about at least one thing that you write about and ideally several. Expressing strong opinions is fine, assuming that you have some reason for holding them and that you remain open to the ideas of other people. For me, these areas of passion are Business Intelligence, its intimate connection with Cultural Transformation and the related area of IT / Business Alignment.

    Passion is not only important because it will hopefully infuse your words, but because it will sustain you returning to write about these areas over a long period of time. There are an awful lot of blogs out there where a bright beginning has petered out because the author had nothing left to say, or has lost interest.

  5. For the same reasons relating to sustaining your blog, I would recommend being yourself. If you really want to present an alternative personality to the world, then good luck to you (and your therapist), you will have to possess enormous perseverance and be a very talented actor.

    Not an ideal way to write your blog

    For me this means the presence of strong elliptical and eclectic qualities to my articles. I can do terse and to the point when it is necessary, but circumlocution is more my stock-in-trade. I’m more comfortable being myself and if this means my audience is one composed of people yearning for elliptic, eclectic, circumlocutory writing, then so be it!

  6. To me being yourself extends to the quantity of your writing. In an era sometimes characterised as one of short attention spans and instant gratification, the orthodox advice is to be punchy and direct. Sometimes the point I want to make in one of my articles (assuming that I can remember what this is by the time I get to the end of writing it) takes some time to develop – like a fine wine I like to think (or a mould the less kind might add).

    Not my target audience

    This means that my writing tends to resemble the River Amazon in both its meandering nature and length. I appreciate that this narrows my potential audience, but hope that it also means that at least a few people get some more out of it than they would from the CliffsNotes version.

  7. Blogging should also be about interaction. If you simply want to broadcast your incredibly wise thoughts, then write a book. I hope that some of the pieces that I write spur others to record their own thoughts, either as comments here, or in their own blog articles. If some of my ideas make it into other people’s PowerPoint decks or project proposals, then I am honoured.

    Equally, virtually everything that I write has been inspired to some degree by other people: co-workers, authors, the people that I come into contact with on the Internet and in real life on a daily basis and so on. I try to explicitly acknowledge (and link to) what has inspired me when I write, but I am sure that thousands of unconscious influencers go un-credited.

  8. While passion and having opinions contribute to developing your own voice, it is important to never think that you have all the answers. In a blogging context this means treating anyone who has taken the time to comment on your writing with the respect that this act deserves. While starting a conversation is clearly the best outcome of someone commenting on your blog, a simple ‘thank you’ from the author should be the very least that you can offer (when people whinge about the England cricket team having cheated their way to victory, this is an obvious exception to the rule).
    What do you want me to do? LEAVE? Then they'll keep being wrong!

    © xkcd.com

    In this area I also try to avoid deleting comments that are derogatory about my ideas. The approach I take is rather to either seek further clarification on why the contributor thinks this way, or to politely argue why I still believe that the points that I have made are valid. Of course I have not always 100% lived up to this aspiration!

  9. As in virtually every aspect of life, treating others as you would like to be treated yourself is not a bad approach. If you enjoy people commenting on your articles or linking to your blog, then maybe proactively doing these things yourself is a good idea. I don’t mean adding comments purely for the sake of it; that sounds awfully like spam. But if you read something that you find interesting, then thank the author.

    Better still, augment what they have written with your own ideas – either on their blog or in a piece on your own site that links back to their article. Even in this day and age, it is amazing how far being nice to people can get you. For the same reason, try to be as polite on-line as you would be in your more traditional professional life.

  10. [Yes I am aware of the irony of having ten bullet points here!]

    Finally, I mentioned the Narcissistic tendencies that can either be a cause or effect of blogging. I think that trying to not take yourself too seriously is a must as an antidote to this. Both the medium and my prose can veer towards the preachy sometimes, so some well-placed self-deprecation to balance this never goes amiss.

I hope that some readers will have been interested in my observations and that they will have helped a further subset of these in their blogging. For those who are pondering whether to join the blogosphre, my simple advice is give it a go. You will either hate it or love it, but at least you won’t die wondering “what if?”
 


 
The New Adventures in Wi-Fi series of articles on Social Media continues by discussing the relatively new world of micro-blogging and the phenomenon that is Twitter here.
 

tweet this Tweet this article on twitter.com
Bookmark this article with:
| Facebook | del.icio.us | digg | Reddit | Stumble

 


Especially for all Business Analytics professionals out there

22 July 2009

Last week I was being interviewed by a journalist about Business Analytics amongst other things. I found myself speaking about the perils faced in extrapolation that are significantly less scary when merely interpolating.

Serendipity had led to the following cartoon appearing on the web-site of that doyen of scientific humour Randall Munroe, namely xkcd.com.

By the third trimester, there will be hundreds of babies inside you.

A lesson for us all - © xkcd.com

I’m sure this drawing must have appeared on some other BI blogs, but what the hell, it merits posting again in my opinion.
 

tweet this Tweet this article on twitter.com
Bookmark this article with:
| Facebook | del.icio.us | digg | Reddit | Stumble

 


Is the time ripe for appointing a Chief Business Intelligence Officer?

22 July 2009
linkedin Business Intelligence Business Intelligence

Once more I have decided to pen this article based on a question that was raised on LinkedIn.com. The group in question on this occasion was Business Intelligence and the thread was entitled Is it time that the CBIO (Chief Business Intelligence Officer) position and organization become commonplace in today’s corporate structure? This was posted by John Thielman.

Standard note: You need to be a member of both LinkedIn.com and the group mentioned to view the discussions.
 
 
The case for a CBIO

The Office of the CBIO

I won’t republish all of John’s initial post, but for those who cannot access the thread these are the essential points that he raised:

  1. There is an ever-increasing need for more and better information in organisations
  2. Increasingly Business Intelligence is seen as a major source of competitive advantage
  3. A CBIO would bring focus and (more importantly) accountability to this area
  4. The CBIO should report directly to the CEO, with strong relations with the rest of the executive team
  5. The CBIO’s team would be a hybrid business / technical one (as I strongly believe the best BI teams should be)
  6. This team should also be at the forefront of driving change, based on the metrics that it generates

Now obviously creating a senior role with a portfolio spanning BI and change is going to be music that falls sweetly on my ears. I did however attempt to be objective in my response, which I reproduce in full below:

As someone who is (primarily) a BI professional, then of course my response could be viewed as entirely self-serving. Nevertheless, I’ll offer my thoughts.

In the BI programmes that I have run, I have had reporting lines into people such as the CIO, CFO or sometimes a combined IT / Operations lead. However (and I think that this is a big however), I have always had programme accountability to the CEO and have always had the entire senior leadership team (business and service departments) as my stakeholders. Generally my direction has come more from these dotted lines than from the solid ones – as you would hope would be the case in any customer-centric IT area.

I have run lots of different IT projects over the years. Things such as: building accounting, purchasing and sales systems; configuring and implementing ERP systems; building front-end systems for underwriters, marketing and executive teams; and so on. Given this background, there is definitely something about BI that makes it different.

Any IT system must be aligned to its users’ needs, that much is obvious. However with BI it goes a long way beyond alignment. In a very real sense, BI systems need to be the business. They are not there to facilitate business transactions, they are there to monitor the heartbeat of the organisation, to help it navigate the best way forward, to get early warning of problems, to check the efficacy of strategies and provide key input to developing them.

In short a good BI system should be focussed on precisely the things that the senior leadership team is focussed on, and in particular what the CEO is focussed on. In order to achieve this you need to understand what makes the business tick and you need to move very close to it. This proximity, coupled with the fact that good BI should drip business value means that I have often felt closer to the overall business leadership team than the IT team.

Please don’t misunderstand my point here. I have been an IT person for 20 years and I am not saying that BI should not be fully integrated with the overall IT strategy – indeed in my book it should be central to it as a major function of all IT systems is to gather information (as well as to support transactions and facilitate interactions with customers). However, there is something of a sense in which BI straddles the IT and business arenas (arenas that I have long argued should be much less distinct from each other than they are in many organisations).

The potentially massive impact of BI, the fact that it speaks the language of business leaders, the need for it to be aligned with driving cultural change and that the fact that the skills required for success in BI are slightly different for those necessary in normal IT projects all argue that something like a CBIO position is maybe not such a bad idea.

Indeed I have begun to see quite a few BI roles that are part of change directorates, or the office of the CEO or CFO. There are also some stand-alone BI roles out there, reporting directing to the board. Clearly there will always be a strong interaction with IT, but perhaps you have detected an emerging trend.

I suppose a shorter version of the above would run something like: my de facto reporting line in BI programmes has always been into the CEO and senior management team, so why not recognise this by making it a de jura reporting line.

BI is a weird combination of being both a specialist and generalist area. Generalist in needing to play a major role in running all aspects of the business, specialist in the techniques and technologies that are key to achieving this.
 
 
Over to the jury

Maybe the idea of a CBIO is one whose time has come. I would be interested in people’s views on this.
 

 

tweet this Tweet this article on twitter.com
Bookmark this article with:
Technorati | del.icio.us | digg | Reddit | Stumble

 


Accuracy

20 July 2009

Micropipette

As might be inferred from my last post, certain sporting matters have been on my mind of late. However, as is becoming rather a theme on this blog, these have also generated some business-related thoughts.
 
 
Introduction

On Friday evening, the Australian cricket team finished the second day of the second Test Match on a score of 152 runs for the loss of 8 (out of 10) first innings wickets. This was still 269 runs behind the England team‘s total of 425.

In scanning what I realise must have been a hastily assembled end-of-day report on the web-site of one of the UK’s leading quality newspapers, a couple are glaring errors stood out. First, the Australian number 4 batsman Michael Hussey was described as having “played-on” to a delivery from England’s shy-and-retiring Andrew Flintoff. Second, the journalist wrote that Australia’s number six batsman, Marcus North, had been “clean-bowled” by James Anderson.

I appreciate that not all readers of this blog will be cricket aficionados and also that the mysteries of this most complex of games are unlikely to be made plain by a few brief words from me. However, “played on” means that the ball has hit the batsman’s bat and deflected to break his wicket (or her wicket – as I feel I should mention as a staunch supporter of the all-conquering England Women’s team, a group that I ended up meeting at a motorway service station just recently).

By contrast, “clean-bowled” means that the ball broke the batsman’s wicket without hitting anything else. If you are interested in learning more about the arcane rules of cricket (and let’s face it, how could you not be interested) then I suggest taking a quick look here. The reason for me bothering to go into this level of detail is that, having watched the two dismissals live myself, I immediately thought that the journalist was wrong in both cases.

It may be argued that the camera sometimes lies, but the cricinfo.com caption (whence these images are drawn) hardly ever does. The following two photographs show what actually happened:

Michael Hussey leaves one and is bowled, England v Australia, 2nd Test, Lord's, 2nd day, July 17, 2009

Michael Hussey leaves one and is bowled, England v Australia, 2nd Test, Lord's, 2nd day, July 17, 2009

Marcus North drags James Anderson into his stumps, England v Australia, 2nd Test, Lord's, 2nd day, July 17, 2009

Marcus North drags James Anderson into his stumps, England v Australia, 2nd Test, Lord's, 2nd day, July 17, 2009

As hopefully many readers will be able to ascertain, Hussey raised his bat aloft, a defensive technique employed to avoid edging the ball to surrounding fielders, but misjudged its direction. It would be hard to “play on” from a position such as he adopted. The ball arced in towards him and clipped the top of his wicket. So, in fact he was the one who was “clean-bowled”; a dismissal that was qualified by him having not attempted to play a stroke.

North on the other hand had been at the wicket for some time and had already faced 13 balls without scoring. Perhaps in frustration at this, he played an overly-ambitious attacking shot (one not a million miles from a baseball swing), the ball hit the under-edge of his horizontal bat and deflected down into his wicket. So it was North, not Hussey, who “played on” on this occasion.

So, aside from saying that Hussey had been adjudged out “handled the ball” and North dismissed “obstructed the field” (two of the ten ways in which a batsman’s innings can end – see here for a full explanation), the journalist in question could not have been more wrong.

As I said, the piece was no doubt composed quickly in order to “go to press” shortly after play had stopped for the day. Maybe these are minor slips, but surely the core competency of a sports journalist is to record what happened accurately. If they can bring insights and colour to their writing, so much the better, but at a minimum they should be able to provide a correct description of events.

Everyone makes mistakes. Most of my blog articles contain at least one typographical or grammatical error. Some of them may include errors of fact, though I do my best to avoid these. Where I offer my opinions, it is possible that some of these may be erroneous, or that they may not apply in different situations. However, we tend to expect professionals in certain fields to be held to a higher standard.

Auditors

For a molecular biologist, the difference between a 0.20 micro-molar solution and a 0.19 one may be massive. For a team of experimental physicists, unbelievably small quantities may mean the difference between confirming the existence of the Higgs Boson and just some background noise.

In business, it would be unfortunate (to say the least) if auditors overlooked major assets or liabilities. One would expect that law-enforcement agents did not perjure themselves in court. Equally politicians should never dissemble, prevaricate or mislead. OK, maybe I am a little off track with the last one. But surely it is not unreasonable to expect that a cricket journalist should accurately record how a batsman got out.
 
 
Twitter and Truth

twitter.com

I made something of a leap from these sporting events to the more tragic news of Michael Jackson’s recent demise. I recall first “hearing” rumours of this on twitter.com. At this point, no news sites had much to say about the matter. As the evening progressed, the self-styled celebrity gossip site TMZ was the first to announce Jackson’s death. Other news outlets either said “Jackson taken to hospital” or (perhaps hedging their bets) “US web-site reports Jackson dead”.

By this time the twitterverse was experiencing a cosmic storm of tweets about the “fact” of Jackson’s passing. A comparably large number of comments lamented how slow “old media” was to acknowledge this “fact”. Eventually of course the dinosaurs of traditional news and reporting lumbered to the same conclusion as the more agile mammals of Twitter.

In this case social media was proved to be both quick and accurate, so why am I now going to offer a defence of the world’s news organisations? Well I’ll start with a passage from one of my all-time favourite satires, Yes Minister, together with its sequel Yes Prime Minister.

In the following brief excerpt Sir Geoffrey Hastings (the head of MI5, the British domestic intelligence service) is speaking to The Right Honourable James Hacker (the British Prime Minister). Their topic of conversation is the recently revealed news that a senior British Civil Servant had in fact been a Russian spy:

Yes Prime Minister

Hastings: Things might get out. We don’t want any more irresponsible ill-informed press speculation.
Hacker: Even if it’s accurate?
Hastings: Especially if it’s accurate. There is nothing worse than accurate irresponsible ill-informed press speculation.

Yes Prime Minister, Vol. I by J. Lynn and A. Jay

Was the twitter noise about Jackson’s death simply accurate ill-informed speculation? It is difficult to ask this question as, sadly, the tweets (and TMZ) proved to be correct. However, before we garland new media with too many wreaths, it is perhaps salutary to recall that there was a second rumour of a celebrity death circulating in the febrile atmosphere of Twitter on that day. As far as I am aware, Pittsburgh’s finest – Jeff Goldblum – is alive and well as we speak. Rumours of his death (in an accident on a New Zealand movie set) proved to be greatly exaggerated.

The difference between a reputable news outlet and hordes of twitterers is that the former has a reputation to defend. While the average tweep will simply shrug their shoulders at RTing what they later learn is inaccurate information, misrepresenting the facts is a cardinal sin for the best news organisations. Indeed reputation is the main thing that news outlets have going for them. This inevitably includes annoying and time-consuming things such as checking facts and validating sources before you publish.

With due respect to Mr Jackson, an even more tragic set of events also sparked some similar discussions; the aftermath of the Iranian election. The Economist published an interesting artilce comparing old and new media responses to this entitiled: Twitter 1, CNN 0. Their final comments on this area were:

[...]the much-ballyhooed Twitter swiftly degraded into pointlessness. By deluging threads like Iranelection with cries of support for the protesters, Americans and Britons rendered the site almost useless as a source of information—something that Iran’s government had tried and failed to do. Even at its best the site gave a partial, one-sided view of events. Both Twitter and YouTube are hobbled as sources of news by their clumsy search engines.

Much more impressive were the desk-bound bloggers. Nico Pitney of the Huffington Post, Andrew Sullivan of the Atlantic and Robert Mackey of the New York Times waded into a morass of information and pulled out the most useful bits. Their websites turned into a mish-mash of tweets, psephological studies, videos and links to newspaper and television reports. It was not pretty, and some of it turned out to be inaccurate. But it was by far the most comprehensive coverage available in English. The winner of the Iranian protests was neither old media nor new media, but a hybrid of the two.

Aside from the IT person in me noticing the opportunity to increase the value of Twitter via improved text analytics (see my earlier article, Literary calculus?), these types of issues raise concerns in my mind. To balance this slightly negative perspective it is worth noting that both accurate and informed tweets have preceded several business events, notably the recent closure of BI start-up LucidEra.

Also main stream media seem to have swallowed the line that Google has developed its own operating system in Chrome OS (rather than lashing the pre-existing Linux kernel on to its browser); maybe it just makes a better story. Blogs and Twitter were far more incisive in their commentary about this development.

Considering the pros and cons, on balance the author remains something of a doubting Thomas (by name as well as nature) about placing too much reliance on Twitter for news; at least as yet.
 
 
Accuracy an Business Intelligence

A balancing act

Some business thoughts leaked into the final paragraph of the Introduction above, but I am interested more in the concept of accuracy as it pertains to one of my core areas of competence – business intelligence. Here there are different views expressed. Some authorities feel that the most important thing in BI is to be quick with information that is good-enough; the time taken to achieve undue precision being the enemy of crisp decision-making. Others insist that small changes can tip finely-balanced decisions one way or another and so precision is paramount. In a way that is undoubtedly familiar to regular readers, I straddle these two opinions. With my dislike for hard-and-fast recipes for success, I feel that circumstances should generally dictate the approach.

There are of course different types of accuracy. There is that which insists that business information reflects actual business events (often more a case for work in front-end business systems rather than BI). There is also that which dictates that BI systems reconcile to the penny to perhaps less functional, but pre-existing scorecards (e.g. the financial results of an organisation).

A number of things can impact accuracy, including, but not limited to: how data has been entered into systems; how that data is transformed by interfaces; differences between terminology and calculation methods in different data sources; misunderstandings by IT people about the meaning of business data; errors in the extract transform and load logic that builds BI solutions; and sometimes even the decisions about how information is portrayed in BI tools themselves. I cover some of these in my previous piece Using BI to drive improvements in data quality.

However, one thing that I think differentiates enterprise BI from departmental BI (or indeed predictive models or other types of analytics), is a greater emphasis on accuracy. If enterprise BI is to aspire to becoming the single version of the truth for an organisation, then much more emphasis needs to be placed on accuracy. For information that is intended to be the yardstick by which a business is measured, good enough may fall short of the mark. This is particularly the case where a series of good enough solutions are merged together; the whole may be even less than the sum of its parts.

A focus on accuracy in BI also achieves something else. It stresses an aspiration to excellence in the BI team. Such aspirations tend to be positive for groups of people in business, just as they are for sporting teams. Not everyone who dreams of winning an Olympic gold medal will do so, but trying to make such dreams a reality generally leads to improved performance. If the central goal of BI is to improve corporate performance, then raising the bar for the BI team’s own performance is a great place to start and aiming for accuracy is a great way to move forward.
 


 
A final thought: England went on to beat Australia by precisely 115 runs in the second Test at Lord’s; the final result coming today at precisely 12:42 pm British Summer Time. The accuracy of England’s bowling was a major factor. Maybe there is something to learn here.
 

tweet this Tweet this article on twitter.com
Bookmark this article with:
Technorati | del.icio.us | digg | Reddit | Stumble

 


An update of the most read articles on this site

1 July 2009

Back in April, I posted My “all-time” most-read 5 articles and mentioned that I would update the list from time to time. At the half-year point of 2009, it seemed appropriate to revisit this area.

I have done two things with the new statistics. First, given the number of articles that I have published, I have expanded the list to 20 articles. Second, to give a different perspective, I have added a run-down of those articles that have received the most views per day.

Of course the first list is more likely to contain older articles (which have had more time to accumulate views), whereas the second list is more likely to include new articles (given that most articles have peak viewing figures soon after they are posted). Despite the vagaries of both approaches, it is probably safe to say that if an article appears on both lists it has been pretty popular. I have highlighted 12 such posts in yellow below.

In future, I may consider calculating how well an article has performed against the typical ageing profile. This would address the shortcomings of both of the current tables and therby offer a more definitive benchmark. However this will be dependent on WordPress.com making it a bit easier to download information about page views.

As before I have focussed just on articles, so views of pages about my career or other background information have been omitted from the following.
 

Most viewed pages
Article Views
1 Measuring the benefits of Business Intelligence 2,054
2 Business is from Mars and IT is from Venus 1,497
3 Trends in Business Intelligence 1,324
4 Business Analytics vs Business Intelligence 1,320
5 A review of “The History of Business Intelligence” by Nic Smith 1,201
6 Mountain Biking and Systems Integration 1,129
7 “Why Business Intelligence projects fail 1,032
8 Mergers and value 929
9 Is outsourcing business intelligence a good idea? 909
10 The Top Business Issues facing CIOs / IT Directors – Results 865
11 “Gartner sees a big discrepancy between BI expectations and realities” – Intelligent Enterprise 787
12 Pigeonholing – A tragedy 775
13 “All that glisters is not gold” – some thoughts on dashboards 732
14 Two pictures paint a thousand words… 729
15 BI implementations are like icebergs 720
16 Vision vs Pragmatism 716
17 The specific benefits of business intelligence in insurance 689
18 Holistic vs Incremental approaches to BI 689
19 Perseverance 624
20 A single version of the truth? 596

 
 

Most views / day (qualification: 300 views)
Article Views / day
1 A single version of the truth? 99.3
2 “Why Business Intelligence projects fail 38.2
3 “Involving users in business intelligence strategy key for success” – Christina Torode on SearchCio-Midmarket.com 33.0
4 Data – Information – Knowledge – Wisdom 29.5
5 Literary calculus? 24.3
6 Mountain Biking and Systems Integration 22.6
7 Measuring the benefits of Business Intelligence 16.4
8 Business Analytics vs Business Intelligence 13.9
9 A review of “The History of Business Intelligence” by Nic Smith 12.9
10 Mergers and value 12.9
11 Trends in Business Intelligence 11.6
12 Two pictures paint a thousand words… 11.6
13 Business Intelligence Competency Centres 10.5
14 Using multiple business intelligence tools in an implementation – Part I 10.2
15 Maureen Clarry stresses the need for change skills in business intelligence on BeyeNetwork 9.1
16 The importance of feasibility studies in business intelligence 9.0
17 Pigeonholing – A tragedy 8.5
18 Is outsourcing business intelligence a good idea? 8.3
19 The Top Business Issues facing CIOs / IT Directors – Results 8.2
20 The Dictatorship of the Analysts 7.1

 

tweet this Tweet this article on twitter.com
Bookmark this article with:
Technorati | del.icio.us | digg | Reddit | Stumble

 


A recording of me being interviewed by Brian Roger of SmartDataCollective.com

30 June 2009

SmartDataCollective.com

I have been a featured blogger on SmartDataCollective.com almost as long as I have been a blogger. SDC.com is Social Media Today’s community site, focussed on all aspects of Business Intelligence, Data Warehousing and Analytics, with a pinch of social media thrown in to the mix.

Brian Roger, the SDC.com editor, was recently kind enough to interview me about my career in BI, the challenges I have faced and what has helped to overcome these. This interview is now available to listen to as part of their Podcast series – click on the image below to visit their site.

sdc-podcast

SmartDataCollective.com Intervew

I would be interested in feedback about any aspect of this piece, which I am grateful to Brian for arranging.
 


 
Social Media Today LLC helps global organizations create purpose-built B2B social communities designed to achieve specific, measurable corporate goals by engaging exactly the customers and prospects they most want to reach. Social Media Today helps large companies leverage the enormous power of social media to build deeper relationships with potential customers and other constituencies that influence the development of new business. They have found that their primary metrics of success are levels of engagement and business leads. One thousand people who come regularly and might buy an SAP, Oracle or Teradata system some day is better than a million people who definitely won’t.

Social Media Today LLC, is a battle-tested, nimble team of former journalists, online managers, and advertising professionals who have come together to make a new kind of media company. With their backgrounds, and passions for, business-to-business and public policy conversations, they have decided to focus their efforts in this area. To facilitate the types of convresations that they would like to see Social Media Today is assembling the world’s best bloggers and providing them with an independent “playground” to include their posts, to comment and rate posts, and to connect with each other. On their flagship site, SocialMediaToday.com, they have brought together many of the most intriguing and original bloggers on media and marketing, covering all aspects of what makes up the connective tissue of social media from a global perspective.
 

tweet this Tweet this article on twitter.com
Bookmark this article with:
Technorati | del.icio.us | digg | Reddit | Stumble

A blast from the past…

25 June 2009

Gutenberg Printing Press

I recently came across a record of my first ever foray into the world of Business Intelligence, which dates back to 1997. This was when I was still predominantly an ERP person and was considering options for getting information out of such systems. The piece in question is a brief memo to my boss (the CFO of the organisation I was working at then) about what I described as the OLAP market.

This was a time of innocence, when Google had not even been incorporated, when no one yet owned an iPod and when, if you tried to talk to someone about social media, they would have assumed that you meant friendly journalists. All this is attested to by the fact that this was a paper memo that was printed and circulated in the internal mail – remember that sort of thing?

Given that the document has just had its twelfth birthday, I don’t think I am breeching any confidences in publishing it, though I have removed the names of the recipients for obvious reasons.
 

INTERNAL MEMORANDUM

To: European CFO
cc: Various interested parties
From: Peter Thomas
Date: 16th June 1997
Subject: What is OLAP?

 
On-Line Analytical Processing (OLAP) is a category of software technology that enables analysts, managers and executives to gain insight into data. This is achieved by providing fast, consistent, interactive access to a wide variety of possible views of information. This has generally been transformed from raw data to reflect the real dimensionality of the enterprise.

There are around 30 vendors claiming to offer OLAP products. A helpful report by Business Intelligence[1] (an independent research company) estimates the market share of these . As many of these companies sell other products, the following cannot be viewed as 100% accurate. However the figures do provide some interesting reading.
 

Vendor

1996

1995

Market Position

Share (%)

Market Position

Share (%)

Oracle

1

19.0%

1

20.0%

Hyperion Software

2

18.0%

2

19.0%

Comshare

3

12.0%

3

16.0%

Cognos

4

9.0%

4

5.0%

Arbor Software

5

4.8%

7

2.9%

Holistic Systems

6

4.3%

6

4.7%

Pilot Software

7

4.0%

5

4.8%

MircoStrategy

8

3.5%

9

2.1%

Planning Sciences

9

2.6%

8

2.3%

Information Advantage

10

1.8%

10

1.4%

 
In this group, some companies (Hyperion, Comshare, Holistic, Pilot Software and Planning Services) provide either complete products or extensive toolkits. In contrast some vendors (such as Arbor and – outside the top ten – Applix) only sell specialist multi-dimensional databases. Others (e.g. Cognos and – outside the top ten – BusinessObjects and Brio Technology), offer client based OLAP tools which are basically sophisticated report writers. The final group (including MicroStrategy and Information Advantage) offer a mixed relational / dimensional approach called Relational OLAP or ROLAP.

If we restrict ourselves to the “one-stop solution” vendors in the above list, it is helpful to consider the relative financial position of the top three.
 

Vendor

Market Cap.[2]

($m)

Turnover

($m)

Profit

($m)

Oracle

32,405

4,223[3]

603

Hyperion Software

335

173[4]

9

Comshare

132

119[5]

(9)

 

[1] The OLAP Report by Nigel Pendse and Richard Creeth © Business Intelligence 1997
[2] As at June 1997
[3] 12 months to March 1997
[4] 12 months to June 1996
[5] 12 months to June 1996

 


 
It is of course also worth pointing out that I used to disagree with what Nigel Pendse wrote a lot less back then!
 

tweet this Tweet this article on twitter.com
Bookmark this article with:
Technorati | del.icio.us | digg | Reddit | Stumble

 


Follow

Get every new post delivered to your Inbox.

Join 3,427 other followers

%d bloggers like this: