A bad workman blames his [Business Intelligence] tools

29 July 2009

Tools
 
Introduction

This is a proverb with quite some history to it. Indeed its lineage has been traced to 13th Century France in: mauvés ovriers ne trovera ja bon hostill (les mauvais ouvriers ne trouveront jamais un bon outil being a rendition in more contemporary French). To me this timeless observation is applicable to present-day Business Intelligence projects. Browsing through on-line forums, it is all too typical to see discussions that start “What is the best BI software available on the market?”, “Who are the leaders in SaaS BI?” and (rather poignantly in my opinion) “Please help me to pick the best technology for a dashboard.” I feel that these are all rather missing the point. Before I explain why, I am going to offer another of my sporting analogies, which I believe is pertinent. Indeed sporting performace is an area to which the aphorism appearing in the title is frequently applied.

If you would like to skip the sporting analogy and cut to the chase, please click here.
 
 
The importance of having the right shoes

Rock climbing is a sport that certainly has its share of machismo; any climbing magazine or web-site will feature images of testosterone-infused youths whose improbable physiques (often displayed to full advantage by the de rigueur absence of any torso-encumbering clothing) propel them to the top of equally improbable climbs.

Given this, many commentators have noted the irony of climbing being conducted by people wearing the equivalent of rubber-covered ballet slippers. The fact that one of the most iconic rock climbing shoes of all time was a fetching shade of pink merely adds piquancy to this observation. Examples of these, the classic FiveTen Anasazi Lace-ups, are featured in the following photo of top British climber, Steve McClure (yes it is the right way up).

The UK's finest sport climer - Steve McClure - sports the Pink'uns

When I started rock climbing, my first pair of shoes were Zephyrs from Spanish climbing firm Boreal. They looked something like this:

The Zephyr by Boreal - $87 - £67

The Zephyr by Boreal - $87 - £67

Although it might not be apparent from the above image, these are intended to be comfortable shoes. Ones to be worn by more experienced climbers on long mountain days, or suitable for beginners, like myself at the time, on shorter climbs. Although not exactly cheap, they are not prohibitively expensive and the rubber on the soles is quite hard-wearing as well.

The Zephyrs worked well for me, but inevitably over time you begin to notice the shoes worn by better climbers at the crag or at the wall. You also cannot fail to miss the much sexier shoes worn by professional climbers in films, climbing magazine articles and (no coincidence here) advertisements. These other shoes also cost more (again no coincidence) and promise better performance. When you are looking to get better at something, it is tempting to take any advantage that you can get. Also, perhaps especially when you are looking to break into a new area, there is some pressure to conform, to look like the “in-crowd”, maybe even simply to distance yourself from the beginner that you were only a few months previously.

This is very shallow behaviour of course, but it is also the rock on which the advertising industry is founded. I wanted to get better as a climber, but would have to admit that other, less noble, motives also drove me to wanting to purchase new rock shoes.

The Galileo by FiveTen - $130 - £85

The Galileo by FiveTen - $130 - £85

The Galileos shown above are made by US company FiveTen and are representative of the type of shoes that I have worn for most my climbing career. FiveTen shoes have been worn by many top climbers over the years (though there have recently been some quite high-profile defections to start-up brand Evolv, who can never seem to decide whether to append a final ‘e’ to their name or not).

Amongst other things, FiveTens are noted for the stickiness of their rubber, which is provided by an organisation called Stealth Rubber and appears on no other rock climbing shoes. Generally the greater the adhesion between your foot and the rock, the greater the force that you can bring to bear on it to drive yourself upwards. Also it helps to have confidence that your foot has a good chance of staying in place, no matter how glassy the rock may be (and no matter how long the fall may be should this not happen). I have worn FiveTen shoes on all of my hardest climbs (none of which have actually been very hard in the grand scheme of things sad to say).

The Solution by La Sportiva - $155 - £120 (link goes to the Sportiva site)

The Solution by La Sportiva - $155 - £120
(the link loads a Flash page on the Sportiva site)

Nevertheless, with what I admit was rather a sense of guilt, I have recently embarked on a dalliance with another rock shoe manufacturer, La Sportiva of Italy. The Sportiva Solutions which are shown above are both the most expensive rock shoes I have ever owned and the most technical. If NASA made a rock shoe, they would probably not be a million miles away from the Solutions. The radical nature of their design can perhaps best be appreciated in three dimensions and you can do this by clicking on the above image.

The Solutions are very, very good rock shoes. I recently had the opportunity to carry out a before and after comparison on the following climb, A Miller’s Tale:

A Miller's Tale (V4/Font 6b+) - Rubicon Wall, Derbyshire, England

A Miller's Tale (V4/Font 6b+)
Rubicon Wall, Derbyshire, England
© http://77jenn.blogspot.com

My partner, who appears in the photo (incidentally sporting FiveTen shoes), climbed this on her second go. By contrast, I had many fruitless attempts wearing my own pair of FiveTens (that, to be fair to FiveTen, were much less technical than the Galileo’s above and were also probably past the end of their useful life). I frequently found my feet skittering off of the highly polished limestone, which resulted in me rapidly returning to terra firma.

A couple of weeks later, equipped with my shiny new Sportivas, my feet did not slip once. Of course the perfect end to this story would have been to say that I then climbed the problem (for an explanation of why some types of climbs are called problems see my earlier article Perseverance). Sadly, though I made much more progress during my second session, I need to go back to finally tick it off of my list.

So here surely is an example of the tool making a difference, or is it? My partner had climbed A Miller’s Tale quite happily without having the advantage of my new footwear. She is 5’3″ (160cm) compared to my 5’11″ (180cm) and the taller you are the easier it is to reach the next hold. Strength is a factor in climbing and I am also stronger in absolute terms than she is. The reason that she succeeded where I failed is simply that she is a better climber than I am. It is an oft-repeated truism in the climbing world that many females have better techniques than men. This, together with the “unfair” advantage of smaller fingers, is the excuse often offered by muscle-bound men who fail to complete a climb that a female then dances her way up. However in my partner’s case, she is also very strong, with her power-to-weight ratio being the key factor. You don’t need to lift massive weights in climbing, just your own body.

So I didn’t really need better rock shoes to prevent my feet from slipping. If I got my body into a better balanced position, then this would have had the same impact. Equally, if my abdominal muscles were stronger, I could have squeezed my feet harder onto the rock, increasing their adhesion (this type of strength, known as core strength for obvious reasons, is crucial to progressing in many types of climbing). What the Solutions did was not to make me a better climber, but to make up for some of my inadequacies. In this way, by allowing me the luxury of not focussing on increasing my strength or improving my technique, you could even argue that they might be bad for my climbing in the long run. I probably protest too much in this last comment, but hopefully the reader can appreciate the point that I am trying to make.

Campus board training

In order to become a better climber I need to do lots of things. I need to strengthen the tendons in my fingers (or at least in nine of them as I ruptured the tendon in my right ring finger playing rugby years ago) so that I can hold on to smaller edges and grasp larger ones for longer. I need to develop my abdominal muscles to hold me onto the rock face better and put more pressure on my feet; particularly when the climb is overhanging. I need to build up muscles in my back, shoulders and arms to be able to move more assuredly between holds that are widely spaced. I must work on my endurance, so that I do not fail climbs because I am worn out by a long series of lower moves. Finally I need to improve my technique: making my footwork more precise; paying more attention to the shape of my body and how this affects my centre of gravity and the purchase I have on holds; getting more comfortable with the tricks of the trade such as heel- and toe-hooks; learning when to be aggressive in my climbing and when to be slow and deliberate; and finally better visualising how my body fits against the rock and the best way to flow economically from one position to the next.

If I can get better in all of these areas, then maybe I will have earned my new technical rock shoes and I will be able to take advantage of the benefits that they offer. Having the right shoes can undoubtedly improve your climbing, but it is no substitute for focussing on the long list in the previous paragraph. There is no real short-cut to becoming a better climber, it just takes an awful lot of work.

A final thing to add in this section is that the Solutions offer advantages to the climber on certain types of climbs. On any overhanging, pocketed rock, they are brilliant. But the way that they shape your foot into a down-turned claw would be a positive disadvantage when trying to pad up a slab. In this second scenario, something like my worn out FiveTens (now sadly consigned to the rubbish tip) would be the tool of choice. It is important to realise that the right tool is often dictated by the task in hand and one that excels in area A may be an also-ran in area B.

Notes:

  1. Lest it be thought that the above manufacturers play only in narrow niches, I should explain that each of Boreal, FiveTen and La Sportiva produce a wide range of rock shoes catering to virtuially every type of climber from the neophyte to the world’s best.
  2. If you think that the pound dollar rates are rather strange in the above exhibits, then a few things are at play. Some are genuine differences, but others are because they are historical rates. for example, I struggled to find a US web-site that still sells Boreal Zephyrs.
  3. If you are interested in finding out more about my adventures in rock climbing, then take a look at my partner’s blog.

 
 
The role of technology in Business Intelligence

I hope that I have established that at least in the world of rock climbing, the technology that you have at your disposal is only one of many factors necessary for success; indeed it is some way from being the most important factor.

Having really poor, or worn out, rock shoes can dent your confidence and even get you into bad habits (such as not using your feet enough). Having really good rock shoes can bring some incremental benefits, but these are not as great as those to be gained by training and experience. Most of the technologically-related benefits will be realised by having reasonably good and reasonably new shoes.

While the level of a professional rock climber’s performance will be undoubtedly be improved by using the best equipment available, a bad climber with $150 rock shoes will still be a bad climber (note this is not intended to be a self-referential comment).

Requirements - Data Analysis - Information - Manage Change

Requirements - Data Analysis - Information - Manage Change

Returning to another of my passions, Business Intelligence, I see some pertinent parallels. In a series of previous articles (including BI implementations are like icebergs, “All that glisters is not gold” – some thoughts on dashboards and Short-term “Trouble for Big Business Intelligence Vendors” may lead to longer-term advantage) , I have laid out my framework for BI success and explained why I feel that technology is not the most important part of a BI programme.

My recommended approach is based on four pillars:

  1. Determine what information is necessary to drive key business decisions.
  2. Understand the various data sources that are available and how they relate to each other.
  3. Transform the data to meet the information needs.
  4. Manage the embedding of BI in the corporate culture.

Obviously good BI technology has a role to play across all of these areas, but it is not the primary concern in any of them. Let us consider what is often one of the most difficult areas to get right, embedding BI in an organisation’s DNA. What is the role of the BI tool here?

Well if you want people to actually use the BI system, it helps if the way that the BI technology operates is not a hindrance to this. Ideally the ease-of-use and intuitiveness of the BI technology deployed should be a plus point for you. However, if you have the ultimate in BI technology, but your BI system does not highlight areas that business people are interested in, does not provide information that influences actual decision-making, or contains numbers that are inaccurate, out-of-date, or unreconciled, then it will not be used. I put this a little more succinctly in a recent article: Using multiple business intelligence tools in an implementation – Part II (an inspired title I realise), which I finished by saying:

If your systems do not have credibility with your users, then all is already lost and no amount of flashy functionality will save you.

Similar points can be made about all of the other pillars. Great BI technology should be the icing on your BI cake, not one of the main ingredients.
 
 
The historical perspective

What Car?

Ajay Ohri from the DecisionStats web-site recently interviewed me in some depth about a range of issues. He specifically asked me about what differentiated the various BI tools and I reproduce my reply here:

The really important question in BI is not which tool is best, but how to make BI projects successful. While many an unsuccessful BI manager may blame the tool or its vendor, this is not where the real issues lie. I firmly believe that successful BI rests on four mutually reinforcing pillars: understand the questions the business needs to answer, understand the data available, transform the data to meet the business needs and embed the use of BI in the organisation’s culture. If you get these things right then you can be successful with almost any of the excellent BI tools available in the marketplace. If you get any one of them wrong, then using the paragon of BI tools is not going to offer you salvation.

I think about BI tools in the same way as I do the car market. Not so many years ago there were major differences between manufacturers. The Japanese offered ultimate reliability, but maybe didn’t often engage the spirit. The Germans prided themselves on engineering excellence, slanted either in the direction of performance or luxury, but were not quite as dependable as the Japanese. The Italians offered out-and-out romance and theatre, with mechanical integrity an afterthought. The French seemed to think that bizarrely shaped cars with wheels as thin as dinner plates were the way forward, but at least they were distinctive. The Swedes majored on a mixture of safety and aerospace cachet, but sometimes struggled to shift their image of being boring. The Americans were still in the middle of their love affair with the large and the rugged, at the expense of convenience and value-for-money. Stereotypically, my fellow-countrymen majored on agricultural charm, or wooden-panelled nostalgia, but struggled with the demands of electronics.

Nowadays, the quality and reliability of cars are much closer to each other. Most manufacturers have products with similar features and performance and economy ratings. If we take financial issues to one side, differences are more likely to related to design, or how people perceive a brand. Today the quality of a Ford is not far behind that of a Toyota. The styling of a Honda can be as dramatic as an Alfa Romeo. Lexus and Audi are playing in areas previously the preserve of BMW and Mercedes and so on. To me this is also where the market for BI tools is at present. It is relatively mature and the differences between product sets are less than before.

Of course this doesn’t mean that the BI field will not be shaken up by some new technology or approach (in-memory BI or SaaS come to mind). This would be the equivalent of the impact that the first hybrid cars had on the auto market. However, from the point of view of implementations, most BI tools will do at least an adequate job and picking one should not be your primary concern in a BI project.

If you are interested, you can read the full interview here.
 
 
The current reality

IBM to acquire SPSS

As my comments to Ajay suggest, maybe in past times there were greater differences between BI vendors and the tools that they supplied. One benefit of the massive consolidation that has occurred in recent years is that the five biggest players: IBM/Cognos, Oracle/Hyperion, SAP/BusinessObjects, Microsoft and (the as yet still independent) SAS all have product portfolios that are both wide and deep. If there is something that you want your BI tool to do, it is likely that any of these organisations can provide you with the software; assuming that your wallet allows it. Both the functionality and scope of offerings from smaller vendors operating in the BI arena have also increased greatly in recent times. Finding a technology that fits your specific needs for functionality, ease-of-use, scalability and reliability should not be a problem.

This general landscape is one against which it is interesting to view the recent acquisition of business analytics firm SPSS by IBM. According to Reuters, IBM’s motivations are as follows:

IBM plans to buy business analytics company SPSS Inc for $1.2 billion in cash to better compete with Oracle Corp and SAP AG in the growing field of business intelligence

Full story here.

As an aside, should both Microsoft and SAS be worried that they are omitted from this list?

Whatever the corporate logic for IBM, to me this is simply more evidence that BI technology is becoming a utility (it should however be noted that this is not the same as BI itself becoming a utility). I believe that this trend will lead to a greater focus on the use of BI technology as part of broad-based BI programmes that drive business value. Though BI has the potential of releasing massive benefits for organisations, the track record has been somewhat patchy. Hopefully as people start to worry less about BI technology and more about the factors that really drive success in BI programmes, this will begin to change.

A precursor to Business Intelligence

As with any technical innovation over the centuries, it is only when the technology itself becomes invisible that the real benefits flow.
 

tweet this Tweet this article on twitter.com
Bookmark this article with:
| Facebook | del.icio.us | digg | Reddit | Stumble

 

16 Comments | business, business intelligence, businessobjects, cognos, cultural transformation, hyperion, ibm, microsoft, oracle, sap, sas, technology | Tagged: , , , , | Permalink
Posted by Peter Thomas

New Social Media Section

26 July 2009

My Keynote Articles page gathers together links to my more substantial blog posts in one place and groups them into broad areas of interest. When I first created this, the four areas were as follows:

1. Business Intelligence
2. Cultural Transformation
3. IT / Business Alignment and IT Strategy
4. General Articles

Back in April 2009, at the height of the Oracle / Sun fervour, I added a new section to reflect the number of articles I was writing about general technology issues such as these and also BI industry news. This was as follows:

5. Industry Commentary

In a similar vein, I have recently been writing more about issues in Social Media (very self-referential for a blog of course). I have also had my artciles syndicated on SmartDataCollective.com for many months.

SocialMediaToday

The thing that finally decided me to have a section dedicated to Social Media was joining SmartDataCollective’s parent site, SocialMediaToday.com, who were recently kind enough to syndicate the first of my series of articles on Social Media: New Adventures in Wi-Fi – Track 1: Blogging (the original article may be viewed here).

My thinking was, if I am having my articles posted on a site devoted to Social Media, then the time has come to recognise this by creating a new section. This is what I have done with:

6. Social Media

As with the other sections, I will keep this list up-to-date as I add new content. In particular my forthcoming pieces on micro-blogging with Twitter.com and professional networking on LinkedIn.com will have a shiny new home.
 

tweet this Tweet this article on twitter.com
Bookmark this article with:
| Facebook | del.icio.us | digg | Reddit | Stumble

 

2 Comments | blogging, business, site update, social media, technology, twitter | Tagged: , , | Permalink
Posted by Peter Thomas

New Adventures in Wi-Fi – Track 1: Blogging

24 July 2009

New Adventures in Wi-Fi (with apologies to R.E.M.)
 
Introduction

I established this blog back in November 2008 – shortly after this I joined twitter.com in December 2008 – I had already been a member of LinkedIn.com since July 2005. However, my involvement with what is now collectively called social media goes back a lot further than this. Back then we tended to use the phrase on-line communities to describe what we were engaged in.

My first foray into this new world was in 1998/99 when I joined a, now defunct, discussion forum (then known as a Bulletin Board). This was focused on computer games. I wasn’t terribly in to such games at the time, I didn’t own a console and my PC was used for more prosaic purposes. Nevertheless, for reasons that I will not bore the reader with, I signed up. Since then I have been a member of a number of on-line forums, mostly with some sporting element, for example rock climbing.

Yahoo! Geocities

In May 1999, my forum activities led me to creating my first web-site (again now also defunct). I started on Geocities (another chance to use the word “defunct”) and then moved to having my own domain and an agreement with a hosting company. I even ended up jointly running a very successful forum with an on-line friend from Australia. Back then the men were real men, the women were real women and the HTML was real HTML. However this article is not about ancient history, but rather about my more recent experiences in social media.

Nowadays, nobody seems to think of it as being odd that you regularly “speak” to people you have never met and who inhabit countries on the other side of the world. People do not slowly back away from you at parties if you drop the fact that you have your own web-site into a conversation (though maybe one reason that the portmanteau of web-log became socially acceptable is that its abridgement to blog sounds the opposite of technological). It was not always thus and maybe I retain something of the spirit of those pioneering days. For example, I am currently typing these words into the HTML pane of WordPress.com. Old habits die hard and WYSIYWG is for softies!

Social media is now mainstream – in fact you could argue that it is real life that has become a minority activity – and things are a lot easier. Although I doggedly insist on still cutting HTML, you can be up and running with a fairly professional-looking blog on WordPress in minutes and without having to know much about any of the technical underpinnings. Software as a Service certainly works really well as an approach to blogging.

Over a number of articles, I am going to touch upon my recent experience of Social Media in the three areas that I first mentioned at the beginning: blogging, micro-blogging and professional networking. Without fully revealing the denouement of this series, I will state now that one of the most interesting things is how well these three areas work in combination and how mutually reinforcing they have become for me. The sequence starts with my thoughts on blogging.
 
 
WordPress and Motivation

WordPress.com

I suppose I have to thank my partner for getting me in to this area as she started her blog long before any of mine. However, having suffered a couple of climbing-related injuries I started my own training blog, both to chart my recovery and to act as a motivational tool.

I started out using Blogger as that was what my partner had used, but got rather frustrated with its lack of support for some basic HTML constructs (e.g. tables). A friend suggested WordPress instead and this became the venue for my training blog. Somewhat amazingly this is not defunct. However, after a period when I religiously posted at least once or twice every week, I haven’t updated it in a long while.

When I wanted to start a preofessional blog, WordPress seemed the way to go and I have been mostly happy with my choice. But what were my motivations for blogging about business-related issues? I guess that there were a few of these, in no particular order:

  1. I wanted to build upon the public profile that appearing in press articles and speaking at seminars had afforded me.
  2. I like writing and the idea of doing this in a more general context than internal strategy papers and memoranda seemed appealing.
  3. Based on the feedback I had received from my public speaking, I believed that I had quite a lot of relevant experience to draw on which might make interesting reading; at least for a niche audience.
  4. Although it would be fair to say that I started writing mostly for myself, over time the idea of building a blog following seemed like a challenge and I like challenges.
  5. In this same category of emergent motivation, after a short while the notion of establishing a corpus of work, spanning my ideas about a range of issues also became a factor. Maybe some element of Narcissism is present in most blogging.
  6. There was a big slice of simple curiosity about the area, how it worked and how I could be a part of it. You get some interaction in public speaking, but I was intrigued by the idea of getting the benefit of the input of a wider range of people.

So I leapt in with both feet and my first article was based on some reflections on attending a Change Management seminar. It was entitled Business is from Mars and IT is from Venus and dealt with what I see as an artificial divide between IT and business groups. I suppose it makes sense to start as you mean to go on and IT / Business alignment has been a theme running through much of what I have written.
 
 
Things that I have learnt so far

In a subsequent piece, Recipes for success?, I expressed my scepticism about articles of the type “My Top Ten Tips for Successful Blogging”, so the following is not meant to be a set of precepts to be followed to the last letter. Instead, with the benefit of over 60,000 page views (small beer compared to many blogs), here are some things that have worked for me. If some of these chime with your own experience, then great. If others are not pertinent to you, then this is only to be expected.

Finally I should also stress that these observations relate mostly to professional blogs, for personal blogs there are essentially no constraints on your creativity (assuming that the results of this are legal of course).

  1. Write about areas that you know something about. You don’t have to be a world authority, but on a professional blog, no one is going to be that interested in your fevered speculations on something that you know nothing about. This is one of many reasons that you will never see me blogging about IT Infrastructure!
  2. When you blog about an area of personal expertise, then you can be pretty free in expressing your opinions, though [note to self] a dose of humility never did anyone any harm.

    If you know as much as him, then knock yourself out. Else proceed with caution!

    When the subject is one in which your own knowledge is less well-developed (for me something like text analytics would fall into this category), then seek out the opinions of experts in the field and quote these (even if you disagree with them). Linking to the places that experts have expressed their thoughts also expands you network and increases the utility of your blog, which becomes part of a wider world.

  3. It helps if you are interested in the majority of the topics that you cover. If you are unmotivated about something, them why write about it? If you decide to do so for some reason (maybe because you haven’t written anything else this week, or because a piece of news is “hot” at present) then your personal ennui will seep into your words and be evident to your readers. No doubt it will generate similar feelings in them.
  4. Beyond the previous point, I would go further and say that it is crucial that you are truly passionate about at least one thing that you write about and ideally several. Expressing strong opinions is fine, assuming that you have some reason for holding them and that you remain open to the ideas of other people. For me, these areas of passion are Business Intelligence, its intimate connection with Cultural Transformation and the related area of IT / Business Alignment.

    Passion is not only important because it will hopefully infuse your words, but because it will sustain you returning to write about these areas over a long period of time. There are an awful lot of blogs out there where a bright beginning has petered out because the author had nothing left to say, or has lost interest.

  5. For the same reasons relating to sustaining your blog, I would recommend being yourself. If you really want to present an alternative personality to the world, then good luck to you (and your therapist), you will have to possess enormous perseverance and be a very talented actor.

    Not an ideal way to write your blog

    For me this means the presence of strong elliptical and eclectic qualities to my articles. I can do terse and to the point when it is necessary, but circumlocution is more my stock-in-trade. I’m more comfortable being myself and if this means my audience is one composed of people yearning for elliptic, eclectic, circumlocutory writing, then so be it!

  6. To me being yourself extends to the quantity of your writing. In an era sometimes characterised as one of short attention spans and instant gratification, the orthodox advice is to be punchy and direct. Sometimes the point I want to make in one of my articles (assuming that I can remember what this is by the time I get to the end of writing it) takes some time to develop – like a fine wine I like to think (or a mould the less kind might add).

    Not my target audience

    This means that my writing tends to resemble the River Amazon in both its meandering nature and length. I appreciate that this narrows my potential audience, but hope that it also means that at least a few people get some more out of it than they would from the CliffsNotes version.

  7. Blogging should also be about interaction. If you simply want to broadcast your incredibly wise thoughts, then write a book. I hope that some of the pieces that I write spur others to record their own thoughts, either as comments here, or in their own blog articles. If some of my ideas make it into other people’s PowerPoint decks or project proposals, then I am honoured.

    Equally, virtually everything that I write has been inspired to some degree by other people: co-workers, authors, the people that I come into contact with on the Internet and in real life on a daily basis and so on. I try to explicitly acknowledge (and link to) what has inspired me when I write, but I am sure that thousands of unconscious influencers go un-credited.

  8. While passion and having opinions contribute to developing your own voice, it is important to never think that you have all the answers. In a blogging context this means treating anyone who has taken the time to comment on your writing with the respect that this act deserves. While starting a conversation is clearly the best outcome of someone commenting on your blog, a simple ‘thank you’ from the author should be the very least that you can offer (when people whinge about the England cricket team having cheated their way to victory, this is an obvious exception to the rule).
    What do you want me to do? LEAVE? Then they'll keep being wrong!

    © xkcd.com

    In this area I also try to avoid deleting comments that are derogatory about my ideas. The approach I take is rather to either seek further clarification on why the contributor thinks this way, or to politely argue why I still believe that the points that I have made are valid. Of course I have not always 100% lived up to this aspiration!

  9. As in virtually every aspect of life, treating others as you would like to be treated yourself is not a bad approach. If you enjoy people commenting on your articles or linking to your blog, then maybe proactively doing these things yourself is a good idea. I don’t mean adding comments purely for the sake of it; that sounds awfully like spam. But if you read something that you find interesting, then thank the author.

    Better still, augment what they have written with your own ideas – either on their blog or in a piece on your own site that links back to their article. Even in this day and age, it is amazing how far being nice to people can get you. For the same reason, try to be as polite on-line as you would be in your more traditional professional life.

  10. [Yes I am aware of the irony of having ten bullet points here!]

    Finally, I mentioned the Narcissistic tendencies that can either be a cause or effect of blogging. I think that trying to not take yourself too seriously is a must as an antidote to this. Both the medium and my prose can veer towards the preachy sometimes, so some well-placed self-deprecation to balance this never goes amiss.

I hope that some readers will have been interested in my observations and that they will have helped a further subset of these in their blogging. For those who are pondering whether to join the blogosphre, my simple advice is give it a go. You will either hate it or love it, but at least you won’t die wondering “what if?”
 

 
The New Adventures in Wi-Fi series of articles on Social Media continues by discussing the relatively new world of micro-blogging and the phenomenon that is Twitter here.
 

tweet this Tweet this article on twitter.com
Bookmark this article with:
| Facebook | del.icio.us | digg | Reddit | Stumble

 

10 Comments | blogging, business, social media, technology, twitter | Tagged: , , | Permalink
Posted by Peter Thomas

Especially for all Business Analytics professionals out there

22 July 2009

Last week I was being interviewed by a journalist about Business Analytics amongst other things. I found myself speaking about the perils faced in extrapolation that are significantly less scary when merely interpolating.

Serendipity had led to the following cartoon appearing on the web-site of that doyen of scientific humour Randall Munroe, namely xkcd.com.

By the third trimester, there will be hundreds of babies inside you.

A lesson for us all - © xkcd.com

I’m sure this drawing must have appeared on some other BI blogs, but what the hell, it merits posting again in my opinion.
 

tweet this Tweet this article on twitter.com
Bookmark this article with:
| Facebook | del.icio.us | digg | Reddit | Stumble

 

1 Comment | business, business analytics, business intelligence, technology | Tagged: | Permalink
Posted by Peter Thomas

Is the time ripe for appointing a Chief Business Intelligence Officer?

22 July 2009
linkedin Business Intelligence Business Intelligence

Once more I have decided to pen this article based on a question that was raised on LinkedIn.com. The group in question on this occasion was Business Intelligence and the thread was entitled Is it time that the CBIO (Chief Business Intelligence Officer) position and organization become commonplace in today’s corporate structure? This was posted by John Thielman.

Standard note: You need to be a member of both LinkedIn.com and the group mentioned to view the discussions.
 
 
The case for a CBIO

The Office of the CBIO

I won’t republish all of John’s initial post, but for those who cannot access the thread these are the essential points that he raised:

  1. There is an ever-increasing need for more and better information in organisations
  2. Increasingly Business Intelligence is seen as a major source of competitive advantage
  3. A CBIO would bring focus and (more importantly) accountability to this area
  4. The CBIO should report directly to the CEO, with strong relations with the rest of the executive team
  5. The CBIO’s team would be a hybrid business / technical one (as I strongly believe the best BI teams should be)
  6. This team should also be at the forefront of driving change, based on the metrics that it generates

Now obviously creating a senior role with a portfolio spanning BI and change is going to be music that falls sweetly on my ears. I did however attempt to be objective in my response, which I reproduce in full below:

As someone who is (primarily) a BI professional, then of course my response could be viewed as entirely self-serving. Nevertheless, I’ll offer my thoughts.

In the BI programmes that I have run, I have had reporting lines into people such as the CIO, CFO or sometimes a combined IT / Operations lead. However (and I think that this is a big however), I have always had programme accountability to the CEO and have always had the entire senior leadership team (business and service departments) as my stakeholders. Generally my direction has come more from these dotted lines than from the solid ones – as you would hope would be the case in any customer-centric IT area.

I have run lots of different IT projects over the years. Things such as: building accounting, purchasing and sales systems; configuring and implementing ERP systems; building front-end systems for underwriters, marketing and executive teams; and so on. Given this background, there is definitely something about BI that makes it different.

Any IT system must be aligned to its users’ needs, that much is obvious. However with BI it goes a long way beyond alignment. In a very real sense, BI systems need to be the business. They are not there to facilitate business transactions, they are there to monitor the heartbeat of the organisation, to help it navigate the best way forward, to get early warning of problems, to check the efficacy of strategies and provide key input to developing them.

In short a good BI system should be focussed on precisely the things that the senior leadership team is focussed on, and in particular what the CEO is focussed on. In order to achieve this you need to understand what makes the business tick and you need to move very close to it. This proximity, coupled with the fact that good BI should drip business value means that I have often felt closer to the overall business leadership team than the IT team.

Please don’t misunderstand my point here. I have been an IT person for 20 years and I am not saying that BI should not be fully integrated with the overall IT strategy – indeed in my book it should be central to it as a major function of all IT systems is to gather information (as well as to support transactions and facilitate interactions with customers). However, there is something of a sense in which BI straddles the IT and business arenas (arenas that I have long argued should be much less distinct from each other than they are in many organisations).

The potentially massive impact of BI, the fact that it speaks the language of business leaders, the need for it to be aligned with driving cultural change and that the fact that the skills required for success in BI are slightly different for those necessary in normal IT projects all argue that something like a CBIO position is maybe not such a bad idea.

Indeed I have begun to see quite a few BI roles that are part of change directorates, or the office of the CEO or CFO. There are also some stand-alone BI roles out there, reporting directing to the board. Clearly there will always be a strong interaction with IT, but perhaps you have detected an emerging trend.

I suppose a shorter version of the above would run something like: my de facto reporting line in BI programmes has always been into the CEO and senior management team, so why not recognise this by making it a de jura reporting line.

BI is a weird combination of being both a specialist and generalist area. Generalist in needing to play a major role in running all aspects of the business, specialist in the techniques and technologies that are key to achieving this.
 
 
Over to the jury

Maybe the idea of a CBIO is one whose time has come. I would be interested in people’s views on this.
 

 

tweet this Tweet this article on twitter.com
Bookmark this article with:
Technorati | del.icio.us | digg | Reddit | Stumble

 

10 Comments | business, business intelligence, change management, it business alignment, technology | Tagged: , | Permalink
Posted by Peter Thomas

Accuracy

20 July 2009

Micropipette

As might be inferred from my last post, certain sporting matters have been on my mind of late. However, as is becoming rather a theme on this blog, these have also generated some business-related thoughts.
 
 
Introduction

On Friday evening, the Australian cricket team finished the second day of the second Test Match on a score of 152 runs for the loss of 8 (out of 10) first innings wickets. This was still 269 runs behind the England team‘s total of 425.

In scanning what I realise must have been a hastily assembled end-of-day report on the web-site of one of the UK’s leading quality newspapers, a couple are glaring errors stood out. First, the Australian number 4 batsman Michael Hussey was described as having “played-on” to a delivery from England’s shy-and-retiring Andrew Flintoff. Second, the journalist wrote that Australia’s number six batsman, Marcus North, had been “clean-bowled” by James Anderson.

I appreciate that not all readers of this blog will be cricket aficionados and also that the mysteries of this most complex of games are unlikely to be made plain by a few brief words from me. However, “played on” means that the ball has hit the batsman’s bat and deflected to break his wicket (or her wicket – as I feel I should mention as a staunch supporter of the all-conquering England Women’s team, a group that I ended up meeting at a motorway service station just recently).

By contrast, “clean-bowled” means that the ball broke the batsman’s wicket without hitting anything else. If you are interested in learning more about the arcane rules of cricket (and let’s face it, how could you not be interested) then I suggest taking a quick look here. The reason for me bothering to go into this level of detail is that, having watched the two dismissals live myself, I immediately thought that the journalist was wrong in both cases.

It may be argued that the camera sometimes lies, but the cricinfo.com caption (whence these images are drawn) hardly ever does. The following two photographs show what actually happened:

Michael Hussey leaves one and is bowled, England v Australia, 2nd Test, Lord's, 2nd day, July 17, 2009

Michael Hussey leaves one and is bowled, England v Australia, 2nd Test, Lord's, 2nd day, July 17, 2009
Marcus North drags James Anderson into his stumps, England v Australia, 2nd Test, Lord's, 2nd day, July 17, 2009

Marcus North drags James Anderson into his stumps, England v Australia, 2nd Test, Lord's, 2nd day, July 17, 2009

As hopefully many readers will be able to ascertain, Hussey raised his bat aloft, a defensive technique employed to avoid edging the ball to surrounding fielders, but misjudged its direction. It would be hard to “play on” from a position such as he adopted. The ball arced in towards him and clipped the top of his wicket. So, in fact he was the one who was “clean-bowled”; a dismissal that was qualified by him having not attempted to play a stroke.

North on the other hand had been at the wicket for some time and had already faced 13 balls without scoring. Perhaps in frustration at this, he played an overly-ambitious attacking shot (one not a million miles from a baseball swing), the ball hit the under-edge of his horizontal bat and deflected down into his wicket. So it was North, not Hussey, who “played on” on this occasion.

So, aside from saying that Hussey had been adjudged out “handled the ball” and North dismissed “obstructed the field” (two of the ten ways in which a batsman’s innings can end – see here for a full explanation), the journalist in question could not have been more wrong.

As I said, the piece was no doubt composed quickly in order to “go to press” shortly after play had stopped for the day. Maybe these are minor slips, but surely the core competency of a sports journalist is to record what happened accurately. If they can bring insights and colour to their writing, so much the better, but at a minimum they should be able to provide a correct description of events.

Everyone makes mistakes. Most of my blog articles contain at least one typographical or grammatical error. Some of them may include errors of fact, though I do my best to avoid these. Where I offer my opinions, it is possible that some of these may be erroneous, or that they may not apply in different situations. However, we tend to expect professionals in certain fields to be held to a higher standard.

Auditors

For a molecular biologist, the difference between a 0.20 micro-molar solution and a 0.19 one may be massive. For a team of experimental physicists, unbelievably small quantities may mean the difference between confirming the existence of the Higgs Boson and just some background noise.

In business, it would be unfortunate (to say the least) if auditors overlooked major assets or liabilities. One would expect that law-enforcement agents did not perjure themselves in court. Equally politicians should never dissemble, prevaricate or mislead. OK, maybe I am a little off track with the last one. But surely it is not unreasonable to expect that a cricket journalist should accurately record how a batsman got out.
 
 
Twitter and Truth

twitter.com

I made something of a leap from these sporting events to the more tragic news of Michael Jackson’s recent demise. I recall first “hearing” rumours of this on twitter.com. At this point, no news sites had much to say about the matter. As the evening progressed, the self-styled celebrity gossip site TMZ was the first to announce Jackson’s death. Other news outlets either said “Jackson taken to hospital” or (perhaps hedging their bets) “US web-site reports Jackson dead”.

By this time the twitterverse was experiencing a cosmic storm of tweets about the “fact” of Jackson’s passing. A comparably large number of comments lamented how slow “old media” was to acknowledge this “fact”. Eventually of course the dinosaurs of traditional news and reporting lumbered to the same conclusion as the more agile mammals of Twitter.

In this case social media was proved to be both quick and accurate, so why am I now going to offer a defence of the world’s news organisations? Well I’ll start with a passage from one of my all-time favourite satires, Yes Minister, together with its sequel Yes Prime Minister.

In the following brief excerpt Sir Geoffrey Hastings (the head of MI5, the British domestic intelligence service) is speaking to The Right Honourable James Hacker (the British Prime Minister). Their topic of conversation is the recently revealed news that a senior British Civil Servant had in fact been a Russian spy:

Yes Prime Minister

Hastings: Things might get out. We don’t want any more irresponsible ill-informed press speculation.
Hacker: Even if it’s accurate?
Hastings: Especially if it’s accurate. There is nothing worse than accurate irresponsible ill-informed press speculation.

Yes Prime Minister, Vol. I by J. Lynn and A. Jay

Was the twitter noise about Jackson’s death simply accurate ill-informed speculation? It is difficult to ask this question as, sadly, the tweets (and TMZ) proved to be correct. However, before we garland new media with too many wreaths, it is perhaps salutary to recall that there was a second rumour of a celebrity death circulating in the febrile atmosphere of Twitter on that day. As far as I am aware, Pittsburgh’s finest – Jeff Goldblum – is alive and well as we speak. Rumours of his death (in an accident on a New Zealand movie set) proved to be greatly exaggerated.

The difference between a reputable news outlet and hordes of twitterers is that the former has a reputation to defend. While the average tweep will simply shrug their shoulders at RTing what they later learn is inaccurate information, misrepresenting the facts is a cardinal sin for the best news organisations. Indeed reputation is the main thing that news outlets have going for them. This inevitably includes annoying and time-consuming things such as checking facts and validating sources before you publish.

With due respect to Mr Jackson, an even more tragic set of events also sparked some similar discussions; the aftermath of the Iranian election. The Economist published an interesting artilce comparing old and new media responses to this entitiled: Twitter 1, CNN 0. Their final comments on this area were:

[...]the much-ballyhooed Twitter swiftly degraded into pointlessness. By deluging threads like Iranelection with cries of support for the protesters, Americans and Britons rendered the site almost useless as a source of information—something that Iran’s government had tried and failed to do. Even at its best the site gave a partial, one-sided view of events. Both Twitter and YouTube are hobbled as sources of news by their clumsy search engines.

Much more impressive were the desk-bound bloggers. Nico Pitney of the Huffington Post, Andrew Sullivan of the Atlantic and Robert Mackey of the New York Times waded into a morass of information and pulled out the most useful bits. Their websites turned into a mish-mash of tweets, psephological studies, videos and links to newspaper and television reports. It was not pretty, and some of it turned out to be inaccurate. But it was by far the most comprehensive coverage available in English. The winner of the Iranian protests was neither old media nor new media, but a hybrid of the two.

Aside from the IT person in me noticing the opportunity to increase the value of Twitter via improved text analytics (see my earlier article, Literary calculus?), these types of issues raise concerns in my mind. To balance this slightly negative perspective it is worth noting that both accurate and informed tweets have preceded several business events, notably the recent closure of BI start-up LucidEra.

Also main stream media seem to have swallowed the line that Google has developed its own operating system in Chrome OS (rather than lashing the pre-existing Linux kernel on to its browser); maybe it just makes a better story. Blogs and Twitter were far more incisive in their commentary about this development.

Considering the pros and cons, on balance the author remains something of a doubting Thomas (by name as well as nature) about placing too much reliance on Twitter for news; at least as yet.
 
 
Accuracy an Business Intelligence

A balancing act

Some business thoughts leaked into the final paragraph of the Introduction above, but I am interested more in the concept of accuracy as it pertains to one of my core areas of competence – business intelligence. Here there are different views expressed. Some authorities feel that the most important thing in BI is to be quick with information that is good-enough; the time taken to achieve undue precision being the enemy of crisp decision-making. Others insist that small changes can tip finely-balanced decisions one way or another and so precision is paramount. In a way that is undoubtedly familiar to regular readers, I straddle these two opinions. With my dislike for hard-and-fast recipes for success, I feel that circumstances should generally dictate the approach.

There are of course different types of accuracy. There is that which insists that business information reflects actual business events (often more a case for work in front-end business systems rather than BI). There is also that which dictates that BI systems reconcile to the penny to perhaps less functional, but pre-existing scorecards (e.g. the financial results of an organisation).

A number of things can impact accuracy, including, but not limited to: how data has been entered into systems; how that data is transformed by interfaces; differences between terminology and calculation methods in different data sources; misunderstandings by IT people about the meaning of business data; errors in the extract transform and load logic that builds BI solutions; and sometimes even the decisions about how information is portrayed in BI tools themselves. I cover some of these in my previous piece Using BI to drive improvements in data quality.

However, one thing that I think differentiates enterprise BI from departmental BI (or indeed predictive models or other types of analytics), is a greater emphasis on accuracy. If enterprise BI is to aspire to becoming the single version of the truth for an organisation, then much more emphasis needs to be placed on accuracy. For information that is intended to be the yardstick by which a business is measured, good enough may fall short of the mark. This is particularly the case where a series of good enough solutions are merged together; the whole may be even less than the sum of its parts.

A focus on accuracy in BI also achieves something else. It stresses an aspiration to excellence in the BI team. Such aspirations tend to be positive for groups of people in business, just as they are for sporting teams. Not everyone who dreams of winning an Olympic gold medal will do so, but trying to make such dreams a reality generally leads to improved performance. If the central goal of BI is to improve corporate performance, then raising the bar for the BI team’s own performance is a great place to start and aiming for accuracy is a great way to move forward.
 

 
A final thought: England went on to beat Australia by precisely 115 runs in the second Test at Lord’s; the final result coming today at precisely 12:42 pm British Summer Time. The accuracy of England’s bowling was a major factor. Maybe there is something to learn here.
 

tweet this Tweet this article on twitter.com
Bookmark this article with:
Technorati | del.icio.us | digg | Reddit | Stumble

 

6 Comments | business, business intelligence, data quality, google, social media, technology, text analytics, twitter | Tagged: , , , , | Permalink
Posted by Peter Thomas

An update of the most read articles on this site

1 July 2009

Back in April, I posted My “all-time” most-read 5 articles and mentioned that I would update the list from time to time. At the half-year point of 2009, it seemed appropriate to revisit this area.

I have done two things with the new statistics. First, given the number of articles that I have published, I have expanded the list to 20 articles. Second, to give a different perspective, I have added a run-down of those articles that have received the most views per day.

Of course the first list is more likely to contain older articles (which have had more time to accumulate views), whereas the second list is more likely to include new articles (given that most articles have peak viewing figures soon after they are posted). Despite the vagaries of both approaches, it is probably safe to say that if an article appears on both lists it has been pretty popular. I have highlighted 12 such posts in yellow below.

In future, I may consider calculating how well an article has performed against the typical ageing profile. This would address the shortcomings of both of the current tables and therby offer a more definitive benchmark. However this will be dependent on WordPress.com making it a bit easier to download information about page views.

As before I have focussed just on articles, so views of pages about my career or other background information have been omitted from the following.
 

Most viewed pages
Article Views
1 Measuring the benefits of Business Intelligence 2,054
2 Business is from Mars and IT is from Venus 1,497
3 Trends in Business Intelligence 1,324
4 Business Analytics vs Business Intelligence 1,320
5 A review of “The History of Business Intelligence” by Nic Smith 1,201
6 Mountain Biking and Systems Integration 1,129
7 “Why Business Intelligence projects fail 1,032
8 Mergers and value 929
9 Is outsourcing business intelligence a good idea? 909
10 The Top Business Issues facing CIOs / IT Directors – Results 865
11 “Gartner sees a big discrepancy between BI expectations and realities” – Intelligent Enterprise 787
12 Pigeonholing – A tragedy 775
13 “All that glisters is not gold” – some thoughts on dashboards 732
14 Two pictures paint a thousand words… 729
15 BI implementations are like icebergs 720
16 Vision vs Pragmatism 716
17 The specific benefits of business intelligence in insurance 689
18 Holistic vs Incremental approaches to BI 689
19 Perseverance 624
20 A single version of the truth? 596

 
 

Most views / day (qualification: 300 views)
Article Views / day
1 A single version of the truth? 99.3
2 “Why Business Intelligence projects fail 38.2
3 “Involving users in business intelligence strategy key for success” – Christina Torode on SearchCio-Midmarket.com 33.0
4 Data – Information – Knowledge – Wisdom 29.5
5 Literary calculus? 24.3
6 Mountain Biking and Systems Integration 22.6
7 Measuring the benefits of Business Intelligence 16.4
8 Business Analytics vs Business Intelligence 13.9
9 A review of “The History of Business Intelligence” by Nic Smith 12.9
10 Mergers and value 12.9
11 Trends in Business Intelligence 11.6
12 Two pictures paint a thousand words… 11.6
13 Business Intelligence Competency Centres 10.5
14 Using multiple business intelligence tools in an implementation – Part I 10.2
15 Maureen Clarry stresses the need for change skills in business intelligence on BeyeNetwork 9.1
16 The importance of feasibility studies in business intelligence 9.0
17 Pigeonholing – A tragedy 8.5
18 Is outsourcing business intelligence a good idea? 8.3
19 The Top Business Issues facing CIOs / IT Directors – Results 8.2
20 The Dictatorship of the Analysts 7.1

 

tweet this Tweet this article on twitter.com
Bookmark this article with:
Technorati | del.icio.us | digg | Reddit | Stumble

 

2 Comments | business, business intelligence, cultural transformation, site update, technology | Permalink
Posted by Peter Thomas

A recording of me being interviewed by Brian Roger of SmartDataCollective.com

30 June 2009

SmartDataCollective.com

I have been a featured blogger on SmartDataCollective.com almost as long as I have been a blogger. SDC.com is Social Media Today’s community site, focussed on all aspects of Business Intelligence, Data Warehousing and Analytics, with a pinch of social media thrown in to the mix.

Brian Roger, the SDC.com editor, was recently kind enough to interview me about my career in BI, the challenges I have faced and what has helped to overcome these. This interview is now available to listen to as part of their Podcast series – click on the image below to visit their site.

sdc-podcast

SmartDataCollective.com Intervew

I would be interested in feedback about any aspect of this piece, which I am grateful to Brian for arranging.
 

 
Social Media Today LLC helps global organizations create purpose-built B2B social communities designed to achieve specific, measurable corporate goals by engaging exactly the customers and prospects they most want to reach. Social Media Today helps large companies leverage the enormous power of social media to build deeper relationships with potential customers and other constituencies that influence the development of new business. They have found that their primary metrics of success are levels of engagement and business leads. One thousand people who come regularly and might buy an SAP, Oracle or Teradata system some day is better than a million people who definitely won’t.

Social Media Today LLC, is a battle-tested, nimble team of former journalists, online managers, and advertising professionals who have come together to make a new kind of media company. With their backgrounds, and passions for, business-to-business and public policy conversations, they have decided to focus their efforts in this area. To facilitate the types of convresations that they would like to see Social Media Today is assembling the world’s best bloggers and providing them with an independent “playground” to include their posts, to comment and rate posts, and to connect with each other. On their flagship site, SocialMediaToday.com, they have brought together many of the most intriguing and original bloggers on media and marketing, covering all aspects of what makes up the connective tissue of social media from a global perspective.
 

tweet this Tweet this article on twitter.com
Bookmark this article with:
Technorati | del.icio.us | digg | Reddit | Stumble

5 Comments | business, business analytics, business intelligence, change management, cultural transformation, data warehousing, management, management information, social media, technology | Tagged: , , , , , | Permalink
Posted by Peter Thomas

A blast from the past…

25 June 2009

Gutenberg Printing Press

I recently came across a record of my first ever foray into the world of Business Intelligence, which dates back to 1997. This was when I was still predominantly an ERP person and was considering options for getting information out of such systems. The piece in question is a brief memo to my boss (the CFO of the organisation I was working at then) about what I described as the OLAP market.

This was a time of innocence, when Google had not even been incorporated, when no one yet owned an iPod and when, if you tried to talk to someone about social media, they would have assumed that you meant friendly journalists. All this is attested to by the fact that this was a paper memo that was printed and circulated in the internal mail – remember that sort of thing?

Given that the document has just had its twelfth birthday, I don’t think I am breeching any confidences in publishing it, though I have removed the names of the recipients for obvious reasons.
 

INTERNAL MEMORANDUM
To: European CFO
cc: Various interested parties
From: Peter Thomas
Date: 16th June 1997
Subject: What is OLAP?

 
On-Line Analytical Processing (OLAP) is a category of software technology that enables analysts, managers and executives to gain insight into data. This is achieved by providing fast, consistent, interactive access to a wide variety of possible views of information. This has generally been transformed from raw data to reflect the real dimensionality of the enterprise.

There are around 30 vendors claiming to offer OLAP products. A helpful report by Business Intelligence[1] (an independent research company) estimates the market share of these . As many of these companies sell other products, the following cannot be viewed as 100% accurate. However the figures do provide some interesting reading.
 

Vendor

1996

1995

Market Position

Share (%)

Market Position

Share (%)
Oracle

1

19.0%

1

20.0%
Hyperion Software

2

18.0%

2

19.0%
Comshare

3

12.0%

3

16.0%
Cognos

4

9.0%

4

5.0%
Arbor Software

5

4.8%

7

2.9%
Holistic Systems

6

4.3%

6

4.7%
Pilot Software

7

4.0%

5

4.8%
MircoStrategy

8

3.5%

9

2.1%
Planning Sciences

9

2.6%

8

2.3%
Information Advantage

10

1.8%

10

1.4%

 
In this group, some companies (Hyperion, Comshare, Holistic, Pilot Software and Planning Services) provide either complete products or extensive toolkits. In contrast some vendors (such as Arbor and – outside the top ten – Applix) only sell specialist multi-dimensional databases. Others (e.g. Cognos and – outside the top ten – BusinessObjects and Brio Technology), offer client based OLAP tools which are basically sophisticated report writers. The final group (including MicroStrategy and Information Advantage) offer a mixed relational / dimensional approach called Relational OLAP or ROLAP.

If we restrict ourselves to the “one-stop solution” vendors in the above list, it is helpful to consider the relative financial position of the top three.
 

Vendor

Market Cap.[2]

($m)

Turnover

($m)

Profit

($m)
Oracle

32,405

4,223[3]

603
Hyperion Software

335

173[4]

9
Comshare

132

119[5]

(9)

 

[1] The OLAP Report by Nigel Pendse and Richard Creeth © Business Intelligence 1997
[2] As at June 1997
[3] 12 months to March 1997
[4] 12 months to June 1996
[5] 12 months to June 1996

 

 
It is of course also worth pointing out that I used to disagree with what Nigel Pendse wrote a lot less back then!
 

tweet this Tweet this article on twitter.com
Bookmark this article with:
Technorati | del.icio.us | digg | Reddit | Stumble

 

Leave a Comment » | business, business intelligence, businessobjects, cognos, hyperion, oracle, technology | Tagged: | Permalink
Posted by Peter Thomas

A single version of the truth?

25 June 2009
linkedin The Data Warehousing Institute The Data Warehousing Institute (TDWI™) 2.0

As is frequently the case, I was moved to write this piece by a discussion on LinkedIn.com. This time round, the group involved was The Data Warehousing Institute (TDWI™) 2.0 and the thread, entitled Is one version of the truth attainable?, was started by J. Piscioneri. I should however make a nod in the direction of an article on Jim Harris’ excellent Obsessive-Compulsive Data Quality Blog called The Data Information Continuum; Jim also contributed to the LinkedIn.com thread.

Standard note: You need to be a member of both LinkedIn.com and the group mentioned to view the discussions.
 
 
Introduction

A Calabi–Yau manifold

Here are a couple of sections from the original poster’s starting comments:

I’ve been thinking: is one version of the truth attainable or is it a bit of snake oil? Is it a helpful concept that powerfully communicates a way out of spreadmart purgatory? Or does the idea of one version of the truth gloss over the fact that context or point of view are an inherent part of any statement about data, which effectively makes truth relative? I’m leaning toward the latter position.

[...]

There can only be one version of the truth if everyone speaks the same language and has a common point of view. I’m not sure this is attainable. To the extent that it is, it’s definitely not a technology exercise. It’s organizational change management. It’s about changing the culture of an organization and potentially breaking down longstanding barriers.

Please join the group if you would like to read the whole post and the subsequent discussions, which were very lively. Here I am only going to refer to these tangentially and instead focus on the concept of a single version of the truth itself.

Readers who are not interested in the ellipitcal section of this article and who would instead like to cut to the chase are invited to click here (warning there are still some ellipses in the latter sections).
 
 
A [very] brief and occasionally accurate history of truth

The demise of a cherry tree

I have discovered a truly marvellous proof of the nature of truth, which this column is too narrow to contain.

– Pierre de Tomas

Instead of trying to rediscover M. Tomas’ proof, I’ll simply catalogue some of the disciplines that have been associated (rightly or wrongly) with trying to grapple with the area:

  • Various branches of Philosophy, including:

    • Metaphysics
    • Epistemology
    • Ethics
    • Logic
  • History
  • Religion (or more perhaps more generally spirituality)
  • Natural Science
  • Mathematics
  • and of course Polygraphism

Lie algebra

Given my background in Pure Mathematics the reader might expect me to trumpet the claims of this discipline to be the sole arbiter of truth; I would reply yes and no. Mathematics does indeed deal in absolute truth, but only of the type: if we assume A and B, it then follows that C is true. This is known as the axiomatic approach. Mathematics makes no claim for the veracity of axioms themselves (though clearly many axioms would be regarded as self-evidently true to the non-professional). I will also manfully resist the temptation to refer to the wrecking ball that Kurt Gödel’s took to axiomatic systems in 1931.

Physical science

I have also made reference (admittedly often rather obliquely) to various branches of science on this blog, so perhaps this is another place to search for truth. However the Physical sciences do not really deal in anything as absolute as truth. Instead they develop models that approximate observations, these are called scientific theories. A good theory will both explain aspects of currently observed phenomena and offer predictions for yet-to-be-observed behaviour (what use is a model if it doesn’t tell us things that we don’t already know?). In this way scientific theories are rather like Business Analytics.

Unlike mathematical theories, the scientific versions are rather resistant to proof. Somewhat unfairly, while a mountain of experiments that are consistent with a scientific theory do not prove it, it takes only one incompatible data point to disprove it. When such an inconvenient fact rears its head, the theory will need to be revised to accommodate the new data, or entirely discarded and replaced by a new theory. This is of course an iterative process and precisely how our scientific learning increases. Warning bells generally start to ring when a scientist starts to talk about their theory being true, as opposed to a useful tool. The same observation could be made of those who begin to view their Business Analytics models as being true, but that is perhaps a story for another time.

The Thinker

I am going to come back to Physical science (or more specifically Physics) a little later, but for now let’s agree that this area is not going to result in defining truth either. Some people would argue that truth is the preserve of one of the other subjects listed above, either Philosophy or Religion. I’m not going to get into a debate on the merits of either of these views, but I will state that perhaps the latter is more concerned with personal truth than supra-individual truth (otherwise why do so many religious people disagree with each other?).

Discussing religion on a blog is also a sure-fire way to start a fire, so I’ll move quickly on. I’m a little more relaxed about criticising some aspects of Philosophy; to me this can all too easily descend into solipism (sometimes even quicker than artificial intelligence and cognitive science do). Although Philosophy could be described as the search for truth, I’m not convinced that this is the same as finding it. Maybe truth itself doesn’t really exist, so attempting to create a single version of it is doomed to failure. However, perhaps there is hope.
 
 
Trusting your GUT feeling

Physicists have a sense of humour too you know...

© xkcd.com

After the preceding divertimento, it is time to return to the more prosaic world of Business Intelligence. However there is first room for the promised reference to Physics. For me, the phrase “a single version of the truth” always has echoes of the search for a Grand Unified Theory (GUT). Analogous to our discussions about truth, there are some (minor) definitional issues with GUT as well.

Some hold that GUT applies to a unification of the electromagnetic, weak nuclear and strong nuclear forces at very high energy levels (the first two having already been paired in the electroweak force). Others that GUT refers to a merging of the particles and forces covered by the Standard Model of Quantum Mechanics (which works well for the very small) with General Relativity (which works well for the very big). People in the first camp might refer to this second unification as a ToE (Theory of Everything), but there is sometimes a limit to how much Douglas Adams’ esteemed work applies to reality.

For the purposes of this article, I’ll perform the standard scientific trick of a simplifying assumption and use GUT in the grander sense of the term.

Scientists have striven to find a GUT for decades, if not centuries, and several candidates have been proposed. GUT has proved to be something of a Holy Grail for Physicists. Work in this area, while not as yet having been successful (at least at the time of writing), has undeniably helped to shed a light on many other areas where our understanding was previously rather dim.

This is where the connection with a single version of the truth comes in. Not so much that such either concept is guaranteed to be achievable, but that a lot of good and useful things can be accomplished on a journey towards both of them. If, in a given organisation, the journey to a single version of the truth reaches its ultimate destination, then great. However if, in an another company, a single version of the truth remains eternally just over the next hill, or round the next corner, then this is hardly disastrous and maybe it is the journey itself (and the aspirations with which it is commenced on) that matters more than the destination.

Before I begin to sound too philosophical (cf. above) let me try to make this more concrete by going back to our starting point with some Mathematics and considering some Venn diagrams.
 
 
Ordo ab chao

In my experience the following is the type of situation that a good Business Intelligence programme should address:

Fragmentation

The problems here are manifold:

  1. Although the various report systems are shown as separate, the real situation is probably much worse. Each of the reporting and analysis systems will overlap, perhaps substantially, with one or more or the other ones. Indeed the overlapping may be so convoluted that it would be difficult to represent this in two dimensions and I am not going to try. This means that you can invariably ask the same question (how much have we sold this month) of different systems and get different answers. It may be difficult to tell which of these is correct, indeed none of them may be a true reflection of business reality.
  2. There are a whole set of things that may be treated differently in the different ellipses. I’ll mention just two for now: date and currency. In one system a transaction may be recorded in a month when it is entered into the system. In another it may be allocated to the month when the event actually occurred (sometimes quite a while before it is entered). In a third perhaps the transaction is only dated once it has been authorised by a supervisor.

    In a multi-currency environment reports may be in the transactional currency, rolled-up to the currency of the country in which they occurred, or perhaps aggregated across many countries in a number of “corporate” currencies. Which rate to use (rate on the day, average for the month, rolling average for the last year, a rate tied to some earlier business transaction etc.) may be different in different systems, equally the rate may well vary according to the date of the transaction (making the last set of comments about which date is used even more pertinent).

  3. A whole set of other issues arise when you begin to consider things such as taxation (are figures nett or gross), discounts, commissions to other parties, phased transactions and financial estimates. Some reports may totally ignore these, others my take account of some but not others. A mist of misunderstanding is likely to arise.
  4. Something that is not drawn on the above diagram is the flow of data between systems. Typically there will be a spaghetti-like flow of bits and bytes between the different areas. What is also not that uncommon is that there is both bifurcation and merging in these flows. For example, some sorts of transactions from Business Unit A may end up in the Marketing database, whereas others do not. Perhaps transactions carried out on behalf of another company in the group appear in Business Unit B’s reports, but must be excluded from the local P&L. The combinations are almost limitless.

    Interfaces can also do interesting things to data, re-labelling it, correcting (or so their authors hope) errors in source data and generally twisting the input to form output that may be radically different. Also, when interfaces are anything other than real-time, they introduce a whole new arena in which dates can get muddled. For instance, what if a business transaction occurred in a front-end system on the last day of a year, but was not interfaced to a corporate database until the first day of the next one – which year does it get allocated to in the two places?

  5. Finally, the above says nothing about the costs (staff and software) of maintaining a heterogeneous reporting landscape; or indeed the costs of wasted time arguing about which numbers are right, or attempting to perform tortuous (and ultimately fruitless) reconciliations.

Now the ideal situation is that we move to the following diagram:

De-fragmentation

This looks all very nice and tidy, but there are still two major problems.

  1. A full realisation of this transformation may be prohibitively expensive, or time-consuming.
  2. Having brought everything together into one place offers an opportunity to standardise terminology and to eliminate the confusion caused by redundancy. However, it doesn’t per se address the other points made from 2. onwards above.

The need to focus on what is possible in a reasonable time-frame and at a reasonable cost may lead to a more pragmatic approach where the number of reporting and analysis systems is reduced, but to a number greater than one. Good project management may indeed dictate a rolling programme of consolidation, with opportunities to review what has worked and what has not and to ascertain whether business value is indeed being generated by the programme.

Nevertheless, I would argue that it is beneficial to envisage a final state for the information architecture, even if there is a tacit acceptance that this may not be realised for years, if at all. Such a framework helps to guide work in a way that making it up as we go along does not. I cover this area in more detail in both Holistic vs Incremental approaches to BI and Tactical Meandering for those who are interested.

It is also inevitable that even in a single BI system data will need to be presented in different ways for different purposes. To take just one example, if you goal is to see how the make up of a book of business has varied over time, then it is eminently sensible to use a current exchange rate for all transactions; thereby removing any skewing of the figures caused by forex fluctuations. This is particularly the case when trying to assess the profitability of business where revenue occurs at a discrete point in the past, but costs may be spread out over time.

However, if it is necessary to look at how the organisation’s cash-flow is changing over time, then the impact of fluctuations in foreign exchange rates must be taken into account. Sadly if an American company wants to report how much revenue it has from its French subsidiary then the figures must reflect real-life euro / dollar rates (unrealised and realised foreign currency gains and losses notwithstanding).

What is important here is labelling. Ideally each report should show the assumptions under which it has been compiled at the top. This would include the exchange rate strategy used, the method by which transactions are allocated to dates, whether figures are nett or gross and which transactions (if any) have been excluded. Under this approach, while it is inevitable that the totals on some reports will not agree, at least the reports themselves will explain why this is the case.

So this is my take on a single version of the truth. It is both a) an aspirational description of the ideal situation and something that is worth striving for and b) a convenient marketing term – a sound-bite if you will – that presents a palatable way of describing a complex set of concepts. I tried to capture this essence in my reply to the LinkedIn.com thread, which was as follows:

To me, the (extremely hackneyed) phrase “a single version of the truth” means a few things:

  1. One place to go to run reports and perform analysis (as opposed to several different, unreconciled, overlapping systems and local spreadsheets / Access DBs)
  2. When something, say “growth” appears on a report, cube, or dashboard, it is always calculated the same way and means the same thing (e.g. if you have growth in dollar terms and growth excluding the impact of currency fluctuations, then these are two measures and should be clearly tagged as such).
  3. More importantly, that the organisation buys into there being just one set of figures that will be used and self-polices attempts to subvert this with roll-your-own data.

Of course none of this equates to anything to do with truth in the normal sense of the word. However life is full of imprecise terminology, which nevertheless manages to convey meaning better than overly precise alternatives.

More’s Utopia was never intended to depict a realistic place or system of government. These facts have not stopped generations of thinkers and doers from aspiring to make the world a better place, while realising that the ultimate goal may remain out of reach. In my opinion neither should the unlikelihood of achieving a perfect single version of the truth deter Business Intelligence professionals from aspiring to this Utopian vision.

I have come pretty close to achieving a single version of the truth in a large, complex organisation. Pretty close is not 100%, but in Business Intelligence anything above 80% is certainly more than worth the effort.
 

tweet this Tweet this article on twitter.com
Bookmark this article with:
Technorati | del.icio.us | digg | Reddit | Stumble

 

16 Comments | business, business intelligence, management information, technology | Tagged: , , , , , , | Permalink
Posted by Peter Thomas

« Previous Entries