Trouble at the top

IRM MDM/DG

Several weeks back now, I presented at IRM’s collocated European Master Data Management Summit and Data Governance Conference. This was my second IRM event, having also spoken at their European Data Warehouse and Business Intelligence Conference back in 2010. The conference was impeccably arranged and the range of speakers was both impressive and interesting. However, as always happens to me, my ability to attend meetings was curtailed by both work commitments and my own preparations. One of these years I will go to all the days of a seminar and listen to a wider variety of speakers.

Anyway, my talk – entitled Making Business Intelligence an Integral part of your Data Quality Programme – was based on themes I had introduced in Using BI to drive improvements in data quality and developed in Who should be accountable for data quality?. It centred on the four-pillar framework that I introduced in the latter article (yes I do have a fetish for four-pillar frameworks as per):

The four pillars of improved data quality

Given my lack of exposure to the event as a whole, I will restrict myself to writing about a comment that came up in the question section of my slot. As per my article on presenting in public, I try to always allow time at the end for questions as this can often be the most interesting part of the talk; for delegates and for me. My IRM slot was 45 minutes this time round, so I turned things over to the audience after speaking for half-an-hour.

There were a number of good questions and I did my best to answer them, based on past experience of both what had worked and what had been less successful. However, one comment stuck in my mind. For obvious reasons, I will not identify either the delegate, or the organisation that she worked for; but I also had a brief follow-up conversation with her afterwards.

She explained that her organisation had in place a formal data governance process and that a lot of time and effort had been put into communicating with the people who actually entered data. In common with my first pillar, this had focused on educating people as to the importance of data quality and how this fed into the organisation’s objectives; a textbook example of how to do things, on which the lady in question should be congratulated. However, she also faced an issue; one that is probably more common than any of us information professionals would care to admit. Her problem was not at the bottom, or in the middle of her organisation, but at the top.

So how many miles per gallon do you get out of that?

In particular, though data governance and a thorough and consistent approach to both the entry of data and transformation of this to information were all embedded into the organisation; this did not prevent the leaders of each division having their own people take the resulting information, load it into Excel and “improve” it by “adjusting anomalies”, “smoothing out variations”, “allowing for the impact of exceptional items”, “better reflecting the opinions of field operatives” and the whole panoply of euphemisms for changing figures so that they tell a more convenient story.

In one sense this was rather depressing, someone having got so much right, but still facing challenges. However, it also chimes with another theme that I have stressed many times under the banner of cultural transformation; it is crucially important than any information initiative either has, or works assiduously to establish, the active support of all echelons of the organisation. In some of my most successful BI/DW work, I have had the benefit of the direct support of the CEO. Equally, it is is very important to ensure that the highest levels of your organisation buy in before commencing on a stepped-change to its information capabilities.

I am way overdue employing another sporting analogy - odd however how must of my rugby-related ones tend to be non-explicit

My experience is that enhanced information can have enormous payback. But it is risky to embark on an information programme without this being explicitly recognised by the senior management team. If you avoid laying this important foundation, then this is simply storing up trouble for the future. The best BI/DW projects are totally aligned with the strategic goals of the organisation. Given this, explaining their objectives and soliciting executive support should be all the easier. This is something that I would encourage my fellow information professionals to seek without exception.
 

Data visualisation

Some pictures speak for themselves:

If you don't know what this is, check out the announcement from the CDF Collaboration at: http://www.fnal.gov/pub/today/archive_2011/today11-04-07_CDFpeakresult.html - All you have to do is click here. HINT: the peak at 140 GeV/c^2 may be important.
 

An informed decision

Caterham 7 vs Data Warehouse appliance - spot the difference

A friend and fellow information professional is currently responsible for both building a new data warehouse and supporting its predecessor, which is based on a different technology platform. In these times of ever-increasing focus on costs, she had been asked to port the old warehouse to the new platform, thereby avoiding some licensing payments. She asked me what I thought about this idea and we chatted for a while. For some reason, our conversation went off at a bit of a tangent and I started to tell her the story of an acquaintance of mine and his recent sad experiences.

+++

My acquaintance, let’s call him Jim to avoid causing any embarassment, had always been interested in cars; driving them, maintaining them, souping them up, endlessly reading car magazines and so on. His dream had always been to build his own car and his eye had always been on a Caterham kit. I suppose for him the pleasure of making a car was at least as great, if not more, as the pleasure of driving one.

It's just like Lego

Jim saved his pennies and eventually got together enough cash to embark on his dream project. Having invested his money, he started to also invest his time and effort. However, after a few weeks of toil, he hit a snag. It was nothing to do with his slowly emerging Caterham, but to do with the more quotidian car he used for his daily commute to work. Its engine had developed a couple of niggles that had been resistant to his own attempts to fix them and he had reluctantly decided that it was in need of some new parts and quite expensive ones at that. Jim had already spent quite a bit of cash on the Caterham and more on some new tools that he needed to assemble it. The last thing he wanted to do now was to have a major outlay on his old car; particularly because, once the Caterham was finished, he had planned to trade it for its scrap-metal worth.

But now things got worse, Jim’s current car failed its MOT (vehicle safety inspection for any non-UK readers) because the faulty engine did not meet emission standards. However, one of his friends came up with a potential solution. He said, “As you have already assembled the Caterham engine, why not put this into your current car and use this instead? You can then swap it out into the Caterham chassis and body when you have built this.”

Headless Jim - with cropped face to protect his anonymity

This sounded like a great idea to Jim, but there were some issues with it. His Cateham was supplied with a Cosworth-developed 2.3-litre Ford Duratec engine. This four-cylinder twin cam unit was the wrong size and shape to fit into the cavity left by removing the worn-out engine from his commuting car. Well as I had mentioned at the start, Jim was a pretty competent amateur mechanic and he thought that he had a good chance of rising to the challenge. He was motivated by the thought of not having to shell out extra cash and in any case he loved tinkering with cars.

So he put in some new brackets to hold the Caterham engine. He then had to grind-down a couple of protruding pieces of the Duratec block to gain the extra 5 mm necessary to squeeze it in. The fuel feeds were in the wrong place, but a bit of plumbing and that was also sorted. Perhaps this might cause an issue with efficiency of the engine burn cycle, but Jim figured that it would probably be OK. Next the vibration dampers were not really up to the job of dealing with the more powerful engine and neither was the exhaust system. No worries, thought Jim, a tap of a hammer here, a bend of a pipe here and he could also add in a couple of components that had been sitting at the back of his garage rusting for years as well. Eventually everything seemed fine.

Jim ventured out of his garage in his old car, with its new engine. He was initially a bit trepidatious, but his work seemed to be hanging together. Sure the car was making a bit of a noise, shaking a bit and the oil temperature seems a bit high, but Jim felt that these were only minor problems. He told himself that all his handiwork had to do was to hang together for a few more months until he finished the rest of the Caterham.

Angular momentum = Sum over i : Ri x mi x Vi

With these nice thoughts in mind, Jim approached a bend. The car flew off the road at a tangent as he realised – too late – that he had been travelling at Caterham speeds into the corner and didn’t have a Caterham chassis, a Caterham suspension, or Caterham brakes. His old car was not up to dealing with the forces created in the turn. His tyres failed to grip and, after what seemed like an eternity of slow-motion spinning and screeching and panic, he find himself in a ditch; healthy, but with a wheel sheared off and smoke coming out of the front of the car. A later inspection confirmed that his commuting car was a write-off, and his insurance policy didn’t fully cover the cost of a new vehicle.

Jim ended up having to buy another day-to-day car, which delayed him from spending the additional money necessary to get the Caterham on the road for quite some time. However, after scrimping and saving for a while, he eventually got back to his dream project, only to find that combination of the modifications he had to make to the Duratec engine, plus the after effects of the crash meant that it was now useless and he needed to purchase a replacement.

So because Jim didn’t want to run to the expense of maintaining his old car while he built his new one, he would instead have to buy a new temporary car plus a new engine for the Caterham. Jim was just as far off from finishing the Caterham as when he had started, despite wasting a lot of time and money along the way. A very sad story.

+++

Suddenly I realised that I had been wittering on about a wholly unrelated subject to my friend’s data warehousing problem. I apologised and turned the conversation back to this. To my astonishment, she told me that she had already made up her mind. I suppose she had taken advantage of the length of time I had spent telling Jim’s story to more profitably weigh the pros and cons of different approaches in her mind and thereby had reached her decision. Anyway, she thanked me for my help, I protested that I hadn’t really offered her any and we each went our separate ways.

I found out later she had decided to pay the maintenance costs on the old data warehouse.


I would like to apologise in advance if anyone at Caterham, Cosworth, Ford, or indeed Peugeot, takes offence to any of the content of the above story or its illustrations. I’m sure that you make very fine products and this article isn’t really about any of them.

Medical malpractice

8 plus 7 equals 15, carry one, er...

I was listening to a discussion with two medical practitioners on the radio today while driving home from work. I’ll remove the context of the diseases they were debating as the point I want to make is not specifically to do with this aspect and dropping it removes a degree of emotion from the conversation. The bone of contention between the two antagonists was the mortality rate from a certain set of diseases in the UK and whether this was to do with the competency of general practitioners (GPs, or “family doctors” for any US readers) and the diagnostic procedures they use, or to do with some other factor.

In defending her colleagues from the accusations of the first interviewee, the general practitioner said that the rate of mortality for sufferers of these diseases in other European countries (she specifically cited Belgium and France) was greater than in the UK. I should probably pause at this point to note that this comment seemed the complete opposite of every other European health survey I have read in recent years, but we will let that pass and instead focus on the second part of her argument. This was that that better diagnoses would be made if the UK hired more doctors (like her), thereby allowing them to spend more time with each patient. She backed up this assertion by then saying that France has many more doctors per 1,000 people than the UK (the figures I found were 3.7 per 1,000 for France and 2.2 per 1,000 for the UK; these were totally different to the figures she quoted, but again I’ll let that pass as she did seem to at least have the relation between the figures in each country the right way round this time).

What the GP seemed to be saying is summarised in the following chart:

Vive la difference

I have no background in medicine, but to me the lady in question made the opposite point to the one she seemed to want to. If there are fewer doctors per capita in the UK than in France, but UK mortality rates are better, it might be more plausible to argue that less doctors implies better survival rates; this is what the above chart suggests. Of course this assertion is open to challenge and – as with most statistical phenomena – there are undoubtedly many other factors. There is also of course the old chestnut of correlation not implying causality (not that the above chart even establishes correlation). However, at the very least, the “facts” as presented did not seem to be a prima facie case for hiring more UK doctors.

Sadly for both the GP in question and for inhabitants of the UK, I think that the actual graph is more like:

This exhibit could perhaps suggest that the second doctor had a potential point, but such simplistic observations, much as we may love to make them, do not always stand up to rigorous statistical analysis. Statistical findings can be as counter-intuitive as many other mathematical results.

Speaking of statistics, when challenged on whether she had the relative mortality rates for France and the UK the right way round, the same GP said, “well you can prove anything with statistics.” We hear this phrase so often that I guess many of us come to believe it. In fact it might be more accurate to say, “selection bias is all pervasive”, or perhaps even “innumeracy will generally lead to erroneous conclusions being drawn.”

When physicians are happy to appear on national radio and exhibit what is at best a tenuous grasp of figures, one can but wonder about the risk of numerically-based medical decisions sometimes going awry. With doctors also increasingly involved in public affairs (either as expert advisers or – in the UK at least – often as members of parliament), perhaps these worries should also be extended into areas of policy making.

Even more fundamentally (but then as an ex-Mathematician I would say this), perhaps the UK needs to reassess how it teaches mathematics. Also maybe UK medical schools need to examine numeric proficiency again just before students graduate as well as many years earlier when candidates apply; just in case something in the process of producing new doctors has squeezed their previous mathematical ability out of them.

Before I begin to be seen as an opponent of the medical profession, I should close by asking a couple of questions that are perhaps closer to home for some readers. How many of the business decisions that are taken using information lovingly crafted by information professionals such as you and me are marred by an incomplete understanding of numbers on the part of [hopefully] a small subsection of users? As IT professionals, what should we be doing to minimise the likelihood of such an occurrence in our organisations?
 

How to use your BI Tool to Highlight Deficiencies in Data

My interview with Microsoft’s Bruno Aziza (@brunoaziza), which I trailed in Another social media-inspired meeting, was published today on his interesting and entertaining bizintelligence.tv site.

You can take a look at the canonical version here and the YouTube version appears below:

The interview touches on themes that I have discussed in:

 

Thanks to Jim Harris’ OCDQ Blog

I would like to start 2011 by thanking Jim Harris for selecting one of my articles – Who should be accountable for data quality? – as a Best Data Quality Blog Post Of 2010 on his Obsessive Compulsive Data Quality blog.

I would recommend Jim’s excellent site as a great repository for current thinking and best practise in this crucial area.
 

Some thoughts on the IRM(UK) DW/BI conference

As previously advertised, I presented at the recent IRM(UK) DW/BI seminar in London. As a speaker I was entitled to attend the full three days, but as is typically the case, other work commitments meant that I only went along on the day of my session, 4th November. A mixture of running into business acquaintances, making sure that audio/visual facilities work and last minute run-throughs of my slides all conspired to ensure that I was able to listen to fewer talks that I would have liked. In comparing notes with other speakers, it is generally the same for them. Maybe I should consider attending a seminar as a delegate sometime!

Nevertheless, I did get along to some presentations and also managed to finally meet Dylan Jones of dataqualitypro.com (@DataQualityPro) in person after running into each other virtually for years. Unfortunatlely, I also managed to fail to connect with a number of tweeps of my acquaintance including: Loretta Mahon Smith (@silverdata) – who even attended my talk without us bumping into each other – and Scott Davis (@scottatlyzasoft); I guess that is just how it goes with seminars sometimes.
 
 
Story-telling and Information Quality

Ma mère l'oye by Gustave Doré (for the avoidance of doubt, I'm not saying that Lori is Mother Goose)

At face value these may seem odd bed-fellows. However, Lori Silverman of Partners for Progress managed to intertwine the two effectively. This was despite being handicapped by an attack of laryngitis that meant that her, already somewhat nasal tones, from time to time morphed into a shriek. Sitting as I was directly beside a loudspeaker, I felt some initial discomfort and even considered departing for a less auricularly challenged part of the conference centre. However I was glad that I decided to tough it out because Lori turned out to be a very entertaining, engaging and insightful speaker. I won’t steal her thunder by revealing her main thesis and instead suggest that you try to catch her speaking at some future point, she is well worth listening to in my opinion.
 
 
Open Source BI makes headway in the Irish Government sector

Jaspersoft and System Dynamics

I next attended a presentation by leading open source BI company Jaspersoft. This was kicked-off by their CEO Brian Gentile who then introduced a case study about an Irish Government department rolling-out the company’s products. The implementer, was System Dynamics, Ireland’s largest indigenous IT business solutions company*.

System Dynamics CEO Tony McGuire and BI Team Lead Emmet Burke both spoke about this recent project, which covered 500+ users. Open source has traditionally had something of a challenge establishing a foothold in the public sector. The assertion made in this session was that the current fiscal challenges faced by the Irish Republic meant that it was becoming an option they were giving greater credence to. I guess, as with many areas of open source applications, it is probably a case of waiting to see whether a trend establishes itself.

John Taylor of Information Builders was speaking in the room that would next host my session and so I was able to catch the last 15 minutes of his presentation on Information Management, which seemed to have been well-attended and well-received.
 
 
Measuring the benefits of BI

My presentation occupied the graveyard slot of 4:30pm and I led by saying that I fully realised that all that stood between delegates and the drinks reception was my talk. Given the lateness of the hour, I had been a little concerned about attendance, but I guess that there were at least 50 or so people present. All of them stuck it out to the bitter end, which was gratifying.

There is always the moment of frisson in public speaking when, at the end of the talk, you ask whether are any questions with an image of tumbleweed spinning across the prairie in your mind (something that happened to me on one previous occasion a long time ago). Thankfully the audience asked a number of interesting and insightful questions, which I answered to the best of my ability. Indeed I was locked in discussions with a couple of delegates long after the meeting had officially broken up.

Measuring the success of BI - Agenda

In my introduction, I began by issuing my customary caveat about the danger of too blindly following any recipe for success. I then provided some background about my first major achievement in data warehousing and went on to present the general framework for success in BI/DW programmes that I developed as a result of this. In concluding the first part of the speech, I attempted to delineate the main benefits of BI and also touched on some of its limitations.

Having laid these hopefully substantial foundations, the meat of the presentation expanded on ideas I briefly touched on in my earlier article Measuring the Benefits of Business Intelligence. This included highlighting some of the reasons why measuring the impact of BI on, say, profitability can be a challenge, but stressing that this was still often an objective that it was possible to achieve. I also spent some time examining in detail different techniques for quantifying the different tangible and intangible impacts of BI (most of which are covered in the above referenced article).

A sporting analogy by the back-door - England's victory in the 2003 Rugby World Cup, which was clearly inspired by the successful launch of the first phase of the EMIR BI/DW system at Chubb Insurance earlier in the year

My closing thought was that, in situations where it is difficult to precisely assess the monetary impact of BI, the wholehearted endorsement of your business customers can be a the best indirect measurement of the success (or otherwise) of your work. I would recommend that fellow BI professionals pay close attention to this important indicator at all stages of their projects.
 
 


 
You can view some of the tweets about IRM(UK) DW/BI here, or here.
 
Disclosure: At the time of writing, System Dynamics is a business partner, but not in the field of business intelligence.
 

Another social media-inspired meeting

Lights, camera, action!

Back in June 2009, I wrote an article entitled A first for me. In this I described meeting up with Seth Grimes (@SethGrimes), an acknowledged expert in analytics and someone I had initially “met” via Twitter.com.

I have vastly expanded my network of international contacts through social media interactions such as these. Indeed I am slated to meet up with a few other people during November; a month in which I have a couple of slots speaking at BI/DW conferences (IRM later this week and Obis Omni towards the end of the month).

Another person that I became a virtual acquaintance of via social media is Bruna Aziza (@brunoaziza), Worldwide Strategy Lead for Business Intelligence at Microsoft. I originally “met” Bruno via LinkedIn.com and then also connected on Twitter.com. Later Bruno asked me for my thoughts on his article, Use Business Intelligence To Compete More Effectively, and I turned these into a blog post called BI and competition.

bizintelligence.tv - by Bruno Aziza of Microsoft

We have kept in touch since and last week Bruno asked me to be interviewed on the bizintelligence.tv channel that he is setting up. It was good to meet in person and I thought that we had some interesting discussions. Though I have done video and audio interviews before with organisations like IBM Cognos, Informatica, Computing Magazine and SmartDataCollective (see the foot of this article for links), these were mostly a while back and so it was interesting to be in front of a camera again.

The bizintelligence.tv format seems to be an interesting one, with key points in BI discussed in a focussed and punchy manner (not an approach that I am generally associated with) and a target audience of busy senior IT managers. As I have remarked elsewhere, it is also notable that the more foresighted of corporations are now taking social media seriously and getting quite good at engaging without any trace of hard selling; something that perhaps compromised the earlier efforts of some organisations in this area (for the avoidance of doubt, this is a general comment and not one levelled at Microsoft).

Bruno and I touched on a number of areas including, driving improvements in data quality, measuring the value of BI programmes, using historical data to justify BI investments (something that I am overdue writing about – UPDATE: now remedied here) and the cultural change aspect of BI. I am looking forward to seeing the results. Watch this space and in the meantime, take a look at some of the earlier interviews that Bruno has conducted.
 


 

Other video and audio interviews that I have recorded:

 

I will be speaking at the Obis Omni 2010 Forum

Obis Omni is a Business Intelligence community and resource with whom I have worked for several years. Since September 2008, I have set aside time to speak at their well-organised annual forums.

2010 is no exception and I will be attending this year’s event in London on 25 November at The Park Plaza Hotel.
 


 
Previous articles relating to Obis Omni events (and those of their sister organisation, The Cloud Circle) include:


 

I will be presenting at the IRM European Data Warehouse and Business Intelligence Conference

This IRM UK event will be taking place in central London from the 3rd to 5th November 2010. It is co-located with two other related IRM conferences: Data Management and Information Quality. Full details may be obtained from the IRM conference web-site here.
 

tweet this Tweet this article on twitter.com
Bookmark this article with:
| Facebook | del.icio.us | digg | Reddit | Stumble