This IRM UK event will be taking place in central London from the 3rd to 5th November 2010. It is co-located with two other related IRM conferences: Data Management and Information Quality. Full details may be obtained from the IRM conference web-site here.
 |
Tweet this article on twitter.com |
| Bookmark this article with: | |||||
|  Facebook |
|  del.icio.us |
|  digg |
|  Reddit |
|  Stumble |
|
2 Comments | 
business intelligence, data warehousing | Tagged: IRM UK | 
Permalink
Posted by Peter Thomas
While spring cleaning at home at the weekend, I came across a DVD of an interview I did for Informatica back in March 2005. This is still accessible on the Informatica web-site and appears in my video library, but I thought that I had lost my copy of the original.
Having made this discovery, I added it to my selection of videos on YouTube.com.
In the interview I stress the need for consistency in management information; the dynamics of the Insurance industry and the business value added by Business Intelligence in a pan-European insurance organisation.
|
Tweet this article on twitter.com |
| Bookmark this article with: | |||||
| | Facebook |
| del.icio.us |
| digg |
| Reddit |
| Stumble |
|
1 Comment | business intelligence, data warehousing, informatica | Tagged: data integration, video | Permalink
Posted by Peter Thomas
With enormous apologies to Randall Munroe of xkcd.com fame; from whose much funnier, and obviously more original, sketch entitled “GOTO” the above was shamelessly adapted.
|
Tweet this article on twitter.com |
| Bookmark this article with: | |||||
| | Facebook |
| del.icio.us |
| digg |
| Reddit |
| Stumble |
|
1 Comment | business intelligence, data warehousing | Tagged: business logic, xkcd.com | Permalink
Posted by Peter Thomas
“Just because Jeffrey Archer exists, it doesn’t follow that Joseph Conrad can’t have existed”
 |
 |
| Jeffrey Archer | Joseph Conrad |
Introduction
The context of the above bon mot was – as is often the case – a discussion on LinkedIn.com. I have been rather absent from the LinkedIn.com discussion groups for the same reasons that I have not been blogging and tweeting. In this case, my attention was drawn to the debate by a colleague.
|
CIOs.com: Chief Information Officer Network | ||
The particular thread was posted by Andy McKnight and is entitled What’s missing from Business Intelligence? and at the time of writing has attracted nearly 60 responses (you have to be a member of the group to view the discussion). It referred to an article published by EMC2 which has the strap-line How CIOs can Reap the Benefits of BI Technology (note: this is a PDF document). Here is a pertinent quote:
The bad news is that only twenty-seven percent of respondents [to a survey of CIOs carried out by IDG Research] who use a BI solution report being extremely successful or very successful with it. Forty-five percent report being only somewhat successful, while seventeen percent say that they are not very, or not at all successful.
I’m not sure what happened to the other 11% of respondents, maybe they just hung up the ‘phone.
Blaming the users
!["Users are the root of all evil" - anonymous [failed] BI Project Manager](http://peterthomas.files.wordpress.com/2010/03/down-with-users.jpg?w=200&h=288 "\"Users are the root of all evil\" - anonymous [failed] BI Project Manager")
Having stated that “BI has, too often, not lived up to expectations”, the paper goes on to list some reasons why. First on the list is the following:
You don’t have to be Einstein to realise that this is the result of a BI project failing, not the cause of it. The equivalent in athletics terms would be to say that you came last in the race because everyone else was faster than you. While obviously true this observation doesn’t help a lot with how to do better next time.
Of course hidden in the comment is the plaintive whine heard emanating from many an unsuccessful project (or indeed product launch), “the problem is the users”. This is arrant nonsense, returning to the start of article if you write a book that is panned by the critics and not bought by the public, then there is at least some chance that the fault lies with you and not them. It is the job of the IT professional to know their users, understand their needs and provide systems that cause delight, not disillusion.
A more interesting observation later on is:
Many BI initiatives falter because the analytics capabilities that are at the core of the system aren’t even used. Many users simply pull data from the warehouse and dump it in a spreadsheet. [...] A true BI implementation includes both reporting and analytics. CIOs indicate a much higher success rate with BI when users embrace both.
I think that there is some truth in this. Some of the BI failures I have seen have gone to the bother of building a warehouse only to front it with flat reports that are only marginally better than what they replaced.
In my career I have taken the opposite approach. While many people warn against analysis paralysis, I have deployed OLAP tools to all users, with fixed format reports de-emphasised, or used mostly for external purposes. This does mean that more effort needs to be put into training, but this is necessary anyway if you want your BI system to be an agent of change (and why else would you be building one if this is not the case?). I cover my general approach to driving user adoption in a series of three articles as follows:
This approach was very successful and we achieved user adoption of 92% – i.e. of those people who attended training, 92% remained active users (defined as using the BI system on average for at least two extended periods each week). We actually felt that the OLAP tools we were implementing were pretty intuitive and easy-to-use and so focussed mostly on how to use them in specific business scenarios. Overall we felt that training was 25% technical and 75% business-related.
Aiming for simplicity
Related to the above point, the EMC2 article also mentions the following reason for failure:
This seems a little oxymoronic as normally it is depth of functionality that confuses people. I think I would disagree with both parts of this point. Out of the box, most BI tools have rich functionality and a reasonably intuitive to use. In one response to the LinkedIn.com thread I said the following:
I have been successful in getting users [...] weaned [...] off ad hoc reports, it wasn’t an easy process and required persistence and selling, but this paid off. [...] It is illuminating seeing business managers (some of whom still dictate memos for their secretaries to type) “slicing and dicing”, drilling down/through and generally interacting away merrily and stating that if all IT was this easy to use and informative, they might have taken to it earlier.
My view here is that you can make the tool as complicated or a simple as you choose. Going back to my first warehouse project, in our somewhat naive early attempts at prototype cubes, we had all available dimensions and all available measures included. I think our idea is that the users could help us sift out the ones that were most important. Instead this approach caused the negative reactions that the article refers to.
We subsequently adopted a rule of having as few dimensions and measures as possible in a cube, without compromising the business need that the cube was trying to address. The second part of this rule was that every cube had to be focussed on answering business questions in at least one area and at most two.
Rather than having a small number of monolithic cubes, we went with the option of a slightly larger number of significantly clearer and simpler ones. I think that this was a factor in our success in driving business adoption.
Should the fact that some BI projects fail dissuade you from BI?
I won’t attempt to dissect the rest of the article, the areas that I comment on above are representative. There are some good points and some less good ones – just like any article, including of course my own. Take a look yourself and see whether the findings and recommendations chime with your own experience of success and failure. What I did want to do was to return to the context of the aphorism that starts this post.
The thesis of the original LinkedIn.com post was that because a significant number of organisations had failed to get enormous benefit from BI, BI itself was therefore somehow flawed. I think this is wrong-headed reasoning. If 1,000 people write a book, how many are likely to become acknowledged as great authors? How many are likely to have the lesser accolade of commercial success? The answer in both cases is “not many”. This is because writing well is a very difficult thing to do (I prove this myself with every blog post!). Not everyone who tries it will be successful. BI is also difficult to do well and a major cause of problems is underestimating this difficulty.
Maybe this is too recherché and example, and maybe if the chances of success with BI are as slim as winning the Purlitzer Prize then it is not worth the effort. So I’ll instead I’ll resort to my favourite area of the sporting analogy. Let’s take the same 1,000 people and say that they all take up a new sport – it is mostly immaterial what the sport is, let’s say tennis. How many of them will go on to become proficient in it? By this I don’t mean that they are the next Roger Federer, just that they become competent enough to serve adequately, master the dark arts of the backhand and sustain a few rallies. My feeling is that the stats would look something like those in the EMC2 report.
Is the prize worth it?
Given this, does it mean that some companies are just not cut out for BI and should ignore the area? Well the answer is “it depends”. Going back to tennis, if some one wants to be good, and has the determination to succeed, that is a necessary (though sadly not sufficient) condition. What may drive such a person on is the objective of achieving a goal, or maybe the pleasure of being able to perform at a certain level.
Focussing on business outcomes, I believe that BI can deliver substantial benefits. In fact I have argued elsewhere that BI can have the greatest payback of any IT project. Of course this presupposes that the BI project is done well. If the prize is potentially that great then maybe – like the aspiring tennis player who wants to become better – trying again makes sense. In recent recruitment I have heard frequent mention of organisations that were building their second warehouse as they didn’t get the first one quite right.
However the comparison with tennis breaks down in that business is a team game. If an organisation as a whole has struggled with BI, then this is not a question of simply accepting your genetic limitations. Companies can “evolve” capabilities by hiring people who have been successful in a field. They can also get benefit from targeted consultancy from practitioners who have a track record of success; this can help them to build an internal capability. This is an approach that I took advantage of myself in the initial six months of my first BI project [note: although I often seem to get mistaken for a BI consultant, I am not touting for business here!].
This means that if a company’s BI architecture is currently the equivalent of a Jeffrey Archer novel, it is still possible to transform it into Heart of Darkness. It will not be easy and will take time and effort, but there are people out there who have been successful and can act as guides.
In closing I should also mention that, if you take appropriate precautions, it is far from inevitable that the end of a BI journey will be finding your own version of Kurtz!
|
Tweet this article on twitter.com |
| Bookmark this article with: | |||||
| | Facebook |
| del.icio.us |
| digg |
| Reddit |
| Stumble |
|
4 Comments | business intelligence, change management, cultural transformation, data warehousing, education, management information | Tagged: emc, heart of darkness, jeffrey archer, joseph conrad | Permalink
Posted by Peter Thomas
I have been a featured blogger on SmartDataCollective.com almost as long as I have been a blogger. SDC.com is Social Media Today’s community site, focussed on all aspects of Business Intelligence, Data Warehousing and Analytics, with a pinch of social media thrown in to the mix.
Brian Roger, the SDC.com editor, was recently kind enough to interview me about my career in BI, the challenges I have faced and what has helped to overcome these. This interview is now available to listen to as part of their Podcast series – click on the image below to visit their site.
SmartDataCollective.com Intervew
I would be interested in feedback about any aspect of this piece, which I am grateful to Brian for arranging.
Social Media Today LLC, is a battle-tested, nimble team of former journalists, online managers, and advertising professionals who have come together to make a new kind of media company. With their backgrounds, and passions for, business-to-business and public policy conversations, they have decided to focus their efforts in this area. To facilitate the types of convresations that they would like to see Social Media Today is assembling the world’s best bloggers and providing them with an independent “playground” to include their posts, to comment and rate posts, and to connect with each other. On their flagship site, SocialMediaToday.com, they have brought together many of the most intriguing and original bloggers on media and marketing, covering all aspects of what makes up the connective tissue of social media from a global perspective.
|
Tweet this article on twitter.com |
| Bookmark this article with: | |||||
 Technorati |
| del.icio.us |
| digg |
| Reddit |
| Stumble |
|
5 Comments | business, business analytics, business intelligence, change management, cultural transformation, data warehousing, management, management information, social media, technology | Tagged: bi, brian roger, information technology, it business alignment, SmartDataCollective.com, SocialMediaToday.com | Permalink
Posted by Peter Thomas
Rather unsurprisingly, this article follows on from: Using multiple business intelligence tools in an implementation – Part I.
On further reflection about this earlier article, I realised that I missed out one important point. This was perhaps implicit in the diagram that I posted (and which I repeat below), but I think that it makes sense for me to make things explicit.
An example of a multi-tier BI architecture with different tools
The point is that in this architecture with different BI tools in different layers, it remains paramount to have consistency in terminology and behaviour for dimensions and measures. So “Country” and “Profit” must mean the same things in your dashboard as it does in your OLAP cubes. The way that I have achieved this before is to have virtually all of the logic defined in the warehouse itself. Of course some things may need to be calculated “on-the-fly” within the BI tool, in this case care needs to be paid to ensuring consistency.
It has been pointed out that the approach of using the warehouse to drive consistency may circumscribe your ability to fully exploit the functionality of some BI tools. While this is sometimes true, I think it is not just a price worth paying, but a price that it is mandatory to pay. Inconsistency of any kind is the enemy of all BI implementations. If your systems do not have credibility with your users, then all is already lost and no amount of flashy functionality will save you.
|
Tweet this article on twitter.com |
| Bookmark this article with: | |||||
| Technorati |
| del.icio.us |
| digg |
| Reddit |
| Stumble |
|
3 Comments | business, business intelligence, dashboards, data warehousing, management information, technology | Tagged: bi, business intelligence, information technology, it business alignment, it management, it projects, it strategy, management information | Permalink
Posted by Peter Thomas
Introduction
In a previous article, A more appropriate metaphor for business intelligence projects, I explained one complication of business intelligence projects. This is that the frequently applied IT metaphor of building is not very applicable to BI. Instead I suggested that BI projects had more in common with archaeological digs. I’m not going to revisit the reasons for the suitability of looking at BI this way here, take a look at the earlier piece if you need convincing, instead I’ll focus on what this means for project estimation.
When you are building up, estimation is easier because each new tier is dependent mostly on completion of the one below, something that the construction team has control over (note: for the sake of simplicity I’m going to ignore the general need to dig foundations for buildings). In this scenario, the initial design will take into account of facts such as the first tier needing to support all of the rest of the floors and that central shafts will be needed to provide access and deliver essential services such as water, electricity and of course network cables. A reductionist approach can be taken, with work broken into discrete tasks, each of which can be estimated with a certain degree of accuracy. The sum of each of these, plus some contingency, hopefully gives you a good feel for the overall project. It is however perhaps salutary to note that even when building up (both in construction and in IT) estimation can still sometimes go spectacularly awry.
When you are digging down, your speed is dependent on what you find. Your progress is dictated by things that are essentially hidden before work starts. If your path ahead (or downwards) is obscured until your have cleared enough earth to uncover the next layer, then each section may hold unexpected surprises and lead to unanticipated delays. While it may be possible to say things like, “well we need to dig down 20m and each metre should take us 10 days”, any given metre might actually take 20 days, or more. There are two issues here; first it is difficult to reduce the overall work into tasks, second it is harder to estimate each task accurately. The further below ground a phase of the dig is, the harder it will be to predict what will happen before ground is broken. Even with exploratory digs, or the use of scanning equipment, this can be very difficult to assess in advance. However it is to the concept of exploratory digs that this article is devoted.
Why a feasibility study is invaluable
At any point in the economic cycle, even more so in today’s circumstances, it is not ideal to tell your executive team that you have no idea how long a project will take, nor how much it might cost. Even with the most attractive of benefits to be potentially seized (and it is my firm belief that BI projects have a greater payback than many other types of IT projects), unless there is some overriding reason that work must commence, then your project is unlikely to gain a lot of support if it is thus characterised. So how to square the circle of providing estimates for BI projects that are accurate enough to present to project sponsors and will not subsequently leave you embarrassed by massive overruns?
It is in addressing this issue that BI feasibility studies have their greatest value. These can be thought of as analogous to the exploratory digs referred to above. Of course there are some questions to be answered here. By definition, a feasibility study cannot cover all of the ground that the real project needs to cover, choices will need to be made. For example, if there are likely to be 10 different data sources for your eventual warehouse, then should you pick one and look at it in some depth, or should you fleetingly examine all 10 areas? Extending our archaeological metaphor, should your exploratory dig be shallow and wide, or a deep and narrow borehole?
A centre-centric approach
In answering this question, it is probably worth considering the fact that not all data sources are alike. There is probably a hierarchy to them, both in terms of importance and in terms of architecture. No two organisations will be the same, but the following diagram may capture some of what I mean here:
Two ways of looking at a systems' hierarchy
The figure shows a couple of ways of looking at your data sources / systems. The one of the left is rather ERP-centric, the one on the right gives greater prominence to front-end systems supporting different lines of business, but wrapped by a common CRM system. There are many different diagrams that could be drawn in many different ways of course. My reason for using concentric circles is to stress that there is often a sense in which information flows from the outside systems (ones primarily focussed on customer interactions and capturing business transactions) to internal systems (focussed on either external or internal reporting, monitoring the effectiveness of processes, or delivering controls).
There may be several layers through which information percolates to the centre; indeed the bands of systems and databases might be as numerous as rings in an onion. The point is that there generally is such a logical centre. Data is often lost on its journey to this centre by either aggregation, or by elements simply not being transferred (e.g. the name of a salesperson is not often recorded on revenue entries in a General Ledger). Nevertheless the innermost segment of the onion is often the most complex, with sometimes arcane rules governing how data is consolidated and transformed on its way to its final destination.
The centre in both of the above diagrams is financial and this is not atypical if what we are considering is an all-pervasive BI system aimed at measuring most, if not all, elements of an organisation’s activity (the most valuable type of BI system in my opinion). Even if your BI project is not all-pervasive (or at least the first phase is more specific), then the argument that there is a centre will probably still hold, however the centre may not be financial in this case.
My suggestion is that this central source of data (of course there may be more than one) is what should be the greatest focus of your feasibility study. There are several reasons for this, some technical, some project marketing-related:
A possible result of building from the outside in to the centre
So my recommendation is that your feasibility study is mostly a narrow, deep dig, focussed on the central data source. If time allows it would be beneficial to supplement this with a more cursory examination of some of the data sources that feed the centre, particularly as this may be necessary to better understand the centre and because it will help you to get a better idea about your overall information architecture. You do not need to figure out every single thing about the central data source, but whatever you can find out will improve the accuracy of your estimate and save you time later. If you include other data sources in a deep / wide hybrid, then these can initially be studied in much less detail as they are often simpler and the assumption is that they will support later deliveries.
The idea of a prototype was mentioned above. This is something that is very important to produce in a feasibility study. Even if we take to one side the undeniable PR value of a prototype, producing one will allow you to go through the entire build process. Even if you do this with hand-crafted transformation of data (rather than ETL) and only a simplistic and incomplete approach to the measures and dimensions you support, you will at least have gone through each of the technical stages required in the eventual live system. This will help to shake out any issues, highlight areas that will require further attention and assist in sizing databases. A prototype can also be used to begin to investigate system and network performance, things that will influence your system topology and thereby project costs. A better appreciation of all of these areas will help you greatly when it comes to making good estimates.
Having understood quite a lot about your most complex data source and a little about other ones and produced a prototype both as a sales tool and to get experience of the whole build process, you should have all the main ingredients for making a credible presentation to your project sponsors. In this it is very important to stress the uncertainties inherent in BI and manage expectations around these. However you should also be very confident in stating that you have done all that can be done to mitigate the impact of these. This approach, of course supported by a compelling business case, will position you very well to pitch your overall BI project.
|
Tweet this article on twitter.com |
| Bookmark this article with: | |||||
| Technorati |
| del.icio.us |
| digg |
| Reddit |
|  NewsVine |
|
3 Comments | bi and the economic crisis, business, business intelligence, data warehousing, management information, project management, technology | Tagged: bi, feasibility study, information architecture, it strategy, methodology, success factors | Permalink
Posted by Peter Thomas
Today I read an article on The Register by Gavin Clarke. This was about Microsoft’s potential response to Oracle’s proposed acquisition of Sun Microsystems and was entitled (rather cryptically in my opinion) Microsoft’s DNA won’t permit Oracle-Sun deal – Ballmer knows his knitting.
Gavin quotes Steve Ballmer, Microsoft CEO, as saying that his corporation will be “sticking to the knitting” in response to Oracle‘s swoop on Sun. He goes on to cover some aspects of the Oracle / Sun link-up; specifically referring to the idea of “BI in a box” that seems to be gaining credence as one rationale for the deal. In his words, this trend is about:
storing, serving, and understanding information [...]: the trend for getting fast access to huge quantities of data on massive networks and making sense of it.
However mention is then made of co-offerings that Oracle and HP have teamed up to make in this space – surely something that would be potentially jeopardised by the Sun acquisition:
Oracle last year announced the HP Oracle Exadata Storage Server and HP Oracle Database Machine, a box from Hewlett-Packard featuring a stack of pre-configured Exadata Storage Servers all running Oracle’s database and its Enterprise Linux.
Returning to Microsoft’s response, the article stresses their modus operandi of focussing on software components and then collaborating with others on hardware. Refernce is also made to Kilimanjaro, Microsoft’s forthcoming SQL Server version that will further emphasise business intelligence capabilities.
In closing Gavin states that:
Acquisition of a hardware company would break the DNA sequence and fundamentally change Microsoft in the way that owning Sun’s hardware business will change Oracle.
It’s tempting to note that DNA is broken (and then recombined) millions of times by RNA Polymerase, that is after all how proteins are synthesised in cells; one characteristic of Microsoft’s success (notwithstanding its recent announcement of its first ever dip in sales) has been a willingness to reinvent parts of its business (else where did the XBox come from), while relying on a steady income stream from others. When it comes to the idea of Microsoft acquiring a major hardware vendor, I agree it seems far-fetched at present, but never say never.
|
Tweet this article on twitter.com |
| Bookmark this article with: | |||||
| Technorati |
| del.icio.us |
| digg |
| Reddit |
| NewsVine |
|
Leave a Comment » | business, business intelligence, data warehousing, microsoft, oracle, oracle and sun, sun | Tagged: bi, business intelligence, hp, information technology, Kilimanjaro, SQL Server | Permalink
Posted by Peter Thomas
Some of the furore following on from the announcement of the proposed acquisition of Sun Microsystems by Oracle appears to have died down today. However, taking a look round the blogosphere and various on-line discussion forums1, there does not seem to be much of a consensus about Oracle’s motivations, or future plans for Sun. There are a number of moving parts to this:
One area that people seem agreed upon is the importance of Java to Oracle’s application strategy, so it makes sense – as a defensive move if nothing else – for them to seek to prevent influence over its future direction falling into the hands of a competitor (which in turn raises the question of when exactly Oracle and Sun started talking and how much overlap there was with the IBM negotiations).
The future of MySQL seems less clear. Some commentators feel that Oracle will support it and allow it to continue to thrive as one of their products. At the other extreme, I have seen suggestions that it will be killed off. Of course as an open source database, this might be easier said than done. There seems to have been a steady trickle of MySQL people out of Sun, pre-acquisition and I would have thought that there is enough expertise and ownership outside of Oracle/Sun for MySQL to have some sort of future regardless of Oracle’s strategy for it.
A bit of a dark horse is OpenOffice.org. A lot of commentary has focused on Oracle positioning themselves to compete with IBM via the acquisition. Perhaps OpenOffice.org offers Larry Ellison another chance to cross swords with his old adversaries at Microsoft.
Moving from software to operating systems, Sun’s Solaris has probably suffered more than most from the rise of Linux, but there have been rumours about Solaris offering Oracle a better route to the current technology Nirvana of cloud computing. Whether this is really the case, I’ll leave to more technically competent authorities to discuss.
But beneath Solaris beats the SPARC chips and other components of Sun’s hardware. Is Oracle’s real aim to offer a complete solution: ERP, CRM, BI and DW in a box? Sun’s hardware has not exactly been flying off the shelf in recent months, but perhaps the sales team at Oracle have other ideas. Maybe their feeling is that all that Sun’s boxes need is to be part of a more alluring overall package. Leveraging Sun’s hardware and operating system is what many people assume is behind Oracle’s strategy. This is certainly the path that would lead to challenging IBM as a company that can meet many of an organisation’s needs as a one-stop-shop.
However, this segues into another observation. If Oracle really has IBM in its sights, then it lacks one crucial piece of ammunition, a global services organisation; the sort of outfit that IBM acquired from the hiving off of PwC’s consulting arm. Maybe now is a good time to but stock in CSC?
But to return to some of the points I made earlier, there is a further possibility. Perhaps Oracle don’t want to move into the fiercely competitive and low-margin arena of hardware sales after all. Perhaps it was Sun’s software assets that were the real goal. Does Oracle really want to position itself as a hardware vendor, no doubt poisoning strong relationships with people such as HP in the process? Maybe not. If this is indeed the case then maybe there will be a spin-off of Sun’s hardware assets, or indeed a sale to someone like HP – assuming that they wanted them.
One of the most intriguing aspects of Oracle’s proposed acquisition of Sun is just how many balls have been thrown up into the air by it. It will be really interesting to see how they fall over the next few months.
| 1. | Some of the blogs that I have read on this subject are acknowledged at the end of my earlier article.
A further main source has been comments on various LinkedIn.com groups, notably: CIO Forum (CIO.com and CIO Magazine), CIOs.com: Chief Information Officer Network and The IT Architect Network. As always, membership of LinkedIn.com and the group is required to view these. Finally, you can sometimes glean a lot from 140 characters, so various comments on Twitter have also been influential. |
|
Tweet this article on twitter.com |
| Bookmark this article with: | |||||
| Technorati |
| del.icio.us |
| digg |
| Reddit |
| NewsVine |
|
5 Comments | business, business intelligence, data warehousing, ibm, microsoft, oracle, oracle and sun, sun, technology | Tagged: bi, business intelligence, CSC, hp, information technology, java, mysql, openoffice.org, PwC, solaris | Permalink
Posted by Peter Thomas
You are currently browsing the archives for the data warehousing category.
 Facebook |
 del.icio.us |
 digg |
 Reddit |
 Stumble |
Using multiple business intelligence tools in an implementation – Part I
16 May 2009Introduction
This post follows on from a question that was asked on the LinkedIn.com Data Warehousing Institute (TDWI™) 2.0 group. Unfortunately the original thread is no longer available for whatever reason, but the gist of the question was whether anyone had experience with using a number of BI tools to cover different functions within an implementation. So the scenario might be: Tool A for dashboards, Tool B for OLAP, Tool C for Analytics, Tool D for formatted reports and even Tool E for visualisation.
In my initial response I admitted that I had not faced precisely this situation, but that I had worked with the set-up shown in the following diagram, which I felt was not that dissimilar:
An example of a multi-tier BI architecture with different tools
Here there is no analytics tool (in the statistical modelling sense – Excel played that role) and no true visualisation (unless you count graphs in PowerPlay that is), but each of dashboards, OLAP cubes, formatted reports and simple list reports are present. The reason that this arrangement might not at first sight appear pertinent to the question asked on LinkedIn.com is that two of the layers (and three of the report technologies) are from one vendor; Cognos at the time, IBM-Cognos now. The reason that I felt that there was some relevance was that the Cognos products were from different major releases. The dashboard tool being from their Version 8 architecture and the OLAP cubes and formatted reports from their Version 7 architecture.
A little history
London Bridge circa 1600
Maybe a note of explanation is necessary as clearly we did not plan to have this slight mismatch of technologies. We initially built out our BI infrastructure without a dashboard layer. Partly this was because dashboards weren’t as much of a hot topic for CEOs when we started. However, I also think it also makes sense to overlay dashboards on an established information architecture (something I cover in my earlier article, “All that glisters is not gold” – some thoughts on dashboards, which is also pertinent to these discussions).
When we started to think about adding icing to our BI cake, ReportStudio in Cognos 8 had just come out and we thought that it made sense to look at this; both to deliver dashboards and to assess its potential future role in our BI implementation. At that point, the initial Cognos 8 version of Analysis Studio wasn’t an attractive upgrade path for existing PowerPlay users and so we wanted to stay on PowerPlay 7.3 for a while longer.
The other thing that I should mention is that we had integrated an in-house developed web-based reporting tool with PowerPlay as the drill down tool. The reasons for this were a) we had already trained 750 users in this tool and it seemed sensible to leverage it and b) employing it meant that we didn’t have to buy an additional Cognos 7 product, such as Impromptu, to support this need. This hopefully explains the mild heterogeneity of our set up. I should probably also say that users could directly access any one of the BI tools to get at information and that they could navigate between them as shown by the arrows in the diagram.
I am sure that things have improved immensely in the Cognos toolset since back then, but at the time there was no truly seamless integration between ReportStudio and PowerPlay as they were on different architectures. This meant that we had to code the passing of parameters between the ReportStudio dashboard and PowerPlay cubes ourselves. Although there were some similarities between the two products, there were also some differences at the time and these, plus the custom integration we had to develop, meant that you could also view the two Cognos products as essentially separate tools. Add in here the additional custom integration of our in-house reporting application with PowerPlay and maybe you can begin to see why I felt that there were some similarities between our implementation and one using different vendors for each tool.
I am going to speak a bit about the benefits and disadvantages of having a single vendor approach later, but for now an obvious question is “did our set-up work?” The answer to this was a resounding yes. Though the IT work behind the scenes was maybe not the most elegant (though everything was eminently supportable), from the users’ perspective things were effectively seamless. To slightly pre-empt a later point, I think that the user experience is what really matters, more than what happens on the IT side of the house. Nevertheless let’s move on from some specifics to some general comments.
The advantages of a single vendor approach to BI
One-stop shopping
I think that it makes sense if I lay my cards on the table up-front. I am a paid up member of the BI standardisation club. I think that you only release the true potential of BI when you take a broad based approach and bring as many areas as you can into your warehouse (see my earlier article, Holistic vs Incremental approaches to BI, for my reasons for believing this).
Within the warehouse itself there should be a standardised approach to dimensions (business entities and the hierarchies they are built into should be the same everywhere – I’m sure this will please all my MDM friends out there) and to measures (what is the point if profitability is defined different ways in different reports?). It is almost clichéd nowadays to speak about “the single version of the truth”, but I have always been a proponent of this approach.
I also think that you should have the minimum number of BI tools. Here however the minimum is not necessarily always one. To misquote one of Württemberg’s most famous sons:
What he actually said was:
but maybe the common rendition is itself paying tribute to the principle that he propounded. Let me pause to cover what are the main reasons quoted for adopting a single vendor approach in BI:
This all seems to make perfect sense and each of the above points can be seen to be reducing the complexity and cost of your BI solution. Surely it is a no-brainer to adopt this approach? Well maybe. Let me offer some alternative perspectives on each item – none of these wholly negates the point, but I think it is nevertheless worth considering a different perspective before deciding what is best for your organisation.
A more important consideration
Overall it is still likely that a single-vendor solution is cheaper than a multi-vendor one, but I hope that I have raised enough points to make you think that this is not guaranteed. Also the cost differential may not be as substantial as might be thought initially. You should certainly explore both approaches and figure out what works best for you. However there is another overriding point to consider here, the one I alluded to earlier; your users. The most important thing is that your users have the best experience and that whatever tools you employ are the ones that will deliver this. If you can do this while sticking to a single vendor then great. However if your users will be better served by different tools in different tiers, then this should be your approach, regardless of whether it makes things a bit more complicated for your team.
Of course there may be some additional costs associated with such an approach, but I doubt that this issue is insuperable. One comparison that it may help to keep in mind is that the per user cost of many BI tools is similar to desktop productivity tools such as Office. The main expense of BI programmes is not the tools that you use to deliver information, but all the work that goes on behind the scenes to ensure that it is the right information, at the right time and with the appropriate degree of accuracy. The big chunks of BI project costs are located in the four pillars that I consistently refer to:
The cost of the BI tools themselves are only a minor part of the above (see also, BI implementations are like icebergs). Of course any savings made on tools may make funds available for other parts of the project. It is however important not to cut your nose off to spite your face here. Picking right tools for the job, be they from one vendor or two (or even three at a push) will be much more important to the overall payback of your project than saving a few nickels and dimes by sticking to a one-vendor strategy just for the sake of it.
Continue reading about this area in: Using multiple business intelligence tools in an implementation – Part II