Is the time ripe for appointing a Chief Business Intelligence Officer?

linkedin Business Intelligence Business Intelligence

Once more I have decided to pen this article based on a question that was raised on The group in question on this occasion was Business Intelligence and the thread was entitled Is it time that the CBIO (Chief Business Intelligence Officer) position and organization become commonplace in today’s corporate structure? This was posted by John Thielman.

Standard note: You need to be a member of both and the group mentioned to view the discussions.
The case for a CBIO

The Office of the CBIO

I won’t republish all of John’s initial post, but for those who cannot access the thread these are the essential points that he raised:

  1. There is an ever-increasing need for more and better information in organisations
  2. Increasingly Business Intelligence is seen as a major source of competitive advantage
  3. A CBIO would bring focus and (more importantly) accountability to this area
  4. The CBIO should report directly to the CEO, with strong relations with the rest of the executive team
  5. The CBIO’s team would be a hybrid business / technical one (as I strongly believe the best BI teams should be)
  6. This team should also be at the forefront of driving change, based on the metrics that it generates

Now obviously creating a senior role with a portfolio spanning BI and change is going to be music that falls sweetly on my ears. I did however attempt to be objective in my response, which I reproduce in full below:

As someone who is (primarily) a BI professional, then of course my response could be viewed as entirely self-serving. Nevertheless, I’ll offer my thoughts.

In the BI programmes that I have run, I have had reporting lines into people such as the CIO, CFO or sometimes a combined IT / Operations lead. However (and I think that this is a big however), I have always had programme accountability to the CEO and have always had the entire senior leadership team (business and service departments) as my stakeholders. Generally my direction has come more from these dotted lines than from the solid ones – as you would hope would be the case in any customer-centric IT area.

I have run lots of different IT projects over the years. Things such as: building accounting, purchasing and sales systems; configuring and implementing ERP systems; building front-end systems for underwriters, marketing and executive teams; and so on. Given this background, there is definitely something about BI that makes it different.

Any IT system must be aligned to its users’ needs, that much is obvious. However with BI it goes a long way beyond alignment. In a very real sense, BI systems need to be the business. They are not there to facilitate business transactions, they are there to monitor the heartbeat of the organisation, to help it navigate the best way forward, to get early warning of problems, to check the efficacy of strategies and provide key input to developing them.

In short a good BI system should be focussed on precisely the things that the senior leadership team is focussed on, and in particular what the CEO is focussed on. In order to achieve this you need to understand what makes the business tick and you need to move very close to it. This proximity, coupled with the fact that good BI should drip business value means that I have often felt closer to the overall business leadership team than the IT team.

Please don’t misunderstand my point here. I have been an IT person for 20 years and I am not saying that BI should not be fully integrated with the overall IT strategy – indeed in my book it should be central to it as a major function of all IT systems is to gather information (as well as to support transactions and facilitate interactions with customers). However, there is something of a sense in which BI straddles the IT and business arenas (arenas that I have long argued should be much less distinct from each other than they are in many organisations).

The potentially massive impact of BI, the fact that it speaks the language of business leaders, the need for it to be aligned with driving cultural change and that the fact that the skills required for success in BI are slightly different for those necessary in normal IT projects all argue that something like a CBIO position is maybe not such a bad idea.

Indeed I have begun to see quite a few BI roles that are part of change directorates, or the office of the CEO or CFO. There are also some stand-alone BI roles out there, reporting directing to the board. Clearly there will always be a strong interaction with IT, but perhaps you have detected an emerging trend.

I suppose a shorter version of the above would run something like: my de facto reporting line in BI programmes has always been into the CEO and senior management team, so why not recognise this by making it a de jura reporting line.

BI is a weird combination of being both a specialist and generalist area. Generalist in needing to play a major role in running all aspects of the business, specialist in the techniques and technologies that are key to achieving this.
Over to the jury

Maybe the idea of a CBIO is one whose time has come. I would be interested in people’s views on this.


Using multiple business intelligence tools in an implementation – Part II

Rather unsurprisingly, this article follows on from: Using multiple business intelligence tools in an implementation – Part I.

On further reflection about this earlier article, I realised that I missed out one important point. This was perhaps implicit in the diagram that I posted (and which I repeat below), but I think that it makes sense for me to make things explicit.

An example of a multi-tier BI architecture with different tools
An example of a multi-tier BI architecture with different tools

The point is that in this architecture with different BI tools in different layers, it remains paramount to have consistency in terminology and behaviour for dimensions and measures. So “Country” and “Profit” must mean the same things in your dashboard as it does in your OLAP cubes. The way that I have achieved this before is to have virtually all of the logic defined in the warehouse itself. Of course some things may need to be calculated “on-the-fly” within the BI tool, in this case care needs to be paid to ensuring consistency.

It has been pointed out that the approach of using the warehouse to drive consistency may circumscribe your ability to fully exploit the functionality of some BI tools. While this is sometimes true, I think it is not just a price worth paying, but a price that it is mandatory to pay. Inconsistency of any kind is the enemy of all BI implementations. If your systems do not have credibility with your users, then all is already lost and no amount of flashy functionality will save you.

Using multiple business intelligence tools in an implementation – Part I

linkedin The Data Warehousing Institute The Data Warehousing Institute (TDWI™) 2.0


This post follows on from a question that was asked on the Data Warehousing Institute (TDWI™) 2.0 group. Unfortunately the original thread is no longer available for whatever reason, but the gist of the question was whether anyone had experience with using a number of BI tools to cover different functions within an implementation. So the scenario might be: Tool A for dashboards, Tool B for OLAP, Tool C for Analytics, Tool D for formatted reports and even Tool E for visualisation.

In my initial response I admitted that I had not faced precisely this situation, but that I had worked with the set-up shown in the following diagram, which I felt was not that dissimilar:

An example of a multi-tier BI architecture with different tools
An example of a multi-tier BI architecture with different tools

Here there is no analytics tool (in the statistical modelling sense – Excel played that role) and no true visualisation (unless you count graphs in PowerPlay that is), but each of dashboards, OLAP cubes, formatted reports and simple list reports are present. The reason that this arrangement might not at first sight appear pertinent to the question asked on is that two of the layers (and three of the report technologies) are from one vendor; Cognos at the time, IBM-Cognos now. The reason that I felt that there was some relevance was that the Cognos products were from different major releases. The dashboard tool being from their Version 8 architecture and the OLAP cubes and formatted reports from their Version 7 architecture.
A little history

London Bridge circa 1600
London Bridge circa 1600

Maybe a note of explanation is necessary as clearly we did not plan to have this slight mismatch of technologies. We initially built out our BI infrastructure without a dashboard layer. Partly this was because dashboards weren’t as much of a hot topic for CEOs when we started. However, I also think it also makes sense to overlay dashboards on an established information architecture (something I cover in my earlier article, “All that glisters is not gold” – some thoughts on dashboards, which is also pertinent to these discussions).

When we started to think about adding icing to our BI cake, ReportStudio in Cognos 8 had just come out and we thought that it made sense to look at this; both to deliver dashboards and to assess its potential future role in our BI implementation. At that point, the initial Cognos 8 version of Analysis Studio wasn’t an attractive upgrade path for existing PowerPlay users and so we wanted to stay on PowerPlay 7.3 for a while longer.

The other thing that I should mention is that we had integrated an in-house developed web-based reporting tool with PowerPlay as the drill down tool. The reasons for this were a) we had already trained 750 users in this tool and it seemed sensible to leverage it and b) employing it meant that we didn’t have to buy an additional Cognos 7 product, such as Impromptu, to support this need. This hopefully explains the mild heterogeneity of our set up. I should probably also say that users could directly access any one of the BI tools to get at information and that they could navigate between them as shown by the arrows in the diagram.

I am sure that things have improved immensely in the Cognos toolset since back then, but at the time there was no truly seamless integration between ReportStudio and PowerPlay as they were on different architectures. This meant that we had to code the passing of parameters between the ReportStudio dashboard and PowerPlay cubes ourselves. Although there were some similarities between the two products, there were also some differences at the time and these, plus the custom integration we had to develop, meant that you could also view the two Cognos products as essentially separate tools. Add in here the additional custom integration of our in-house reporting application with PowerPlay and maybe you can begin to see why I felt that there were some similarities between our implementation and one using different vendors for each tool.

I am going to speak a bit about the benefits and disadvantages of having a single vendor approach later, but for now an obvious question is “did our set-up work?” The answer to this was a resounding yes. Though the IT work behind the scenes was maybe not the most elegant (though everything was eminently supportable), from the users’ perspective things were effectively seamless. To slightly pre-empt a later point, I think that the user experience is what really matters, more than what happens on the IT side of the house. Nevertheless let’s move on from some specifics to some general comments.
The advantages of a single vendor approach to BI

One-stop shopping
One-stop shopping

I think that it makes sense if I lay my cards on the table up-front. I am a paid up member of the BI standardisation club. I think that you only release the true potential of BI when you take a broad based approach and bring as many areas as you can into your warehouse (see my earlier article, Holistic vs Incremental approaches to BI, for my reasons for believing this).

Within the warehouse itself there should be a standardised approach to dimensions (business entities and the hierarchies they are built into should be the same everywhere – I’m sure this will please all my MDM friends out there) and to measures (what is the point if profitability is defined different ways in different reports?). It is almost clichéd nowadays to speak about “the single version of the truth”, but I have always been a proponent of this approach.

I also think that you should have the minimum number of BI tools. Here however the minimum is not necessarily always one. To misquote one of Württemberg’s most famous sons:

Everything should be made as simple as possible, but no simpler.

What he actually said was:

It can scarcely be denied that the supreme goal of all theory is to make the irreducible basic elements as simple and as few as possible without having to surrender the adequate representation of a single datum of experience.

but maybe the common rendition is itself paying tribute to the principle that he propounded. Let me pause to cover what are the main reasons quoted for adopting a single vendor approach in BI:

  1. Consistent look-and-feel: The tools will have a common look-and-feel, making it easier for people to use them and simplifying training.
  2. Better interoperability: Interoperability between the tools is out-of-the-box, saving on time and effort in developing and maintaining integration.
  3. Clarity in problem resolution: If something goes wrong with your implementation, you don’t get different vendors blaming each other for the problem.
  4. Simpler upgrades: You future proof your architecture, when one element has a new release, it is the vendor’s job to ensure it works with everything else, not yours.
  5. Less people needed: You don’t need to hire an expert for each different vendor tool, thereby reducing the size and cost of your BI team.
  6. Cheaper licensing: It should be cheaper to buy a bundled solution from one vendor and ongoing maintenance fees should also be less.

This all seems to make perfect sense and each of the above points can be seen to be reducing the complexity and cost of your BI solution. Surely it is a no-brainer to adopt this approach? Well maybe. Let me offer some alternative perspectives on each item – none of these wholly negates the point, but I think it is nevertheless worth considering a different perspective before deciding what is best for your organisation.

  1. Consistent look-and-feel: It is not always 100% true that different tools from the same vendor have the same look-and-feel. This might be down to quality control at the vendor, it might be because the vendor has recently acquired part of their product set and not fully integrated it as yet, or – even more basically – it may be because different tools are intended to do different things. To pick one example from outside of BI that has frustrated me endlessly over the years: PowerPoint and Word seem to have very little in common, even in Office 2007. Hopefully different tools from the same vendor will be able to share the same metadata, but this is not always the case. Some research is probably required here before assuming this point is true. Also, picking up on the Bauhaus ethos of form dictating function, you probably don’t want to have your dashboard looking exactly like your OLAP cubes – it wouldn’t be a dashboard then would it? Additional user training will generally be required for each tier in your BI architecture and a single-vendor approach will at best reduce this somewhat.
  2. Better interoperability: I mention an problem with interoperability of the Cognos toolset above. This is is hopefully now a historical oddity, but I would be amazed if similar issues do not arise at least from time to time with most BI vendors. Cognos itself has now been acquired by IBM and I am sure everyone in the new organisation is doing a fine job of consolidating the product lines, but it would be incredible if there were not some mismatches that occur in the process. Even without acquisitions it is likely that elements of a vendor’s product set get slightly out of alignment from time to time.
  3. Clarity in problem resolution: This is hopefully a valid point, however it probably won’t stop your BI tool vendor from suggesting that it is your web-server software, or network topology, or database version that is causing the issue. Call me cynical if you wish, I prefer to think of myself as a seasoned IT professional!
  4. Simpler upgrades: Again this is also most likely to be a plus point, but problems can occur when only parts of a product set have upgrades. Also you may need to upgrade Tool A to the latest version to address a bug or to deliver desired functionality, but have equally valid reasons for keeping Tool B at the previous release. This can cause problems in a single supplier scenario precisely because the elements are likely to be more tightly coupled with each other, something that you may have a chance of being insulated against if you use tools from different vendors.
  5. Less people needed: While there might be half a point here, I think that this is mostly fallacious. The skills required to build an easy-to-use and impactful dashboard are not the same as building OLAP cubes. It may be that you have flexible and creative people who can do both (I have been thus blessed myself in the past in projects I ran), but this type of person would most likely be equally adept whatever tool they were using. Again there may be some efficiencies in sharing metadata, but it is important not to over-state these. You may well still need a dashboard person and an OLAP person, if you don’t then the person who can do both with probably not care about which vendor provides the tools.
  6. Cheaper licensing: Let’s think about this. How many vendors give you Tool B free when you purchase Tool A? Not many is the answer in my experience, they are commercial entities after all. It may be more economical to purchase bundles of products from a vendor, but also having more than one in the game may be an even better way of ensuring that cost are kept down. This is another area that requires further close examination before deciding what to do.

A more important consideration

Overall it is still likely that a single-vendor solution is cheaper than a multi-vendor one, but I hope that I have raised enough points to make you think that this is not guaranteed. Also the cost differential may not be as substantial as might be thought initially. You should certainly explore both approaches and figure out what works best for you. However there is another overriding point to consider here, the one I alluded to earlier; your users. The most important thing is that your users have the best experience and that whatever tools you employ are the ones that will deliver this. If you can do this while sticking to a single vendor then great. However if your users will be better served by different tools in different tiers, then this should be your approach, regardless of whether it makes things a bit more complicated for your team.

Of course there may be some additional costs associated with such an approach, but I doubt that this issue is insuperable. One comparison that it may help to keep in mind is that the per user cost of many BI tools is similar to desktop productivity tools such as Office. The main expense of BI programmes is not the tools that you use to deliver information, but all the work that goes on behind the scenes to ensure that it is the right information, at the right time and with the appropriate degree of accuracy. The big chunks of BI project costs are located in the four pillars that I consistently refer to:

  1. Understand the important business decisions and what figures are necessary to support these.
  2. Understand the data available in the organisation, how it relates to other data and to business decisions.
  3. Transform the data to provide information answering business questions.
  4. Focus on embedding the use of information in the corporate DNA.

The cost of the BI tools themselves are only a minor part of the above (see also, BI implementations are like icebergs). Of course any savings made on tools may make funds available for other parts of the project. It is however important not to cut your nose off to spite your face here. Picking right tools for the job, be they from one vendor or two (or even three at a push) will be much more important to the overall payback of your project than saving a few nickels and dimes by sticking to a one-vendor strategy just for the sake of it.

Continue reading about this area in: Using multiple business intelligence tools in an implementation – Part II

“Why taking a few punches on the financial crisis just might save IT” by Patrick Gray on TechRepublic

linkedin TechRepublic

Back on April 27th I wrote an article, The scope of IT’s responsibility when businesses go bad, that was inspired by a thread that Patrick Gray had started on the Chief Information Officer (CIO) Network group. This was entitled Is IT partially to blame for the financial crisis? (as ever you need to be a member of and the group to view these links).

Since then, there have been nearly 80 comments made by a wide variety of people, with an equally wide range of opinions. As can often happen in on-line discussions, positions were taken, attitudes were hardened and eventually some sort of stalemate was reached; probably as the protagonists were too weary to fight any more. In this respect seasoned IT professionals can be no different to teenagers discussing the merits of different genres of music! I certainly employed my method acting approach at a new level on this thread.

As a result of the overall feedback, Patrick has now composed a blog article on, an outlet that has also featured one of my favourite technology writers, Ilya Bogorad (see this earlier blog post). Patrick’s piece is titled Why taking a few punches on the financial crisis just might save IT and takes a thought-provoking stance with respect to the comments that his thread engendered. Here are the introductory remarks:

Patrick Gray believes that IT leaders still looking to find a seat at the C-level table might gain that influential position by taking a share of the responsibility for the failures that led to financial crisis.

It is certainly worth reading this article, but I recommend that you do so with an open mind.

Patrick Gray is the founder and president of Prevoyance Group, and author of Breakthrough IT: Supercharging Organizational Value through Technology. Prevoyance Group provides strategic IT consulting services to Fortune 500 and 1000 companies. Patrick can be reached at

Maureen Clarry stresses the need for change skills in business intelligence on BeyeNetwork

The article


Maureen Clarry begins her latest BeyeNETWORK article, Leading Change in Business Intelligence, by stating:

If there was a standard list of core competencies for leaders of business intelligence (BI) initiatives, the ability to manage complex change should be near the top of the list.

I strongly concur with Maureen’s observation and indeed the confluence of BI and change management is a major theme of this blog; as well as the title of one of my articles on the subject. Maureen clearly makes the case that “business intelligence is central to supporting […] organizational changes” and then spends some time on Prosci’s ADKAR model for leading change; bringing this deftly back into the BI sphere. Her closing thoughts are that such a framework can help a lot in driving the success of a BI project.
My reflections

I find it immensely encouraging that an increasing number of BI professionals and consultants are acknowledging the major role that change plays in our industry and in the success of our projects. In fact it is hard to find some one who has run a truly successful BI project without paying a lot of attention to how better information will drive different behaviour – if it fails to do this, then “why bother?” as Maureen succinctly puts it.

Without describing it as anything so grand as a framework, I have put together a trilogy of articles on the subject of driving cultural transformation via BI. These are as follows:

Marketing Change
Education and cultural transformation
Sustaining Cultural Change

However the good news about many BI professionals and consultants embracing change management as a necessary discipline does not seem to have filtered through to all quarters of the IT world. Many people in senior roles still seem to see BI as just another technology area. This observation is born out of the multitude of BI management roles that request an intimate knowledge of specific technology stacks. These tend to make only a passing reference to experience of the industry in question and only very infrequently mention the change management aspects of BI at all.

Of course there are counterexamples, but the main exceptions to this trend seem to be where BI is part of a more business focused area, maybe Strategic Change, or the Change Management Office. Here it would be surprising if change management skills were not stressed. When BI is part of IT it seems that the list of requirements tends to be very technology focussed.

In an earlier article, BI implementations are like icebergs I argued that, in BI projects, the technology – at least in the shape of front-end slice-and-dice tools – is not nearly as important as understanding the key business questions that need to be answered and the data available to answer them with. In “All that glisters is not gold” – some thoughts on dashboards, I made similar points about this aspect of BI technology.

I am not alone in holding these opinions, many of the BI consultants and experienced BI managers that I speak to feel the same way. Given this, why is there the disconnect that I refer to above? It is a reasonable assumption that when a company is looking to set up a new BI department within IT, it is the CIO who sets the tone. Does this lead us inescapably to the the conclusion that many CIOs just don’t get BI?

I hope that this is not the case, but I see increasing evidence that there may be a problem. I suppose the sliver lining to this cloud is that, while such attitudes exist, they will lead to opportunities for more enlightened outfits, such as the one fronted by Maureen Clarry. However it would be even better to see the ideas that Maureen espouses moving into the mainstream thinking of corporate IT.

Maureen Clarry is the Founder and President/CEO of CONNECT: The Knowledge Network, a consulting firm that specializes in helping IT people and organizations to achieve their strategic potential in business. CONNECT was recognized as the 2000 South Metro Denver Small Business of the Year and has been listed in the Top 25 Women-Owned Businesses and the Top 150 Privately Owned Businesses in Colorado. Maureen also participates on the Data Warehousing Advisory Board for The Daniels College of Business at the University of Denver and was recognized by the Denver Business Journal as one of Denver’s Top Women Business Leaders in 2004. She has been on the faculty of The Data Warehousing Institute since 1997, has spoken at numerous other seminars, and has published several articles and white papers. Maureen regularly consults and teaches on organizational and leadership issues related to information technology, business intelligence and business.

Some thoughts on IT-Business Alignment from the Chase Zander IT Director Forum

This Chase Zander seminar, which I earlier previewed on this site, took place yesterday evening in Birmingham. There was a full house of 20 plus IT Directors, CIOs and other senior IT managers who all engaged fully in some very stimulating and lively discussions.

As I previously mentioned, our intention in this meeting was to encourage debate and sharing of experiences and best practice between the delegates. My role was to faciliate the first session, focussed on IT-Business alignment. I started by sharing a few slides with that group that explained the research we had conducted to determine the content of the forum.

Click to view the introductory presentation
Click to view the introductory presentation as a PDF

After sharing what in my opinion was a not wholly satisfactory definition of IT-Business alignment, I opened up the floor to a discussion of what IT-Business alignment actually was and why it mattered. We used some of the other slides later in the meeting, but most of the rest of the evening was devoted to interaction between the delegates. Indeed the ensuing conversations were so wide ranging that the theme was also carried over to the second session, hosted by my associate Elliot Limb.

Territory initially covered included the suggestion that IT should be an integral part of the business, rather than a separate entity aligned to it (a theme that I covered in my earlier article Business is from Mars and IT is from Venus, which interestingly I penned after a previous Chase Zander forum, this one focussed on change management). The group also made a strong connection between IT-Business alignment and trust. A count of hands in response to the question “do you feel that you have the 100% unqualified confidence of your CEO?” revealed a mixed response and we tried to learn from the experiences of those who responded positively.

The relationship between IT and change was also debated. Some felt that IT, with its experience of project-based work, was ideally placed to drive change in organisations. Others believed that change should be a business function, with IT sticking to its more traditional role. Different organisations were in different places with respect to this issue – one attendee had indeed seen his current organisation take both approaches in the recent past. It was also agreed that there were different types of change: positive change in reaction to some threat or opportunity and the less positive change for change’s sake that can sometimes affect organisations.

Suggestions for enhancing IT-Business alignment included: being very transparent about IT service level agreements and trends in them; focussing more on relationships with senior managers, the CEO and CFO in particular; better calculating the cost of IT activities (including business resource) and using this to prioritise and even directly charge for IT services; applying marketing techniques to IT; learning to better manage business expectations, taking on more realistic workloads and knowing when to say ‘no’; and paying more attention to business processes, particularly via capability maturity modelling.

It was agreed that it generally took quite some time to establish trust between a CIO and the rest of the senior management team. This might be done by initially sorting out problems on the delivery and support side and, only once confidence had been built up, would the CIO be able to focus more on strategic and high value-added activities. This process was not always aided by the not atypical 3-5 year tenure of CIOs.

Later discussions also touched on whether CIOs would generally expect (or want to) become CEOs and, if not, why was this the case. The perspective of both the delegates and the Chase Zander staff was very interesting on this point. There was a degree of consensus formed around the statement that IT people liked taking on challenging problems, sorting them out and then moving on to the next one. While there was some overlap between this perspective and the role of a CEO in both having their hand on the tiller of an organisation and challenging the management team to meet stretch goals, there was less than a perfect fit. Maybe this factor indicated something of a different mindset in many IT professionals.

In the context of forming better relationships with business managers and IT trying to be less transactional in its dealings with other areas, the question of why there were so few women in senior IT positions also came up. This is a large topic that could spawn an entire forum in its own right.

Overall the meeting was judged to be a success. From my perspective it was also interesting to meet a good cross-section of IT professionals working in different industries and to talk about both what the different challenges that we faced and what we had in common.

Continue reading about this area in: The scope of IT’s responsibility when businesses go bad

A more appropriate metaphor for Business Intelligence projects

A traditional metaphor for IT projects
A traditional metaphor for IT projects

IT people are familiar with a number of metaphors for their projects. The most typical relates to building; IT projects are compared to erecting a skyscraper. The IT literature is suffused with language derived from this metaphor. We build systems. We develop blueprints for them. We design architectures (two-for-one there). This analogy has some strength and there are indeed superficial similarities between the two areas. However, as with most metaphors, if over-extended their applicability often breaks down. I recall one CEO in particular who was obsessed by the “building team” moving on to the next “site”; regardless of the current one requiring further work and dedicated maintenance. One of his predecessors often referred to wanting a “diesel submarine” built, as opposed to a “nuclear one”. Before I fall into the same trap of over-exploiting the metaphor, let’s move hurriedly on.

As I mention above, aside from the occasional misapplication, the building analogy works reasonably well for many IT projects; does it also work for business intelligence? I think that there are some problems in applying the metaphor. Building tends to follow a waterfall project plan (as do many IT projects). Of course there may be some iterations, even many of them, but the idea is that the project is made up of discrete base-level tasks whose duration can be estimated with a degree of accuracy. Examples of such a task might include writing a functional specification, developing a specific code module, or performing integration testing between two sub-systems. Adding up all the base-level tasks and the rates of the people involved gets you a cost estimate. Working out the dependencies between the base-level tasks gets you an overall duration estimate.

The problem with BI projects is that some of the base-level tasks are a bit different. An example might be: develop an understanding of a legacy data table, how it relates to other legacy data sets and to more modern systems (this sits under area two of my model of BI development – see BI implementations are like icebergs). This is not an exercise that is very easy to estimate in advance. Indeed it may not be possible to produce an adequate estimate until a substantial amount of work has been done. Even at a late stage in the task, something may be discovered which expands the work required dramatically; surprises may lurk round every corner.

Why is this? Well with legacy data, the people who developed the system may have done so many years ago. Since then, they may have left the company or moved on to other areas, taking their knowledge with them. Their place may have been taken by successive tranches of new staff. Perhaps poor initial documentation meant that later workers did not fully understand the full nature of the system, but nevertheless did their best to build upon it. Perhaps the documentation was good at first, but has not been kept up-to-date and now describes a system that no longer exists.

By definition, legacy systems will have been around for some time and layers of changes will have accumulated on top of each other. Maybe, as a company has expanded, new data has been interfaced to the system from different business units and territories; perhaps each of these cases has its own dedicated interface code, each subtly different from those of other systems. Even where people exist in an organisation who preserve an “oral tradition” about the system, handed down to them over generations; these people may not appreciate how their data interacts with other data – even if the person who looks after another legacy system sits in the adjacent cubicle.

Waterfall Project Plan (intentionally blurred somewhat)

Although these challenges can also occur when trying to understand the data in more modern systems, they are particularly acute with older ones. For a start, the people who designed these systems are more likely to be around. Also legacy systems often sit at the centre of a Byzantine web of inter-connections, batch-processes, over-night jobs and the occasional more modern service. It can be a real mess and this is a situation with which any data analyst with a reasonable amount of experience will be very familiar.

The difficulty of estimating the duration of tasks such as properly analysing legacy data makes overall estimation of BI projects more of an art than a science. Of course techniques such as time-boxing tasks can be applied, but these are not always 100% appropriate. A 75% analysed data source (even assuming that the estimate that only 25% work is left is accurate) is not an analysed data source. Leaving dark corners of knowledge is likely to be reflected in BI cubes and reports that do not reconcile back to their sources. Probably the best way to deal with this problem is to be extremely open about the challenges up-front with executive sponsors and when submitting estimates. It helps to also stress the level of uncertainty in progress reports. The more honest you are initially, the better you will be able to explain any overruns and the more likely it is that you will be believed.

These types of issues mean that the – hopefully more orderly – process of constructing a building is not a fully accurate way to describe a BI project. That is unless the metaphor is extended to include an occurance that is all too common during construction in The City of London. Given the age of Londinium, whenever ground is broken on a new project, it is more likely than not that a mediaeval, Anglo-Saxon or Roman site is unearthed (often all three). These finds, while of enormous interest to academics, can result in projects being put on hold (sometimes for years) while the dig is fully assessed, artefacts are carefully removed and catalogued by experts and so on. Sometimes the remains are of such importance that a structure preserving and protecting them (and even allowing public viewing) has to be made part of the design of the foundations of new building. Many office blocks in The City have such viewing galleries in their basements. Such eventualities can create massive and unexpected overruns in central London building projects.

So this leads me to suggest a different metaphor for BI projects. Major elements of them are much more like archaeological digs than traditional building. The extent and importance of a dig is very difficult to ascertain before work starts and both may change during the course of a project. It is not atypical that an older site is discovered underneath an initial dig, doubling the amount of work required.

So, my belief is that BI professionals should not be likened to architects or structural engineers. Instead the epithet of archaeologist is much more appropriate. And if the fedora fits, wear it!

Fortune and glory in BI?
Fortune and glory in BI?

Apologies for the initial typo’ in the article heading, which persists in the perma-link. Of course I have the odd error scattered around the place, but in a heading! Maybe I need to employ a proof-reader.

The Top Business Issues facing CIOs / IT Directors – Results

Back in January, in collaboration with Chase Zander, I started a process of consulting with senior IT managers to develop a list of the top business issues that they faced. This exercise was intended to shape the content of a IT Director Forum that we were planning. This will now be happening on 26th March in Birmingham (for further information see this post).

Questionaire Responses
The Top Business Issues faced by CIOs / IT Directors

Back then, I promised to share some of the findings from this study. These are summarised in the above diagram. The input was based on public comments made by a selection of senior people on the CIO group of, plus e-mails sent to me on the topic and feedback received by Chase Zander.

A textual version of the data appeas below (sample size ~60):

  Issue % of Votes  

  IT / Business Alignment 27%  
  Cost-saving 13%  
  Managing change 8%  
  Status of the IT Director 8%  
  Legacy Systems 5%  
  Customer focus 5%  
  Enterprise Architecture 5%  
  Business Intelligence 5%  
  Avoiding the latest and greatest 3%  
  Cloud Computing 3%  
  Only one response 17%  

  Total 100%  

I would like to thank all of the IT professionals who contributed to this survey.

An invitation to Chase Zander’s IT Director Forum

Please click on the image to view the full-sized invitation
Please click on the image to view the full-sized invitation

Some time ago I posted about about the IT Director Forum that I was helping Chase Zander to set up. This is happening on the evening of March 26th in Birmingham, England.

Seminar topics: People who should attend:
  • How Business and IT be better aligned?
  • How IT can add more value while being more cost effective?
  • IT Directors
  • Managing Directors
  • Senior Business-Facing IT Professionals

A summary of the research that led to developing these topics may be viewed here.

Rather than just making presentations, the objective is to have round-table discussions with delegates sharing their experiences. I will be facilitating the IT / Business Alignment session and Elliott Limb will be handling the Adding value with IT session. Elliott is an IT and Business Leader and Author of forthcoming book Credibility – Bridging the IT / business divide.

At present, there are a few places still available. Any UK-based IT managers who are interested in attending can contact Emily White ( or 0870 997 9014) to make a reservation.

Some reasons why IT projects fail

© Alex Messenger -
© Alex Messenger -

Having yesterday been somewhat disparaging about the efforts of others to delineate the reasons for BI projects failing, I realised that I had recently put together just such a list myself. By way of context, this was in response to being asked for some feedback in a subject area where I had little expertise and experience. Instead of bailing out of answering, I put together a more general response, a lightly edited and mildly expanded version of which appears below.

Please note that there is no claim on my part that this list is exhaustive; in common with all humans, us IT types can be very creative in finding new ways to fail, I am sure there are some out there that I have not come across yet.

  1. The objectives of the project not being clear. By this I mean the business objectives. There are two layers of problems, the actual business issues may not be understood well enough to form an effective response and, if the business knows what it needs to do in general terms, IT may not fully appreciate this for a number of reasons (mostly down to lack of communication) or may be unable to translate this into a suitable programme of work (possibly because of a lack of knowledge of how the business operates). Where IT is not part of the senior management team, or sees itself as a department apart, this issue is more likely to occur.
  2. Strategy formation being skipped. If you don’t understand what a project is meant to be about, it is difficult (to say the least) to form a strategy. However, if the test in point 1. is passed, then it may be tempting (or there may be pressure applied) to get to the end game as soon as possible without either forming a strategy for the project, or fitting this into both over-arching business and IT strategies (which one fervently hopes are complementary). As I know all too well, the strategy formation step can be tough one and people may sometimes be keen to skip it. The current economic climate may lead to this happening more frequently and my opinion is that this will be storing up trouble for the future.
  3. Fragmented systems’ landscapes. Related to the above, it is often very difficult to make progress when there is a patchwork of different systems and approaches throughout an organisation and little desire to address this short-coming. Often some sort of revolution (albeit sometimes a quiet one sustained over many years) is required to move forward. Sometimes this requires some crisis, internal or external, as virtually every organisation is inherently conservative; no matter what their marketing spiel may claim to the contrary.
  4. Lack of Change Management. Projects often also have an organisational change aspect (what else are they for?) and the problems here are: a) people do not like change and resist it; and b) many otherwise able managers are not experienced in change – indeed we tend to educate most managers to be efficient in a steady-state environment. Even when this aspect is recognised, it is often underestimated and work does not start until too late in the game.
  5. People. Aside from these, the main other problem is people. Projects, even small ones, are difficult and not everyone is up to running them. Simple errors in execution can derail projects that otherwise tick all of the boxes.

Of course any passing Gartner analyst is more than welcome to rip this to shreds if they see fit.