“Businesses Are Still Crazy for BI After All These Years” – CIO.com

Thomas Wailgum at CIO.com

Thomas Wailgum has written an article at CIO.com in which he talks about continuing demand for BI, but adds that this, in turn, suggests that in many organisations BI has yet to deliver on its promise. As Thomas puts it:

“I see pent-up enterprise-wide frustration, aimed squarely at IT and CIOs for failing to give the business what it needs and deserves”

He sees the fundamental problem as being fragmented systems and stand-alone BI applications. This sounds like challenges that I have faced before. I agree that BI only realises it potential when a more strategic and wide-ranging approach is taken. Something I refer to in many places on this blog, but possibly most directly in Holistic vs Incremental approaches to BI.

My basic point is that while it is sensible to take a pragmatic, incremental approach to implementing BI (collecting successes as you go and building momentum), this needs to be within the framework of a more encompassing vision for what the eventual BI system will be like and do.

I don’t believe that you can do BI by halves and remain somewhat sceptical about the claims of some of the newer BI products to do away with the necessary hard work.

Holistic vs Incremental approaches to BI

There is a strong link here to my Vision vs Pragmatism article. In this I argued that Vision and Pragmatism are both essential for the success of any project, be that related to change, to IT, and certainly when using IT to drive change. Unsurprisingly, similar comments apply to whether a holistic or incremental approach to BI is the superior route. However, in this case, I will come down more firmly on the side of one of the options.
The benefits of an incremental approach

Of course the secret of the success of many projects is their incremental nature. Incremental deliveries, particularly those early on in a project, enable you to do a number of things, including: –

  1. Proving that business value can be added the work that you are doing
  2. Showing tangible evidence of progress
  3. Demonstrating that the project team is responsive to business priorities
  4. Chopping up funding into more digestible parts
  5. Providing early exposure to change management issues; allowing time to learn from mistakes when still operating at on a smaller scale

Overall incremental work can enhance the credibility of a project team and thereby made it easier to secure senior management support. Such work is indispensable to any project.
How does the sum of the parts measure up?

However there is a point to be made here in favour of a holistic approach which goes beyond my previous preference for always having an overarching vision. This is something that is specific to business intelligence and relates to the nature of information delivery. In a nutshell the sum of several incremental BI developments may be considerably less than the whole if each is not part of an overall strategy.

BI is about having the information necessary to run the business. However, it is also about how that information is delivered and how internally consistent it is. Often BI projects aim to address a fragmentation of existing reporting systems that leads to confusion amongst users and even a general distrust of figures. It is entirely possible to perpetuate this situation, simply replacing older reporting technology with shiny new ones. Each of these new systems may be easier to use that its predecessor and offer significantly greater access to information, but the fragmented nature of information provision will not have been addressed; it may even have been made worse.
A single platform

The ideal for a BI solution is to have a single platform which supports all pertinent reporting needs. There will undoubtedly be different segments of this, tailored to different groups of users, but these should use subsets of the same dimensions and measures and the same reporting and analysis tools should be used. Adhering to these precepts means that when users of one part of the system need to employ another part, they are not taking a step into terra incognita, but instead are familiar with their surroundings and get the sense that the same logic pervades all of the system.

On a practical level, this approach minimises costs due to software licenses and simplifies your technical architecture, again keeping a lid on expenditure. Fewer people are also needed to both build and maintain a single, central system than many divergent ones. Just as importantly, a single-platform approach means that training becomes focussed on business issues rather than the functionality of a different reporting suites. My experience suggests that, after an initial investment in thorough training for users, introduction of new reporting capabilities can be very smooth and efficient in such a set-up.

Of course developing good BI takes time and effort. Getting to the eventual ideal state that I have described above will undoubtedly take some time (in my most recent BI project it took five years to fully realise). This means that there is no real alternative to the incremental approach that I described at the beginning. However, taking a more holistic approach ensures that your incremental deliveries are aligned with both each other and overall business needs. It also means that with each incremental release there is a related reduction in fragmentation. This is the difference between slowly unveiling a large, coherent edifice and revealing several separate sculptures one at a time.
The link with cultural transformation

In particular if an aim of your BI project is to transform how users behave (of course this should be a central aim of any BI project, what else is BI for?), then this is going to be most easily achieved with a holistic approach where each phase builds on the success of the previous ones. In this scenario, each incremental delivery can be seen more as extending the remit of your BI system to a new area, rather than adding on a new module. Phase N+1 always reinforces the messages from Phases 1 to N. Each step reduces fragmentation, increases consistency and further improves decision-making. This is the best way to make sure that your BI efforts exceed the sum of their parts, rather than falling short of them. Such a rigorous approach is also the best way to ensure that you meet your cultural transformation objectives.

Vision vs Pragmatism

The ESA's Herschel infrared space observatory

This time last year, I was a member of a panel on a webinar hosted by Computing and Accountancy Age magazines. This post is not specifically about this webinar, but rather about positions that I regularly found myself taking in response to questions. The questions were along the lines of “Do you think that X or Y is more important in trying to achieve Z?”, my frequent reply was “both”. In fact at one point I recall deprecating my own fence-sitting.

Fence-sitting is not normally seen as the most noble of human activities, it tends to suggest a lack of decisiveness, even timidity. However, when faced with a question as basic as “What is more important for survival, food or water?”, then “both” seems to be the only intellectually credible stance to take. Allowing for the nit-picking point that you will die of thirst quicker than you will starve, over the medium term food and water are equally important. I feel the same about vision and pragmatism in business projects and in business people.

There is nothing that homo sapiens likes more than to pigeonhole his or her fellows. We tend to take a binary approach to people’s skills. Fred is a visionary, but you wouldn’t want him to run a project. Jane is brilliant at the details, but she doesn’t see the big picture. Perhaps we are more comfortable with the idea that the strength of any colleague is automatically balanced by a weakness; it brings them back down to a reasonable level – what the Australians call tall poppy syndrome. Maybe the way that we think about visionary people is also influenced by the connotations of the word, bringing to mind soothsayers, prophets and oracles. All of these historical figures had an other-worldly persona (often literally). They were not like “normal” people. Culturally, those who have visions are seen as a race apart. As Fitzgerald might have said “Let me tell you about the visionary. They are different from you and me.”

Setting aside any psychological angle, there are two points to be made here. First, of course people are all different and are endowed with varying abilities. This means that any successful team needs to have a balance of personalities and skill-sets. If you have some one who is purely a visionary on a team, then that is a great strength (most of the time), but orthodoxy suggests that this needs to be balanced by people who have less ethereal skills. So far, so hum-drum.

The second point is a potentially more interesting one. Maybe, contrary to what I have written above, visionaries are not so different from the rest of us. Instead of being skin-clad augurs with wild hair, maybe visionaries are people who can embrace a certain way of thinking when necessary. Maybe vision is something that you can turn on and off. This certainly chimes with most theory about personality types. Something that is often forgotten is that extrovert / introvert is not a binary choice, but a continuum. Also where some one places on this scale on average, may be quite different to where they place at a particular moment. Some one who is 75% introvert on average may be exceptionally extrovert in certain circumstances. Applying the same logic, some one who is not normally visionary, may be so sometimes and vice versa. So instead of the orthodoxy of having a team made up of discrete personality types, maybe we should realise that the behaviour of team members and what they can contribute may change over time.

There is clearly a lot that could be discussed here, I am going to restrict myself to talking about vision and what is often seen as it alter-ego, pragmatism. The question I will consider is “What is more important for a project, vision or pragmatism?”. This is where I return to fence-sitting, my answer is a resounding “both”. Vision is necessary to work out what to do, pragmatism is necessary to do it; a food and water situation. In fact I would argue that the optimum way to run a project is to initially develop a vision of the ideal outcome, ignoring any constraints. Such an approach is often seen as unrealistic and is tagged with unfavourable epithets such as “ivory tower” or “blue sky thinking”. However it is a necessary step. I much prefer the idea of thinking of what could be achieved and then applying constraints of time, funding and appetite for change, than the opposite where any potential progress is immediately ham-strung by such considerations. If vision is used to define a desirable, but potentially unattainable, Utopia and then pragmatism is used to pare this down to what is achievable, then the resulting strategy will retain some of the shape of the original ideas. It is likely to result in an approach that has a central theme, that is coherent and which will offer a platform for further progress. Applying pragmatism first is likely to yield a fragmented programme that is uncertain what issues it is meant to be addressing and, by seeking to do only what is incremental, will inevitably fall short of what could be possible (even given constraints).

Looking at this issue the other way round. If there is not the second-pass of applying pragmatism to the initial vision (even sometimes to the degree that the vision is rejected as unworkable), then failure is all but guaranteed. Pragmatism is the structural engineer finding solutions to the challenges posed by the architect’s design. It is figuring out the “how” after vision has established the “what” and “why”. It is also one of the main attributes that is necessary for governing execution, suggesting as it does a flexible approach and the maxim that “what counts most is what works best”. It is difficult to envisage how anything other than pragmatism would lead to success in these phases of a project. There is however something else to consider here. Something that sustains projects through execution is often the initial vision. This gives the team a sense of what they are doing and why they are doing it. This can be crucial when the inevitable setbacks are faced. Vision may also need to be switched back on when a major obstacle needs to be overcome or a change in direction is required. Rather than thinking of vision and pragmatism being sequential phases, perhaps they are alternating mind-sets that continue to vie for pre-eminence during a project. On average vision has the upper hand early on and pragmatism in the middle and later stages, but at any given point, it maybe desirable for the positions to be reversed.

This is another echo of the earlier comments about personality types and it is to this area that I will return in closing. Certainly projects need both visionaries and pragmatists; however these can often be the same people. I would argue strongly that a number of people are capable of both developing visions and aggressively pruning these to make them realisable, or chopping them into phases with phase B predicated on the success of phase A. Further I think that a make-up that embodies both vision and pragmatism, together with having the ability to flip between them as dictated by circumstances, tends to be the ideal one for managing projects. Certainly having one person who can encompass “what”, “why” and “how” seems efficient, but this holistic view of the process tends to go hand-in-hand with a passion to deliver. This passion is a product part of vision (believing in your own ideas) and part of pragmatism (owning the delivery of these ideas) and a very powerful factor behind successful projects.

Continue reading about ideas related to this area in: Holistic vs Incremental approaches to BI.