On further reflection about this earlier article, I realised that I missed out one important point. This was perhaps implicit in the diagram that I posted (and which I repeat below), but I think that it makes sense for me to make things explicit.
An example of a multi-tier BI architecture with different tools
The point is that in this architecture with different BI tools in different layers, it remains paramount to have consistency in terminology and behaviour for dimensions and measures. So “Country” and “Profit” must mean the same things in your dashboard as it does in your OLAP cubes. The way that I have achieved this before is to have virtually all of the logic defined in the warehouse itself. Of course some things may need to be calculated “on-the-fly” within the BI tool, in this case care needs to be paid to ensuring consistency.
It has been pointed out that the approach of using the warehouse to drive consistency may circumscribe your ability to fully exploit the functionality of some BI tools. While this is sometimes true, I think it is not just a price worth paying, but a price that it is mandatory to pay. Inconsistency of any kind is the enemy of all BI implementations. If your systems do not have credibility with your users, then all is already lost and no amount of flashy functionality will save you.
This post follows on from a question that was asked on the LinkedIn.com Data Warehousing Institute (TDWI™) 2.0 group. Unfortunately the original thread is no longer available for whatever reason, but the gist of the question was whether anyone had experience with using a number of BI tools to cover different functions within an implementation. So the scenario might be: Tool A for dashboards, Tool B for OLAP, Tool C for Analytics, Tool D for formatted reports and even Tool E for visualisation.
In my initial response I admitted that I had not faced precisely this situation, but that I had worked with the set-up shown in the following diagram, which I felt was not that dissimilar:
An example of a multi-tier BI architecture with different tools
Here there is no analytics tool (in the statistical modelling sense – Excel played that role) and no true visualisation (unless you count graphs in PowerPlay that is), but each of dashboards, OLAP cubes, formatted reports and simple list reports are present. The reason that this arrangement might not at first sight appear pertinent to the question asked on LinkedIn.com is that two of the layers (and three of the report technologies) are from one vendor; Cognos at the time, IBM-Cognos now. The reason that I felt that there was some relevance was that the Cognos products were from different major releases. The dashboard tool being from their Version 8 architecture and the OLAP cubes and formatted reports from their Version 7 architecture.
A little history
London Bridge circa 1600
Maybe a note of explanation is necessary as clearly we did not plan to have this slight mismatch of technologies. We initially built out our BI infrastructure without a dashboard layer. Partly this was because dashboards weren’t as much of a hot topic for CEOs when we started. However, I also think it also makes sense to overlay dashboards on an established information architecture (something I cover in my earlier article, “All that glisters is not gold” – some thoughts on dashboards, which is also pertinent to these discussions).
When we started to think about adding icing to our BI cake, ReportStudio in Cognos 8 had just come out and we thought that it made sense to look at this; both to deliver dashboards and to assess its potential future role in our BI implementation. At that point, the initial Cognos 8 version of Analysis Studio wasn’t an attractive upgrade path for existing PowerPlay users and so we wanted to stay on PowerPlay 7.3 for a while longer.
The other thing that I should mention is that we had integrated an in-house developed web-based reporting tool with PowerPlay as the drill down tool. The reasons for this were a) we had already trained 750 users in this tool and it seemed sensible to leverage it and b) employing it meant that we didn’t have to buy an additional Cognos 7 product, such as Impromptu, to support this need. This hopefully explains the mild heterogeneity of our set up. I should probably also say that users could directly access any one of the BI tools to get at information and that they could navigate between them as shown by the arrows in the diagram.
I am sure that things have improved immensely in the Cognos toolset since back then, but at the time there was no truly seamless integration between ReportStudio and PowerPlay as they were on different architectures. This meant that we had to code the passing of parameters between the ReportStudio dashboard and PowerPlay cubes ourselves. Although there were some similarities between the two products, there were also some differences at the time and these, plus the custom integration we had to develop, meant that you could also view the two Cognos products as essentially separate tools. Add in here the additional custom integration of our in-house reporting application with PowerPlay and maybe you can begin to see why I felt that there were some similarities between our implementation and one using different vendors for each tool.
I am going to speak a bit about the benefits and disadvantages of having a single vendor approach later, but for now an obvious question is “did our set-up work?” The answer to this was a resounding yes. Though the IT work behind the scenes was maybe not the most elegant (though everything was eminently supportable), from the users’ perspective things were effectively seamless. To slightly pre-empt a later point, I think that the user experience is what really matters, more than what happens on the IT side of the house. Nevertheless let’s move on from some specifics to some general comments.
The advantages of a single vendor approach to BI
One-stop shopping
I think that it makes sense if I lay my cards on the table up-front. I am a paid up member of the BI standardisation club. I think that you only release the true potential of BI when you take a broad based approach and bring as many areas as you can into your warehouse (see my earlier article, Holistic vs Incremental approaches to BI, for my reasons for believing this).
Within the warehouse itself there should be a standardised approach to dimensions (business entities and the hierarchies they are built into should be the same everywhere – I’m sure this will please all my MDM friends out there) and to measures (what is the point if profitability is defined different ways in different reports?). It is almost clichéd nowadays to speak about “the single version of the truth”, but I have always been a proponent of this approach.
I also think that you should have the minimum number of BI tools. Here however the minimum is not necessarily always one. To misquote one of Württemberg’s most famous sons:
Everything should be made as simple as possible, but no simpler.
What he actually said was:
It can scarcely be denied that the supreme goal of all theory is to make the irreducible basic elements as simple and as few as possible without having to surrender the adequate representation of a single datum of experience.
but maybe the common rendition is itself paying tribute to the principle that he propounded. Let me pause to cover what are the main reasons quoted for adopting a single vendor approach in BI:
Consistent look-and-feel: The tools will have a common look-and-feel, making it easier for people to use them and simplifying training.
Better interoperability: Interoperability between the tools is out-of-the-box, saving on time and effort in developing and maintaining integration.
Clarity in problem resolution: If something goes wrong with your implementation, you don’t get different vendors blaming each other for the problem.
Simpler upgrades: You future proof your architecture, when one element has a new release, it is the vendor’s job to ensure it works with everything else, not yours.
Less people needed: You don’t need to hire an expert for each different vendor tool, thereby reducing the size and cost of your BI team.
Cheaper licensing: It should be cheaper to buy a bundled solution from one vendor and ongoing maintenance fees should also be less.
This all seems to make perfect sense and each of the above points can be seen to be reducing the complexity and cost of your BI solution. Surely it is a no-brainer to adopt this approach? Well maybe. Let me offer some alternative perspectives on each item – none of these wholly negates the point, but I think it is nevertheless worth considering a different perspective before deciding what is best for your organisation.
Consistent look-and-feel: It is not always 100% true that different tools from the same vendor have the same look-and-feel. This might be down to quality control at the vendor, it might be because the vendor has recently acquired part of their product set and not fully integrated it as yet, or – even more basically – it may be because different tools are intended to do different things. To pick one example from outside of BI that has frustrated me endlessly over the years: PowerPoint and Word seem to have very little in common, even in Office 2007. Hopefully different tools from the same vendor will be able to share the same metadata, but this is not always the case. Some research is probably required here before assuming this point is true. Also, picking up on the Bauhaus ethos of form dictating function, you probably don’t want to have your dashboard looking exactly like your OLAP cubes – it wouldn’t be a dashboard then would it? Additional user training will generally be required for each tier in your BI architecture and a single-vendor approach will at best reduce this somewhat.
Better interoperability: I mention an problem with interoperability of the Cognos toolset above. This is is hopefully now a historical oddity, but I would be amazed if similar issues do not arise at least from time to time with most BI vendors. Cognos itself has now been acquired by IBM and I am sure everyone in the new organisation is doing a fine job of consolidating the product lines, but it would be incredible if there were not some mismatches that occur in the process. Even without acquisitions it is likely that elements of a vendor’s product set get slightly out of alignment from time to time.
Clarity in problem resolution: This is hopefully a valid point, however it probably won’t stop your BI tool vendor from suggesting that it is your web-server software, or network topology, or database version that is causing the issue. Call me cynical if you wish, I prefer to think of myself as a seasoned IT professional!
Simpler upgrades: Again this is also most likely to be a plus point, but problems can occur when only parts of a product set have upgrades. Also you may need to upgrade Tool A to the latest version to address a bug or to deliver desired functionality, but have equally valid reasons for keeping Tool B at the previous release. This can cause problems in a single supplier scenario precisely because the elements are likely to be more tightly coupled with each other, something that you may have a chance of being insulated against if you use tools from different vendors.
Less people needed: While there might be half a point here, I think that this is mostly fallacious. The skills required to build an easy-to-use and impactful dashboard are not the same as building OLAP cubes. It may be that you have flexible and creative people who can do both (I have been thus blessed myself in the past in projects I ran), but this type of person would most likely be equally adept whatever tool they were using. Again there may be some efficiencies in sharing metadata, but it is important not to over-state these. You may well still need a dashboard person and an OLAP person, if you don’t then the person who can do both with probably not care about which vendor provides the tools.
Cheaper licensing: Let’s think about this. How many vendors give you Tool B free when you purchase Tool A? Not many is the answer in my experience, they are commercial entities after all. It may be more economical to purchase bundles of products from a vendor, but also having more than one in the game may be an even better way of ensuring that cost are kept down. This is another area that requires further close examination before deciding what to do.
A more important consideration
Overall it is still likely that a single-vendor solution is cheaper than a multi-vendor one, but I hope that I have raised enough points to make you think that this is not guaranteed. Also the cost differential may not be as substantial as might be thought initially. You should certainly explore both approaches and figure out what works best for you. However there is another overriding point to consider here, the one I alluded to earlier; your users. The most important thing is that your users have the best experience and that whatever tools you employ are the ones that will deliver this. If you can do this while sticking to a single vendor then great. However if your users will be better served by different tools in different tiers, then this should be your approach, regardless of whether it makes things a bit more complicated for your team.
Of course there may be some additional costs associated with such an approach, but I doubt that this issue is insuperable. One comparison that it may help to keep in mind is that the per user cost of many BI tools is similar to desktop productivity tools such as Office. The main expense of BI programmes is not the tools that you use to deliver information, but all the work that goes on behind the scenes to ensure that it is the right information, at the right time and with the appropriate degree of accuracy. The big chunks of BI project costs are located in the four pillars that I consistently refer to:
Understand the important business decisions and what figures are necessary to support these.
Understand the data available in the organisation, how it relates to other data and to business decisions.
Transform the data to provide information answering business questions.
Focus on embedding the use of information in the corporate DNA.
The cost of the BI tools themselves are only a minor part of the above (see also, BI implementations are like icebergs). Of course any savings made on tools may make funds available for other parts of the project. It is however important not to cut your nose off to spite your face here. Picking right tools for the job, be they from one vendor or two (or even three at a push) will be much more important to the overall payback of your project than saving a few nickels and dimes by sticking to a one-vendor strategy just for the sake of it.
The subject of this article ought to be reasonably evident from its title. However there is perhaps some room for misinterpretation around even this. Despite the recent furore about definitions, most reasonable people should be comfortable with a definition of business intelligence. My take on this is that BI is simply using information to drive better business decisions. In this definition, the active verb “drive” and the subject “business decisions” are the key elements; something that is often forgotten in a rush for technological fripperies.
The central issue
Having hopefully addressed of the “BI” piece of the BICC acronym, let’s focus on the “CC” part. I’ll do this in reverse order, first of all considering what is meant by “centre”. As ever I will first refer to my trusted Oxford English Dictionary for help. In a discipline, such as IT, which is often accused of mangling language and even occasionally using it to obscure more than to clarify, a back-to-basics approach to words can sometimes yield unexpected insights.
centre / séntər / n. & v. (UScenter) 3 a a place or group of buildings forming a central point in a district, city, etc., or a main area for an activity (shopping centre, town centre).
(O.E.D.)
Ignoring the rather inexcusable use of the derived adjective “central” in the definition of the noun “centre”, then it is probably the “main area for an activity” sense that is meant to be conveyed in the final “C” of BICC. However, there is also perhaps some illumination to be had in considering another meaning of the word:
n. 1 a the middle point, esp. of a line, circle or sphere, equidistant from the ends, or from any point on the circumference or surface.
(O.E.D.)
As well as appealing to the mathematician in me, this meaning gives the sense that a BICC is physically central geographically, or metaphorically central with respect to business units. Of course this doesn’t meant than a BICC needs to be at the precise centre of gravity of an organisation, with each branch contributing a “weight” calculated by its number of staff, or revenue; but it does suggest that the competency centre is located at a specific point, not dispersed through the organisation.
Of course, not all organisations have multiple locations. The simplest may not have multiple business units either. However, there is a sense by which “centre” means that a BICC should straddle whatever diversity there is an organisation. If it is in multiple countries, then the BICC will be located in one of these, but serve the needs of the others. If a company has several different divisions, or business units, or product streams; then again the BICC should be a discrete area that supports all of them. Often what will make most sense is for the BICC to be located within an organisation’s Head Office function. There are a number of reasons for this:
Head Office similarly straddles geographies and business units and so is presumably located in a place that makes sense to do this from (maybe in an organisation’s major market, certainly close to a transport hub if the organisation is multinational, and so on).
If a BICC is to properly fulfil the first two letters of its abbreviation, then it will help if it is collocated with business decision-makers. Head Office is one place than many of these are found, including generally the CEO, the CFO, the Head of Marketing and Business Unit Managers. Of course key decision makers will also be spread throughout the organisation (think of Regional and Country Managers), but it is not possible to physically collocate with all of these.
Another key manager who is hopefully located in Head Office is the CIO (though this is dispiritingly not always the case, with some CIOs confined to IT ghettos, far from the rest of the executive team and with a corresponding level of influence). Whilst business issues are pre-eminent in BI, of course there is a major technological dimension and a need to collaborate closely with those charged with running the organisation’s IT infrastructure and those responsible for care and feeding of source data systems.
If a BI system is to truly achieve its potential, then it must become all pervasive; including a wide range of information from profitability, to sales, to human resources statistics, to expense numbers. This means that it needs to sit at the centre of a web of different systems: ERP, CRM, line of business systems, HR systems etc. Often the most convenient place to do this from will be Head Office.
Thusfar, I haven’t commented on the business benefits of a BICC. Instead I have confined myself to explaining what people mean by the second “C” in the name and why this might be convenient. Rather than making this an even longer piece, I am going to cover both the benefits and disadvantages of a BICC in a follow-on article. Instead let’s now move on to considering the first “C”: Competency.
Compos centris
Returning to our initial theme of generating insights via an examination of the meaning of words in a non-IT context, let’s start with another dictionary definition:
competence /kómpit’nss/ n. (also competency /kómpitənsi/) 1 (often foll. by for, or to + infin.) ability; the state of being competent.
and given the recursive reliance of the above on the definition of competent…
competent /kómpit’nt/ adj. 1 a (usu. foll. by to + infin.) properly qualified or skilled (not competent to drive); adequately capable, satisfactory. b effective (a competent bastman*).
(O.E.D.)
* People who are not fully conversant with the mysteries of cricket may substitute “batter” here.
To me the important thing to highlight here is that, while it is to be hoped that a BICC will continue to become more competent once it is up and running, in order to successfully establish such a centre, a high degree of existing competence is a prerequisite. It is not enough to simply designate some floor space and allocate a number of people to your BICC, what you need is at least a core of seasoned professionals who have experience of delivering transformational information and know how to set about doing it.
Understand the important business decisions and what figures are necessary to support these.
Understand the data available in the organisation, how it relates to other data and to business decisions.
Transform the data to provide information answering business questions.
Focus on embedding the use of information in the corporate DNA.
So a successful BICC must include: people with strong analytical skills and an understanding of general business practices; high-calibre designers; reliable and conscientious ETL and general programmers; experts in the care, feeding and design of databases; excellent quality assurance professionals; resource conversant with both whatever front-end tools you are using to deliver information and general web programming; staff with skills in technical project management; people who can both design and deliver training programmes; help desk personnel; and last, but by no means least, change managers.
Of course if your BI project is big enough, then you may be able to afford to have people dedicated to each of these roles. If resources are tighter (and where is this not the case nowadays?) then it is better to have people who can wear more than one hat: business analysts who can also design; BI programmers who will also take support calls; project managers who will also run training classes; and so on. This approach saves money and also helps to deal with the inevitable peaks and troughs of resource requirements at different stages in a project. I would recommend setting things up this way (or looking to stretch your people’s abilities into new areas) even if you have the luxury of a budget that would allow a more discrete approach. The challenge of course is going to be finding and retaining such multi-faceted staff.
Also, it hopefully goes without saying that BI is a very business-focussed area and some BICCs will explicitly include business people in them. Even if you do not go this far, then the BICC will have to form a strong partnership with key business stakeholders, often spread across multiple territories. The skill to manage this effectively is in itself a major requirement of the leading personnel of the centre.
Given all of the above, the best way to staff a BICC is with members of a team who have already been successful with a BI project within your organisation; maybe one that was confined to a given geographic region or business unit. If you have no such team, then starting with a BICC is probably a bridge too far. Instead my recommendation would be to build up some competency via a smaller BI project. Alternatively, if you have more than one successful BI team (and, despite the manifold difficulties in getting BI right, such things are not entirely unheard of) then maybe blending these together makes sense. This is unless there is some overriding reason not to (e.g. vastly different team cultures or methodologies. In this case, picking a “winner” may be a better course of action.
Such a team will already have the skills outlined above in abundance (else they could never have been successful). It is also likely that whatever information was needed in their region or business unit will be at least part of what is needed at the broader level of a BICC. Given that there are many examples of BI projects not delivering or consuming vastly more resource than anticipated, then leveraging those exceptional people who have managed to swim against this tide is eminently sensible. Such battle-hardened professionals will know what pitfalls to avoid, which areas are most important to concentrate on and can use their existing products to advertise the benefits of a wider system. If you have such people at the core of your BICC, then it will be easier to integrate new joiners and quickly shepherd them up the learning curve (something that can be particularly long in BI due to the many different aspects of the work).
Of course having been successful in one business unit or region is not enough to guarantee success on a larger scale. I spoke about some of the challenges of doing this in an earlier article, Developing an international BI strategy. Another issue that is likely to raise its head is the political dimension, in particular where different business units or regions already have a management information strategy at some stage of development. This is another area that I will also cover in more detail in a forthcoming piece.
Conclusions
It seems that simply musing on the normal meanings of the words “competency” and “centre” has led us into some useful discussions. As mentioned above, at least two other blog postings will expand upon areas that have been highlighted in this piece. For now what I believe we have learned so far is:
BICCs should (by definition) straddle multiple geographies and/or business units.
There are sound reasons for collocating the BICC with Head Office.
There is need for a wide range of skills in your BICC, both business-focussed and technical.
At least the core of your BICC should be made up of competent (and experienced) BI professionals .
More thoughts on the benefits and disadvantages of business intelligence competency centres and also the politcs that they have to negotiate will appear on this blog in future weeks.
In a previous article, A more appropriate metaphor for business intelligence projects, I explained one complication of business intelligence projects. This is that the frequently applied IT metaphor of building is not very applicable to BI. Instead I suggested that BI projects had more in common with archaeological digs. I’m not going to revisit the reasons for the suitability of looking at BI this way here, take a look at the earlier piece if you need convincing, instead I’ll focus on what this means for project estimation.
When you are building up, estimation is easier because each new tier is dependent mostly on completion of the one below, something that the construction team has control over (note: for the sake of simplicity I’m going to ignore the general need to dig foundations for buildings). In this scenario, the initial design will take into account of facts such as the first tier needing to support all of the rest of the floors and that central shafts will be needed to provide access and deliver essential services such as water, electricity and of course network cables. A reductionist approach can be taken, with work broken into discrete tasks, each of which can be estimated with a certain degree of accuracy. The sum of each of these, plus some contingency, hopefully gives you a good feel for the overall project. It is however perhaps salutary to note that even when building up (both in construction and in IT) estimation can still sometimes go spectacularly awry.
When you are digging down, your speed is dependent on what you find. Your progress is dictated by things that are essentially hidden before work starts. If your path ahead (or downwards) is obscured until your have cleared enough earth to uncover the next layer, then each section may hold unexpected surprises and lead to unanticipated delays. While it may be possible to say things like, “well we need to dig down 20m and each metre should take us 10 days”, any given metre might actually take 20 days, or more. There are two issues here; first it is difficult to reduce the overall work into tasks, second it is harder to estimate each task accurately. The further below ground a phase of the dig is, the harder it will be to predict what will happen before ground is broken. Even with exploratory digs, or the use of scanning equipment, this can be very difficult to assess in advance. However it is to the concept of exploratory digs that this article is devoted.
Why a feasibility study is invaluable
At any point in the economic cycle, even more so in today’s circumstances, it is not ideal to tell your executive team that you have no idea how long a project will take, nor how much it might cost. Even with the most attractive of benefits to be potentially seized (and it is my firm belief that BI projects have a greater payback than many other types of IT projects), unless there is some overriding reason that work must commence, then your project is unlikely to gain a lot of support if it is thus characterised. So how to square the circle of providing estimates for BI projects that are accurate enough to present to project sponsors and will not subsequently leave you embarrassed by massive overruns?
It is in addressing this issue that BI feasibility studies have their greatest value. These can be thought of as analogous to the exploratory digs referred to above. Of course there are some questions to be answered here. By definition, a feasibility study cannot cover all of the ground that the real project needs to cover, choices will need to be made. For example, if there are likely to be 10 different data sources for your eventual warehouse, then should you pick one and look at it in some depth, or should you fleetingly examine all 10 areas? Extending our archaeological metaphor, should your exploratory dig be shallow and wide, or a deep and narrow borehole?
A centre-centric approach
In answering this question, it is probably worth considering the fact that not all data sources are alike. There is probably a hierarchy to them, both in terms of importance and in terms of architecture. No two organisations will be the same, but the following diagram may capture some of what I mean here:
Two ways of looking at a systems' hierarchy
The figure shows a couple of ways of looking at your data sources / systems. The one of the left is rather ERP-centric, the one on the right gives greater prominence to front-end systems supporting different lines of business, but wrapped by a common CRM system. There are many different diagrams that could be drawn in many different ways of course. My reason for using concentric circles is to stress that there is often a sense in which information flows from the outside systems (ones primarily focussed on customer interactions and capturing business transactions) to internal systems (focussed on either external or internal reporting, monitoring the effectiveness of processes, or delivering controls).
There may be several layers through which information percolates to the centre; indeed the bands of systems and databases might be as numerous as rings in an onion. The point is that there generally is such a logical centre. Data is often lost on its journey to this centre by either aggregation, or by elements simply not being transferred (e.g. the name of a salesperson is not often recorded on revenue entries in a General Ledger). Nevertheless the innermost segment of the onion is often the most complex, with sometimes arcane rules governing how data is consolidated and transformed on its way to its final destination.
The centre in both of the above diagrams is financial and this is not atypical if what we are considering is an all-pervasive BI system aimed at measuring most, if not all, elements of an organisation’s activity (the most valuable type of BI system in my opinion). Even if your BI project is not all-pervasive (or at least the first phase is more specific), then the argument that there is a centre will probably still hold, however the centre may not be financial in this case.
My suggestion is that this central source of data (of course there may be more than one) is what should be the greatest focus of your feasibility study. There are several reasons for this, some technical, some project marketing-related:
As mentioned above, the centre is often the toughest nut to crack. If you can gain at least some appreciation of how it works and how it may be related to other, more peripheral systems, then this is a big advance for the project. Many of the archaeological uncertainties referred to above will be located in the central data store. Other data sources are likely to be simpler and thus you can be more confident about approaching these and estimating the work required.
A partial understanding of the centre is often going to be totally insufficient. This is because your central analyses will often have to reconcile precisely to other reports, such as those generated by your ERP system. As managers are often measured by these financial scorecards, if you BI system does not give the same total, it will have no credibility and will not be used by these people.
Because of its very nature, an understanding of the centre will require at least passing acquaintance with the other systems that feed data to it. While you will not want to spend as much time on analysing these other systems during the feasibility study, working out some elements of how they interact will be helpful for the main project.
One output from your feasibility study should be a prototype. While this will not be very close to the finished article and may contain data that is both unreconciled and partial (e.g. for just one country or line of business), it should give project sponsors some idea of what they can expect from the eventual system. If this prototype deals with data from the centre then it is likely to be of pertinence to a wide range of managers.
Strongly related to the last point, and in particular if the centre consists of financial data, then providing tools to analyse this is likely to be something that you will want to do early on in the main project. This is both because this is likely to offer a lot of business value and because, if done well, this will be a great advert for the rest of your project. If this is a key project deliverable, then learning as much as possible about the centre during the feasibility study is very important.
Finally what you are looking to build with your BI system is an information architecture. If you are doing this, then it makes sense to start in the middle and work your way outwards. This will offer a framework off of which other elements of your BI system can be hung. The danger with starting on the outside and working inwards is that you can end up with the situation illustrated below.
A possible result of building from the outside in to the centre
Recommendations
So my recommendation is that your feasibility study is mostly a narrow, deep dig, focussed on the central data source. If time allows it would be beneficial to supplement this with a more cursory examination of some of the data sources that feed the centre, particularly as this may be necessary to better understand the centre and because it will help you to get a better idea about your overall information architecture. You do not need to figure out every single thing about the central data source, but whatever you can find out will improve the accuracy of your estimate and save you time later. If you include other data sources in a deep / wide hybrid, then these can initially be studied in much less detail as they are often simpler and the assumption is that they will support later deliveries.
The idea of a prototype was mentioned above. This is something that is very important to produce in a feasibility study. Even if we take to one side the undeniable PR value of a prototype, producing one will allow you to go through the entire build process. Even if you do this with hand-crafted transformation of data (rather than ETL) and only a simplistic and incomplete approach to the measures and dimensions you support, you will at least have gone through each of the technical stages required in the eventual live system. This will help to shake out any issues, highlight areas that will require further attention and assist in sizing databases. A prototype can also be used to begin to investigate system and network performance, things that will influence your system topology and thereby project costs. A better appreciation of all of these areas will help you greatly when it comes to making good estimates.
Having understood quite a lot about your most complex data source and a little about other ones and produced a prototype both as a sales tool and to get experience of the whole build process, you should have all the main ingredients for making a credible presentation to your project sponsors. In this it is very important to stress the uncertainties inherent in BI and manage expectations around these. However you should also be very confident in stating that you have done all that can be done to mitigate the impact of these. This approach, of course supported by a compelling business case, will position you very well to pitch your overall BI project.
Note: I have just had it pointed out to me that I missed out HR from the above diagrams. I hope that none of the HR professionals who read this blog will be too offended by my oversight.
Information is one of the key components of any IT organization (I would personally argue it’s more important than the technology aspect). Two facts disturb me when one looks at IT’s role in the financial crisis:
1) We in IT have been pushing data warehouse and business intelligence technology for years, saying these technologies should allow for “proactive” decision making at all levels of an organization, and an ability to spot trends and changes in a business’ underlying financial health.
2) The finance industry is usually spends more on IT than any other industry.
This being the case, if BI actually does what we’ve pitched it to do, shouldn’t one of these fancy analytical tools spotted the underlying roots of the financial crisis in at least one major bank? Is IT partially culpable for either not looking at the right data, or selling a bill of goods in terms of the “intelligence” aspect of BI?
I have written elsewhere on LinkedIn.com about business intelligence’s role in the financial crisis. My general take is that if the people who were committing organisations to collateralised debt obligations and other even more esoteric assent-backed securities were unable (or unwilling) to understand precisely the nature of the exposure that they were taking on, then how could this be reflected in BI systems. Good BI systems reflect business realities and risk is one of those realities. However if risk is as ill-understood as it appears to have been in many financial organisations, then it is difficult to see how BI (or indeed it’s sister area of business analytics) could have shed light where the layers of cobwebs were so dense.
So far, so orthodox, but Patrick’s question got me thinking along a different line, one that is more closely related to the ideas that I propounded in Business is from Mars and IT is from Venus last year. I started wondering, ‘is it just too easy for IT to say, “the business people did not understand the risks, so how were we expected to?”?’ (I think I have that punctuation right, but would welcome corrections from any experts reading this). This rather amorphous feeling was given some substance when I read some of the other responses.
However, I don’t want to focus too much on any one comment. My approach will be instead to take a more personal angle and describe some of the thoughts that the comments provoked in me (I am using “provoked” here in a positive sense, maybe “inspired” would have been a better choice of word). If you want to read my comments with the full context, then please click on the link above. What I am going to do here is to present some excerpts from each of my two lengthier contributions. The first of these is as follows (please note that I have also corrected a couple of typos and grammatical infelicities):
Rather than being defensive, and as a BI professional I would probably have every right to be so, I think that Patrick has at least half a point. If some organisations had avoided problems (or mitigated their impact) through the use of good BI (note the adjective) in the current climate, then BI people (me included) would rush to say how much we had contributed. I have certainly done this when the BI systems that I have implemented helped an organisation to swing from record losses to record profits.
Well if we are happy to do this, then we have to take some responsibility when things don’t go so well. It worries me when IT people say that non-IT managers are accountable for the business and IT is just accountable for IT. Surely in a well-functioning organisation, IT is one department that shares responsibility for business success with all the other front-line and service departments.
I have seen it argued with respect to failed financial institutions that IT can only provide information and that other executives take decisions. Well if this is the case, then I question how well the information has been designed to meet business needs and to drive decisions. To me this is evidence of bad BI (note the adjective again).
There are some specific mitigating factors for IT within the current climate, including poor internal (non-IT) governance and the fact that even the people who were writing some financial instruments did not understand the potential liabilities that the we taking on. If this is the case, then how can such risk be rolled up meaningfully? However these factors do not fully exculpate IT in my opinion. I am not suggesting for a second that IT take prime responsibility, but to claim no responsibility whatsoever is invidious.
So yes either poor information, or a lack of information (both of which are IT’s fault – as well as that of non-IT business folk) are a contributory factors to the current problems.
Also, while IT managers see themselves as responsible only for some collateral department, semi-detached from the rest of the business, we will see poor IT and poor information continuing to contribute to business failure.
This is the second passage:
[…]
I just wonder how it is that IT people at such firms can say that any failures are 100% nothing to do with them, as opposed to say 1% responsibility, or something of that nature.
Part of the role of professionals working in BI is to change the organisation so that numerical decision making (backed up of course by many other things, including experience and judgement) becomes part of the DNA. We are to blame for this not being the case in many organisations and can’t simply throw our hands up and say “wasn’t me”.
[…]
I will freely admit that there was a large dose of Devil’s Advocate in my two responses. As I have stated at the beginning of this piece, I am not so masochistic to believe that IT caused the current financial crisis, however I do not think that IT can be fully absolved of all blame.
My concerns about IT’s role relate to the situation that I see in some companies where IT is a department set apart, rather than being a central part of the overall business. In this type of circumstance (which is perhaps more common than anyone would like to think), the success of the IT and the non-IT parts of the business are decoupled.
Under these arrangements, it would be feasible for IT to be successful and the business to suffer major losses, or for the business to post record profits while IT fails to deliver projects. Of couse such decoupling can happen in other areas; for example Product A could have a stellar year, while Product B fails miserably – the same could happen with countries or regions. However there is something else here, a sense that IT can sometimes be an organisation within an organisation, in a way that other service departments generally are not.
Rather than expanding further on this concept here, I recommend you read Jim Anderson’s excellent article Here’s What’s Really Wrong With IT And How To Fix It on his blog, The Business of IT. I think that there is a good deal of alignment between Jim and I on this issue; indeed I was very encouraged to find his blog and see that his views were not a million miles from my own.
I would also like to thank Patrick for posting his initial question. It’s good when on-line forums lead you to take an alternative perspective on things.
This Chase Zander seminar, which I earlier previewed on this site, took place yesterday evening in Birmingham. There was a full house of 20 plus IT Directors, CIOs and other senior IT managers who all engaged fully in some very stimulating and lively discussions.
As I previously mentioned, our intention in this meeting was to encourage debate and sharing of experiences and best practice between the delegates. My role was to faciliate the first session, focussed on IT-Business alignment. I started by sharing a few slides with that group that explained the research we had conducted to determine the content of the forum.
Click to view the introductory presentation as a PDF
After sharing what in my opinion was a not wholly satisfactory definition of IT-Business alignment, I opened up the floor to a discussion of what IT-Business alignment actually was and why it mattered. We used some of the other slides later in the meeting, but most of the rest of the evening was devoted to interaction between the delegates. Indeed the ensuing conversations were so wide ranging that the theme was also carried over to the second session, hosted by my associate Elliot Limb.
Territory initially covered included the suggestion that IT should be an integral part of the business, rather than a separate entity aligned to it (a theme that I covered in my earlier article Business is from Mars and IT is from Venus, which interestingly I penned after a previous Chase Zander forum, this one focussed on change management). The group also made a strong connection between IT-Business alignment and trust. A count of hands in response to the question “do you feel that you have the 100% unqualified confidence of your CEO?” revealed a mixed response and we tried to learn from the experiences of those who responded positively.
The relationship between IT and change was also debated. Some felt that IT, with its experience of project-based work, was ideally placed to drive change in organisations. Others believed that change should be a business function, with IT sticking to its more traditional role. Different organisations were in different places with respect to this issue – one attendee had indeed seen his current organisation take both approaches in the recent past. It was also agreed that there were different types of change: positive change in reaction to some threat or opportunity and the less positive change for change’s sake that can sometimes affect organisations.
Suggestions for enhancing IT-Business alignment included: being very transparent about IT service level agreements and trends in them; focussing more on relationships with senior managers, the CEO and CFO in particular; better calculating the cost of IT activities (including business resource) and using this to prioritise and even directly charge for IT services; applying marketing techniques to IT; learning to better manage business expectations, taking on more realistic workloads and knowing when to say ‘no’; and paying more attention to business processes, particularly via capability maturity modelling.
It was agreed that it generally took quite some time to establish trust between a CIO and the rest of the senior management team. This might be done by initially sorting out problems on the delivery and support side and, only once confidence had been built up, would the CIO be able to focus more on strategic and high value-added activities. This process was not always aided by the not atypical 3-5 year tenure of CIOs.
Later discussions also touched on whether CIOs would generally expect (or want to) become CEOs and, if not, why was this the case. The perspective of both the delegates and the Chase Zander staff was very interesting on this point. There was a degree of consensus formed around the statement that IT people liked taking on challenging problems, sorting them out and then moving on to the next one. While there was some overlap between this perspective and the role of a CEO in both having their hand on the tiller of an organisation and challenging the management team to meet stretch goals, there was less than a perfect fit. Maybe this factor indicated something of a different mindset in many IT professionals.
In the context of forming better relationships with business managers and IT trying to be less transactional in its dealings with other areas, the question of why there were so few women in senior IT positions also came up. This is a large topic that could spawn an entire forum in its own right.
Overall the meeting was judged to be a success. From my perspective it was also interesting to meet a good cross-section of IT professionals working in different industries and to talk about both what the different challenges that we faced and what we had in common.
tactics/táktiks/ n.pl.2 a the plans and means adopted in carrying out a scheme or achieving some end. (O.E.D.)
meander/miándər/ n.1 a a curve in a winding river etc. b a crooked or winding path or passage. (O.E.D.)
I was reminded of the expression “tactical meandering”, which I used to use quite a bit, by a thread on the LinkedIn.com Business Intelligence Group forum. The title of this was Is BI recession-proof? (as always, you need to be a member of both LinkedIn.com and the group to view this).
The conversation on the thread turned to the fact that, in the current economic climate, there may be less focus on major, strategic BI initiatives and more on quick, tactical ones that address urgent business needs.
My take on this is that it is a perfectly respectable approach, indeed it is one that works pretty well in my experience regardless of the economic climate. There is however one proviso, that the short-term work is carried out with one eye on a vision of what the future BI landscape will look like. Of course this assumes that you have developed such a vision in the first place, but if you haven’t why are you talking about business intelligence when report writing is probably what you are engaged in (regardless of how fancy the tools may be that you are using to deliver these).
I talked about this specific area extensively in my earlier article, Holistic vs Incremental approaches to BI and also offered some more general thoughts in Vision vs Pragmatism. In keeping with the latter piece, and although the initial discussions referred to above related to BI, I wanted to use this article to expand the scope to some other sorts of IT projects (and maybe to some non-IT projects as well).
Some might argue (as people did on the LinkedIn.com thread) that all tactical work has to be 100% complementary to you strategic efforts. I would not be so absolute. To me you can wander quite some way from your central goals if it makes sense to do so in order to meet pressing business requirements in a timely and cost-effective manner. The issue is not so much how far you diverge from your established medium-term objectives, but that you always bear these in mind in your work. Doing something that is totally incompatible with your strategic work and even detracts from it may not be sensible (though it may sometimes still be necessary), but delivering value by being responsive to current priorities demonstrates your flexibility and business acumen; two characteristics that you probably want people to associate with you and your team.
Tactical meandering sums up the approach pretty well in my opinion. A river can wander a long way from a line drawn from its source to its mouth. Sometimes it can bend a long way back on itself in order to negotiate some obstacle. However, the ultimate destination is set and progress towards it continues, even if this is sometimes tortuous.
Oxbow Lake Formation
Expanding on the geographic analogy, sometimes meanders become so extreme that the river joins back to its main course, cutting off the loop and leaving an oxbow lake on one side. This is something that you will need to countenance in your projects. Sometimes an approach, or a technology, or a system was efficacious at a point in time but now needs to be dropped, allowing the project to move on. These eventualities are probably inevitable and the important thing is to flag up their likelihood in advance and to communicate clearly when they occur.
My experience is that, if you keep you strategic direction in mind, the sum of a number of tactical meanders can advance you quite some way towards your goals; importantly adding value at each step. The quickest path from A to B is not always a straight line.
Insurance – specifically Property Casualty Insurance – is the industry that I have worked within for the last twelve years. During this time, I managed teams spanning IT, Finance and Operations. However the successes that I am most proud of have been in the related fields of Business Intelligence and Cultural Transformation that appear in the title of this blog.
Insure/insho′or/ v.tr.1 secure the payment of a sum of money in the event of loss or damage to property, life a person etc. (O.E.D.)
Insurance is all about risk; evaluating risk, transferring risk, reducing risk. The essentials of the industry can be appreciated via a rather colourful fable provided in Success in Insurance (S.R. Diacon and R.L. Carter). This tale was originally told by someone at The Association of British Insurers:
Once upon a time there were 11 men; each of them owned a pig.
Unexpectedly one of the pigs died. The owner could not afford £90 for a new pig and so he had to leave the country and go to work in the town instead. The remaining 10 men went to see a wise man. ‘It could happen to any of us,’ they said. ‘What can we do?’
‘Could you each afford £10 for a new pig if your pig died?’ asked the wise man. They all agreed that they could manage that. ‘Very well,’ said the wise man. ‘If you each give me £10, I’ll buy you a pig if yours dies this year.’ They all agreed.
That year one pig did die. The price of pigs had gone up to £95 by now, but the wise man replaced the pig, so none of the men suffered and the wise man had £5 left for the trouble and risk he had taken.
Pricing Insurance products
Of course in the above example, there were two crucial factors for the wise man. First the outcome that only one pig actually died; if instead there had been two pig-related fatalities, the perhaps less-wise man would have been out-of-pocket by £90. Second, the related issue of him setting the price of the pig Insurance policy at £10; if it had been set at £9 he would again have suffered a loss. It is clear that it takes a wise man to make accurate predictions about future events and charge accordingly. In essence this is one thing that makes Insurance different to many other areas of business.
If you work in manufacturing, your job will of course have many challenges, but determining how much it costs to make one of your products should not be one of them. The constituent costs are mostly known and relatively easy to add up. They might include things such as: raw materials and parts; factory space and machinery; energy; staff salaries and benefits; marketing and advertising; and distribution. Knowing these amounts, it should be possible to price a product in such a way that revenue from sales normally exceeds costs of production.
In Insurance a very large part of the cost of production is, by definition, not known at the point at which prices are set. This is the amount that will eventually be paid out in claims; how many new pigs will need to be bought in the example above. If you consider areas such as asbestosis, it can immediately be seen that the cost of Insurance policies may be spread over many years or even decades. The only way to predict the eventual costs of an Insurance product with any degree of confidence, and thereby set its price, is to rely upon historical information to make informed predictions about future claims activity.
By itself, this aspect of Insurance places enormous emphasis on the availability of quality information to drive decisions, but there are other aspects of Insurance that reinforce this basic need.
Distribution strategy
In most areas of commerce the issue of how you get your product to market is a very important one. In Insurance, there are a range of questions in this area. Do you work with brokers or direct with customers? Do you partner with a third party – e.g. a bank, a supermarket or an association – to reach their customers?
Even for Insurance companies that mostly or exclusively work with brokers, which brokers? The broker community is diverse ranging from the large multinational brokers; to middle-sized organisations, that are nevertheless players in a given country or line of business; and to small independent brokers, with a given specialism or access to a niche market. Which segment should an Insurance company operate with, or should it deal with all sectors, but in different ways?
The way to determine an effective broker strategy is again through information about how these relationships have performed and in which ways they are trending. Sharing elements of this type of high-quality information with brokers (of course just about the business placed with them) is also a good way to deepen business relationships and positions the Insurer as a company that really understands the risks that it is underwriting.
Changing risks
The changing face of risk
At the beginning of this article I stated that Insurance is all about risk. As in the pig fable, it is about policy holders reducing their risk by transferring this to an Insurance company that pools these with other risks. External factors can impinge on this risk transfer. Hurricane season is is always a time of concern for Insurance companies with US property exposures, but over the last few years we have had our share of weather-related problems in Europe as well. The area of climate change is one that directly impinges upon Insurers and better understanding its potential impact is a major challenge for them.
With markets, companies, supply-chains and even labour becoming more global, Insurance programmes increasingly cover multiple countries and Insurance companies need to be present in more places (generally a policy covering risks in a country has to be written by a company – or subsidiary – based in that country). This means that Insurance professionals can depend less on first-hand experience of risks that may be on the other side of the world and instead need reliable and consistent information about trends in books of business.
The increasingly global aspect of Insurance also brings into focus different legal and regulatory regimes, which both directly impinge on Insurers and change the profile of risks faced by their customers. As we are experiencing in the current economic crisis, legal and regulatory regimes can sometimes change rapidly, altering exposures and impacting on pricing.
The present economic situation affects Insurance in the same ways that it does all companies, but there are also some specific Insurance challenges. First of all, with the value of companies declining in most markets, there is likely to be an uptick in litigation, leading to an increase in claims against Directors and Officers policies. Also falling property values mean that less Insurance is required to cover houses and factories, leading to a contraction in the market. Declining returns in equity and fixed income markets mean that one element of Insurance income – the return on premiums invested in the period between them being received and any claims being paid out – has become much less.
So shifts in climate, legal and regulatory regimes and economic conditions all present challenges in how risk is managed; further stressing the importance of excellent business intelligence in Insurnace.
The Insurance Cycle
If this litany of problems was not enough to convince the reader of the necessity of good information in Insurance, there is one further issue which makes managing all of the above issues even more complex. This is the fact that Insurance is a cyclical industry.
An example of The Insurance Cycle
The above chart (which I put together based on data from Tillinghast) shows the performance of the London Marine Insurance market as a whole between 1985 to 2002. If you picked any other market in any other location, you would get a similar sinusoidal curve, though there might well be phase differences as the cycles for different types of Insurance are not all in lock-step.
To help readers without a background in Insurance, the ratio displayed is essentially a measure of the amount of money going out of an Insurance Company (mostly its operating expenses plus claims) divided by the amount of money coming in (mostly Insurance premiums). This is called the combined ratio. A combined ratio less than 100% broadly indicates a profit and one above 100% broadly indicates a loss.
It may be seen that the London Marine market as a whole has swung from profit to loss, to profit, to loss and back to profit over these 18 years. This article won’t cover the drivers of this phenomenon in any detail, but one factor is that when profits are being made, more capital is sucked into the market, which increases capacity, drives down costs and eventually erodes profitability. As with many things in life rather than stopping at break-even, this process overshoots resulting in losses and the withdrawal of capital. Prices then rise and profitability returns, starting a new cycle.
Given this environmental background to the Insurance business, it is obvious that it is very important to an Insurance company to work out its whereabouts in the cycle at any time. It is particularly crucial to anticipate turning points because this is when corporate strategies may need to change very rapidly. There may be a great opportunity for defence to change to attack, alternatively a previously expansionary strategy may need to be reined in order to weather a more trying business climate.
In order to make predictions about the future direction of the cycle, there is no substitute for having good information and using this to make sound analyses.
Summary
I hope that the article has managed to convey some of the special challenges faced by Insurance companies and why many of these dramatically increase the value of good business intelligence.
Essentially Insurance is all about making good decisions. Should I underwrite this newly presented risk? Should I renew an existing policy or not? What price should I set for a policy? When should I walk away from business? When should I aggressively expand? All of these decisions are wholly dependent on having high-quality information and because of this business intelligence can have an even greater leverage in Insurance than in other areas of industry.
Given this it is not unreasonable to state in closing that while good information is essential to any organisation, it is the very lifeblood of an Insurance company. My experience is that Business Intelligence offers the best way to meet these pressing business needs.
You can read more about my thoughts on Business Intelligence and Insurance in:
Back in January, in collaboration with Chase Zander, I started a process of consulting with senior IT managers to develop a list of the top business issues that they faced. This exercise was intended to shape the content of a IT Director Forum that we were planning. This will now be happening on 26th March in Birmingham (for further information see this post).
The Top Business Issues faced by CIOs / IT Directors
Back then, I promised to share some of the findings from this study. These are summarised in the above diagram. The input was based on public comments made by a selection of senior people on the CIO group of LinkedIn.com, plus e-mails sent to me on the topic and feedback received by Chase Zander.
A textual version of the data appeas below (sample size ~60):
Issue
% of Votes
IT / Business Alignment
27%
Cost-saving
13%
Managing change
8%
Status of the IT Director
8%
Legacy Systems
5%
Customer focus
5%
Enterprise Architecture
5%
Business Intelligence
5%
Avoiding the latest and greatest
3%
Cloud Computing
3%
Only one response
17%
Total
100%
I would like to thank all of the IT professionals who contributed to this survey.
You must be logged in to post a comment.