Automating the business intelligence process?

14 May 2009
Balanced Insight Merv Adrian - IT Market Strategies for Suppliers
Balanced Insight Merv Adrian

 
 
Introduction

I enjoy reading the thoughts of vastly experienced industry analyst Merv Adrian on his blog, Market Strategies for IT Suppliers, and also on twitter via @merv. Merv covers industry trends and a wide variety of emerging and established technologies and companies. I would encourage you to subscribe to his RSS feed.

In a recent artcile, Balanced Insight – Automating BI Design to Deployment, Merv reviews the Consensus tool and approach developed by Ohio-based outfit Balanced Insight. I suggest that you read Merv’s thoughts first as I won’t unnecessarily repeat a lot of what he says here. His article also has links to a couple of presentations featuring the use of Consensus to build both Cognos 8 and Proclarity prototypes, which are interesting viewing.
 
 
An overview of Balanced Insight

Disclaimer: I haven’t been the beneficiary of a briefing from Balanced Insight, and so my thoughts are based solely on watching their demos, some information from their site and – of course – Merv’s helpful article.

The company certainly sets expectations high with the strap line of their web site:

Agile & Aligned Business Intelligence - With Balanced Insight Consensus® deliver in half the time without compromising cross project alignment.

Promising to “deliver in half the time without compromising cross project alignment” is a major claim and something that I will try to pay close attention to later.

The presentations / demonstrations start with a set-up of a fictional company (different ones in different demos) who want to find out more about issues in their business: outstanding receivables, or profit margins [Disclosure: the fact that the second demo included margins on mountain bikes initially endeared me to the company]. In considering these challenges, Balanced Insight offers the following slide contrasting IT’s typical response with the, presumably superior, one taken by them:

IT's approach to information problems vs Balanced Insight's

IT's approach to information problems vs Balanced Insight's

I agree with Balanced Insight’s recommendation, but rather take issue with the assumption that IT always starts by looking exclusively at data when asked to partake in information-based initiatives. I have outlined what I see as the four main pillars of a business intelligence project at many places on this blog, most recently in the middle of my piece on Business Intelligence Competency Centres. While of course it is imperative to understand the available data (what would be the alternative?), the first step in any BI project is to understand the business issues and, in particular, the questions that the business wants an answer to. If you search the web for BI case studies or methodologies, I can’t imagine many of these suggesting anything other than Balanced Insight’s recommended approach.

Moving on, the next stage of both the demos introduces the company’s “information packages”. These are panes holding business entities and have two parts; the upper half contains “Topics and Categories” (things such as date or product), the bottom half contains measurements. The “Topics and Categories” can be organised into hierarchies, for example: day is within week, which is within month, quarter and year. At this point most BI professionals will realise that “Topics and Categories” are what we all call “Dimensions” – but maybe Balanced Insight have a point picking a less technical-sounding name. So what the “information package” consists of is a list of measures and dimensions pertaining to a particular subject area – it is essentially a loose specification for a data mart.

The interesting point is what happens next, the Consensus Integrator uses the “information package” to generate what the vendor claims is an optimised star-schema database (in a variety of databases). It then creates a pre-built prototype that references the schema; this can be in a selection of different BI tools. From what I can tell from the demos, the second stage appears to consist of creating an XML file that is then read by the BI tool. In the first example, the “Topics and Categories” become dimensions in Cognos AnalysisStudio and the measures remain measures. In both demos sample data is initially used, but in the ProClarity one a version with full data is also shown – it is unclear whether this was populated via Consensus or not. The “information package” can also be exported to data modelling tools such as ERwin.

One of the Balanced Insight presentations then mentions that “all that’s left to do is then to develop your ETL”. I appreciate that it is difficult to go into everything in detail in a short presentation, but this does rather seem to be glossing over a major area, indeed one of my four pillars of BI projects referred to above. Such rather off-hand comments do not exactly engender confidence. If there is a better story to tell here, then Balanced Insight’s presentations should try to tell it.
 
 
The main themes

There are a few ideas operating here. First that Balanced Insight’s tools can support a process which will promote best practice in defining and documenting the requirements of a BI project and allow a strong degree of user interaction. Second that the same tools can quickly and easily produce functioning prototypes that can be used to refine these same requirements and also make discussions with business stakeholders more concrete. Finally that the prototypes can employ a variety of database and BI tools – so maybe you prototype on a cheap / free database and BI tool, then implement on a more expensive, and industrial strength, combination later.

Balanced Insight suggest that their product helps to address “the communication gap between IT and the business”. I think it is interesting using the “information package” as a document repository, which may be helpful at other stages of the project. But there are other ways of achieving this as well. How business friendly these are probably depends on how the BI team set them up. I have seen Excel and small Access databases work well without even buying a specific tool. Also I think that if a BI team needs a tool to ensure it sticks to a good process, then there is probably a bigger problem to worry about.

Of course, the production of regular prototypes is a key technique to employ in any BI project and it seems that Balanced Insight may be on to something here, particularly if the way that their “information package” presents subject areas makes it easier for the BI team and business people to discuss things. However, it is not that arduous to develop prototypes directly in most BI tools. To put this in a context drawn from my own experience, building Cognos cubes to illustrate the latest iteration of business requirement gathering was often a matter of minutes, compared to business analysts putting in many days of hard work before this stage.

Having decided to use Consensus to capture information about measures and dimensions, the ability to then transfer these to a range of BI tools in interesting. This may offer the opportunity to change tools during the initial stages of the project and to try out different tools with the same schema and data to assess their effectiveness. This may also be something that is a useful tool when negotiating with BI vendors. However, again I am not sure exactly how big of a deal this is. I would be interested in better understanding how users have taken advantage of this feature.
 
 
A potential fly in the ointment

It would be easy to offer a couple of other criticisms of the approach laid out in the demos; namely that it seems to be targeted at developing point solutions rather than a pervasive BI architecture and that (presumably related to this) the examples shown are very basic. However, I’m willing to given them the benefit of the doubt, a sales pitch is probably not the place for a lengthy exploration of broad and complex issues. So I think my overall response to Balanced Insight’s Consensus product could be summed up as guardedly positive.

Nevertheless, there is one thing that rather worries me and this can best be seen by looking at the picture below. [As per the disclaimer above, the following diagram is based on my own understanding of the product and has not been provided by Balanced Insight.]

My perception of how Balanced Insight addresses needs for information

My perception of how Balanced Insight addresses needs for information

I think I understand the single black arrow on the right of the diagram, I’m struggling to work out what Consensus offers (aside from documentation) for the two black arrows on the left hand side. Despite the fact that Balanced Insight disparaged the approach of looking at available data in their presentation, there is no escaping the fact that some one will have to do this at some point. Connections will then have to be made between the available data and the business questions that need answering.

In both demos Consensus is pre-populated with dimensions, measures and linkages of these to sample data. How this happens is not covered, but this is a key area for any BI project. Unless Balanced Insight have some deus ex machina that helps to cut the length of this stage, then I begin to become a little sceptical about their claim to halve the duration of BI work.

Of course my concerns could be unfounded. It will be interesting to see how things develop for the company and whether their bold claims stand the test of time.
 

tweet this Tweet this article on twitter.com
Bookmark this article with:
Technorati | del.icio.us | digg | Reddit | Stumble

 

1 Comment | balanced insight, business, business intelligence, it business alignment, management information, project management, technology | Tagged: , , , , , , | Permalink
Posted by Peter Thomas

Business Intelligence Competency Centres

11 May 2009

Introduction

The subject of this article ought to be reasonably evident from its title. However there is perhaps some room for misinterpretation around even this. Despite the recent furore about definitions, most reasonable people should be comfortable with a definition of business intelligence. My take on this is that BI is simply using information to drive better business decisions. In this definition, the active verb “drive” and the subject “business decisions” are the key elements; something that is often forgotten in a rush for technological fripperies.
 
 
The central issue

Having hopefully addressed of the “BI” piece of the BICC acronym, let’s focus on the “CC” part. I’ll do this in reverse order, first of all considering what is meant by “centre”. As ever I will first refer to my trusted Oxford English Dictionary for help. In a discipline, such as IT, which is often accused of mangling language and even occasionally using it to obscure more than to clarify, a back-to-basics approach to words can sometimes yield unexpected insights.

  centre / séntər / n. & v. (US center) 3 a a place or group of buildings forming a central point in a district, city, etc., or a main area for an activity (shopping centre, town centre).
(O.E.D.)		  

Ignoring the rather inexcusable use of the derived adjective “central” in the definition of the noun “centre”, then it is probably the “main area for an activity” sense that is meant to be conveyed in the final “C” of BICC. However, there is also perhaps some illumination to be had in considering another meaning of the word:

Centre of a Sphere

  n. 1 a the middle point, esp. of a line, circle or sphere, equidistant from the ends, or from any point on the circumference or surface.
(O.E.D.)		  

As well as appealing to the mathematician in me, this meaning gives the sense that a BICC is physically central geographically, or metaphorically central with respect to business units. Of course this doesn’t meant than a BICC needs to be at the precise centre of gravity of an organisation, with each branch contributing a “weight” calculated by its number of staff, or revenue; but it does suggest that the competency centre is located at a specific point, not dispersed through the organisation.

Of course, not all organisations have multiple locations. The simplest may not have multiple business units either. However, there is a sense by which “centre” means that a BICC should straddle whatever diversity there is an organisation. If it is in multiple countries, then the BICC will be located in one of these, but serve the needs of the others. If a company has several different divisions, or business units, or product streams; then again the BICC should be a discrete area that supports all of them. Often what will make most sense is for the BICC to be located within an organisation’s Head Office function. There are a number of reasons for this:

  1. Head Office similarly straddles geographies and business units and so is presumably located in a place that makes sense to do this from (maybe in an organisation’s major market, certainly close to a transport hub if the organisation is multinational, and so on).
  2. If a BICC is to properly fulfil the first two letters of its abbreviation, then it will help if it is collocated with business decision-makers. Head Office is one place than many of these are found, including generally the CEO, the CFO, the Head of Marketing and Business Unit Managers. Of course key decision makers will also be spread throughout the organisation (think of Regional and Country Managers), but it is not possible to physically collocate with all of these.
  3. Another key manager who is hopefully located in Head Office is the CIO (though this is dispiritingly not always the case, with some CIOs confined to IT ghettos, far from the rest of the executive team and with a corresponding level of influence). Whilst business issues are pre-eminent in BI, of course there is a major technological dimension and a need to collaborate closely with those charged with running the organisation’s IT infrastructure and those responsible for care and feeding of source data systems.
  4. If a BI system is to truly achieve its potential, then it must become all pervasive; including a wide range of information from profitability, to sales, to human resources statistics, to expense numbers. This means that it needs to sit at the centre of a web of different systems: ERP, CRM, line of business systems, HR systems etc. Often the most convenient place to do this from will be Head Office.

Thusfar, I haven’t commented on the business benefits of a BICC. Instead I have confined myself to explaining what people mean by the second “C” in the name and why this might be convenient. Rather than making this an even longer piece, I am going to cover both the benefits and disadvantages of a BICC in a follow-on article. Instead let’s now move on to considering the first “C”: Competency.
 
 
Compos centris

Returning to our initial theme of generating insights via an examination of the meaning of words in a non-IT context, let’s start with another dictionary definition:

Motar board

  competence /kómpit’nss/ n. (also competency /kómpitənsi/) 1 (often foll. by for, or to + infin.) ability; the state of being competent.

and given the recursive reliance of the above on the definition of competent…
  competent /kómpit’nt/ adj. 1 a (usu. foll. by to + infin.) properly qualified or skilled (not competent to drive); adequately capable, satisfactory. b effective (a competent bastman*).
(O.E.D.)		  

* People who are not fully conversant with the mysteries of cricket may substitute “batter” here.

To me the important thing to highlight here is that, while it is to be hoped that a BICC will continue to become more competent once it is up and running, in order to successfully establish such a centre, a high degree of existing competence is a prerequisite. It is not enough to simply designate some floor space and allocate a number of people to your BICC, what you need is at least a core of seasoned professionals who have experience of delivering transformational information and know how to set about doing it.

There are many skills that will be necessary in such a group. These match the four main pillars of a BI implementation (I cover these in more depth in several places on the blog, including BI implementations are like icebergs and the middle section of Is outsourcing business intelligence a good idea?):

  1. Understand the important business decisions and what figures are necessary to support these.
  2. Understand the data available in the organisation, how it relates to other data and to business decisions.
  3. Transform the data to provide information answering business questions.
  4. Focus on embedding the use of information in the corporate DNA.

So a successful BICC must include: people with strong analytical skills and an understanding of general business practices; high-calibre designers; reliable and conscientious ETL and general programmers; experts in the care, feeding and design of databases; excellent quality assurance professionals; resource conversant with both whatever front-end tools you are using to deliver information and general web programming; staff with skills in technical project management; people who can both design and deliver training programmes; help desk personnel; and last, but by no means least, change managers.

Of course if your BI project is big enough, then you may be able to afford to have people dedicated to each of these roles. If resources are tighter (and where is this not the case nowadays?) then it is better to have people who can wear more than one hat: business analysts who can also design; BI programmers who will also take support calls; project managers who will also run training classes; and so on. This approach saves money and also helps to deal with the inevitable peaks and troughs of resource requirements at different stages in a project. I would recommend setting things up this way (or looking to stretch your people’s abilities into new areas) even if you have the luxury of a budget that would allow a more discrete approach. The challenge of course is going to be finding and retaining such multi-faceted staff.

Also, it hopefully goes without saying that BI is a very business-focussed area and some BICCs will explicitly include business people in them. Even if you do not go this far, then the BICC will have to form a strong partnership with key business stakeholders, often spread across multiple territories. The skill to manage this effectively is in itself a major requirement of the leading personnel of the centre.

Given all of the above, the best way to staff a BICC is with members of a team who have already been successful with a BI project within your organisation; maybe one that was confined to a given geographic region or business unit. If you have no such team, then starting with a BICC is probably a bridge too far. Instead my recommendation would be to build up some competency via a smaller BI project. Alternatively, if you have more than one successful BI team (and, despite the manifold difficulties in getting BI right, such things are not entirely unheard of) then maybe blending these together makes sense. This is unless there is some overriding reason not to (e.g. vastly different team cultures or methodologies. In this case, picking a “winner” may be a better course of action.

Such a team will already have the skills outlined above in abundance (else they could never have been successful). It is also likely that whatever information was needed in their region or business unit will be at least part of what is needed at the broader level of a BICC. Given that there are many examples of BI projects not delivering or consuming vastly more resource than anticipated, then leveraging those exceptional people who have managed to swim against this tide is eminently sensible. Such battle-hardened professionals will know what pitfalls to avoid, which areas are most important to concentrate on and can use their existing products to advertise the benefits of a wider system. If you have such people at the core of your BICC, then it will be easier to integrate new joiners and quickly shepherd them up the learning curve (something that can be particularly long in BI due to the many different aspects of the work).

Of course having been successful in one business unit or region is not enough to guarantee success on a larger scale. I spoke about some of the challenges of doing this in an earlier article, Developing an international BI strategy. Another issue that is likely to raise its head is the political dimension, in particular where different business units or regions already have a management information strategy at some stage of development. This is another area that I will also cover in more detail in a forthcoming piece.
 
 
Conclusions

It seems that simply musing on the normal meanings of the words “competency” and “centre” has led us into some useful discussions. As mentioned above, at least two other blog postings will expand upon areas that have been highlighted in this piece. For now what I believe we have learned so far is:

  • BICCs should (by definition) straddle multiple geographies and/or business units.
  • There are sound reasons for collocating the BICC with Head Office.
  • There is need for a wide range of skills in your BICC, both business-focussed and technical.
  • At least the core of your BICC should be made up of competent (and experienced) BI professionals .

More thoughts on the benefits and disadvantages of business intelligence competency centres and also the politcs that they have to negotiate will appear on this blog in future weeks.
 

tweet this Tweet this article on twitter.com
Bookmark this article with:
Technorati | del.icio.us | digg | Reddit | NewsVine

 

12 Comments | business, business intelligence, business intelligence competency centres, change management, cultural transformation, international projects, management, management information, project management, technology | Tagged: , , , , , | Permalink
Posted by Peter Thomas

The importance of feasibility studies in business intelligence

6 May 2009

Introduction

Feasibility Study

In a previous article, A more appropriate metaphor for business intelligence projects, I explained one complication of business intelligence projects. This is that the frequently applied IT metaphor of building is not very applicable to BI. Instead I suggested that BI projects had more in common with archaeological digs. I’m not going to revisit the reasons for the suitability of looking at BI this way here, take a look at the earlier piece if you need convincing, instead I’ll focus on what this means for project estimation.

When you are building up, estimation is easier because each new tier is dependent mostly on completion of the one below, something that the construction team has control over (note: for the sake of simplicity I’m going to ignore the general need to dig foundations for buildings). In this scenario, the initial design will take into account of facts such as the first tier needing to support all of the rest of the floors and that central shafts will be needed to provide access and deliver essential services such as water, electricity and of course network cables. A reductionist approach can be taken, with work broken into discrete tasks, each of which can be estimated with a certain degree of accuracy. The sum of each of these, plus some contingency, hopefully gives you a good feel for the overall project. It is however perhaps salutary to note that even when building up (both in construction and in IT) estimation can still sometimes go spectacularly awry.

When you are digging down, your speed is dependent on what you find. Your progress is dictated by things that are essentially hidden before work starts. If your path ahead (or downwards) is obscured until your have cleared enough earth to uncover the next layer, then each section may hold unexpected surprises and lead to unanticipated delays. While it may be possible to say things like, “well we need to dig down 20m and each metre should take us 10 days”, any given metre might actually take 20 days, or more. There are two issues here; first it is difficult to reduce the overall work into tasks, second it is harder to estimate each task accurately. The further below ground a phase of the dig is, the harder it will be to predict what will happen before ground is broken. Even with exploratory digs, or the use of scanning equipment, this can be very difficult to assess in advance. However it is to the concept of exploratory digs that this article is devoted.
 
 
Why a feasibility study is invaluable

At any point in the economic cycle, even more so in today’s circumstances, it is not ideal to tell your executive team that you have no idea how long a project will take, nor how much it might cost. Even with the most attractive of benefits to be potentially seized (and it is my firm belief that BI projects have a greater payback than many other types of IT projects), unless there is some overriding reason that work must commence, then your project is unlikely to gain a lot of support if it is thus characterised. So how to square the circle of providing estimates for BI projects that are accurate enough to present to project sponsors and will not subsequently leave you embarrassed by massive overruns?

It is in addressing this issue that BI feasibility studies have their greatest value. These can be thought of as analogous to the exploratory digs referred to above. Of course there are some questions to be answered here. By definition, a feasibility study cannot cover all of the ground that the real project needs to cover, choices will need to be made. For example, if there are likely to be 10 different data sources for your eventual warehouse, then should you pick one and look at it in some depth, or should you fleetingly examine all 10 areas? Extending our archaeological metaphor, should your exploratory dig be shallow and wide, or a deep and narrow borehole?
 
 
A centre-centric approach

In answering this question, it is probably worth considering the fact that not all data sources are alike. There is probably a hierarchy to them, both in terms of importance and in terms of architecture. No two organisations will be the same, but the following diagram may capture some of what I mean here:

Two ways of looking at a systems' hierarchy

Two ways of looking at a systems' hierarchy

The figure shows a couple of ways of looking at your data sources / systems. The one of the left is rather ERP-centric, the one on the right gives greater prominence to front-end systems supporting different lines of business, but wrapped by a common CRM system. There are many different diagrams that could be drawn in many different ways of course. My reason for using concentric circles is to stress that there is often a sense in which information flows from the outside systems (ones primarily focussed on customer interactions and capturing business transactions) to internal systems (focussed on either external or internal reporting, monitoring the effectiveness of processes, or delivering controls).

There may be several layers through which information percolates to the centre; indeed the bands of systems and databases might be as numerous as rings in an onion. The point is that there generally is such a logical centre. Data is often lost on its journey to this centre by either aggregation, or by elements simply not being transferred (e.g. the name of a salesperson is not often recorded on revenue entries in a General Ledger). Nevertheless the innermost segment of the onion is often the most complex, with sometimes arcane rules governing how data is consolidated and transformed on its way to its final destination.

The centre in both of the above diagrams is financial and this is not atypical if what we are considering is an all-pervasive BI system aimed at measuring most, if not all, elements of an organisation’s activity (the most valuable type of BI system in my opinion). Even if your BI project is not all-pervasive (or at least the first phase is more specific), then the argument that there is a centre will probably still hold, however the centre may not be financial in this case.

My suggestion is that this central source of data (of course there may be more than one) is what should be the greatest focus of your feasibility study. There are several reasons for this, some technical, some project marketing-related:

  1. As mentioned above, the centre is often the toughest nut to crack. If you can gain at least some appreciation of how it works and how it may be related to other, more peripheral systems, then this is a big advance for the project. Many of the archaeological uncertainties referred to above will be located in the central data store. Other data sources are likely to be simpler and thus you can be more confident about approaching these and estimating the work required.
  2. A partial understanding of the centre is often going to be totally insufficient. This is because your central analyses will often have to reconcile precisely to other reports, such as those generated by your ERP system. As managers are often measured by these financial scorecards, if you BI system does not give the same total, it will have no credibility and will not be used by these people.
  3. Because of its very nature, an understanding of the centre will require at least passing acquaintance with the other systems that feed data to it. While you will not want to spend as much time on analysing these other systems during the feasibility study, working out some elements of how they interact will be helpful for the main project.
  4. One output from your feasibility study should be a prototype. While this will not be very close to the finished article and may contain data that is both unreconciled and partial (e.g. for just one country or line of business), it should give project sponsors some idea of what they can expect from the eventual system. If this prototype deals with data from the centre then it is likely to be of pertinence to a wide range of managers.
  5. Strongly related to the last point, and in particular if the centre consists of financial data, then providing tools to analyse this is likely to be something that you will want to do early on in the main project. This is both because this is likely to offer a lot of business value and because, if done well, this will be a great advert for the rest of your project. If this is a key project deliverable, then learning as much as possible about the centre during the feasibility study is very important.
  6. Finally what you are looking to build with your BI system is an information architecture. If you are doing this, then it makes sense to start in the middle and work your way outwards. This will offer a framework off of which other elements of your BI system can be hung. The danger with starting on the outside and working inwards is that you can end up with the situation illustrated below.

A possible result of building from the outside in to the center

A possible result of building from the outside in to the centre


 
Recommendations

So my recommendation is that your feasibility study is mostly a narrow, deep dig, focussed on the central data source. If time allows it would be beneficial to supplement this with a more cursory examination of some of the data sources that feed the centre, particularly as this may be necessary to better understand the centre and because it will help you to get a better idea about your overall information architecture. You do not need to figure out every single thing about the central data source, but whatever you can find out will improve the accuracy of your estimate and save you time later. If you include other data sources in a deep / wide hybrid, then these can initially be studied in much less detail as they are often simpler and the assumption is that they will support later deliveries.

The idea of a prototype was mentioned above. This is something that is very important to produce in a feasibility study. Even if we take to one side the undeniable PR value of a prototype, producing one will allow you to go through the entire build process. Even if you do this with hand-crafted transformation of data (rather than ETL) and only a simplistic and incomplete approach to the measures and dimensions you support, you will at least have gone through each of the technical stages required in the eventual live system. This will help to shake out any issues, highlight areas that will require further attention and assist in sizing databases. A prototype can also be used to begin to investigate system and network performance, things that will influence your system topology and thereby project costs. A better appreciation of all of these areas will help you greatly when it comes to making good estimates.

Having understood quite a lot about your most complex data source and a little about other ones and produced a prototype both as a sales tool and to get experience of the whole build process, you should have all the main ingredients for making a credible presentation to your project sponsors. In this it is very important to stress the uncertainties inherent in BI and manage expectations around these. However you should also be very confident in stating that you have done all that can be done to mitigate the impact of these. This approach, of course supported by a compelling business case, will position you very well to pitch your overall BI project.
 

tweet this Tweet this article on twitter.com
Bookmark this article with:
Technorati | del.icio.us | digg | Reddit | NewsVine

 

3 Comments | bi and the economic crisis, business, business intelligence, data warehousing, management information, project management, technology | Tagged: , , , , , | Permalink
Posted by Peter Thomas

“Why do CFOs and CEOs hate IT? – ERP” – Thomas Wailgum at CIO.com

27 April 2009

Thomas Wailgum at CIO.com

This is my second article in response to pieces by Thomas Wailgum at CIO.com (you can read the first one here). In Thomas’ latest piece, entitled Why CFOs and CEOs Hate IT: ERP, he touches on an area of which I have lengthy experience, ERP.

I spent the first eight years of my career working for a a software house, whose central product was in what we now call the ERP space. The big boys at Oracle Financials (then without PeopleSoft and J.D. Edwards in-train) were one of our main rivals and I had the pleasure of being involved in several bids where the little guys prevailed against their more renowned competition.

Later in my career, I was a player in a global selection process involving Oracle, PeopleSoft (then a separate company) and SAP and in laying the foundations for a US/European PeopleSoft implementation. Many years later again, after I had recorded a number of successes in another of my core areas, business intelligence, I was asked to add Financial IT once more to my portfolio. In this capacity, I oversaw the implementation of (by this time) Oracle PeopleSoft Financials in Denmark, Italy and then Australia, Hong Kong, Labuan and Singapore.

So, in one way or another, ERP and I have been around the block a few times. Given this, I could identify with some of Thomas’ observations. Many of these can be summed up in the phrase “an ERP system is for life, not just for Christmas.” Here are a few of Thomas’ thoughts:

A typical company in the CFO survey will spend an average of $1.2 million each year (each year!) to maintain, modify and update its ERP system.

ERP systems have become a noose around companies’ necks which tighten as the business changes every year, each customization gets made to the system and costs continue to spiral upward.

In some ways, ERP implementation is just like any other IT project and is difficult to get right for exactly the same reasons. But, as Thomas points out, some things that make ERP stand out are the massive initial outlays, the continuing cost of modifying what you originally thought you needed and the sheer size and complexity of most modern ERP systems.

You can think of your average ERP system from one of the large vendors as analogous to Microsoft Word. Because Word has to appeal to a lot of different users, with different needs and specialisms, it is chock-full of every single feature that anyone could ever need. However, no single person ever uses more than a fraction of these. I think of myself as a reasonably advanced Word user, but I would bet that I utilise no more than 10% of its capabilities. All of the functionality can make it tough for an entry-level user to employ Word in a basic way to do basic things (or if we are talking about Word 2007, it makes it tough for even an expert user to figure out how to do stuff). The same criticism can be applied to ERP systems. Because they include so much functionality for different companies in different industries, it can sometimes be difficult to configure them to do something as simple as entering and paying an invoice. Difficult that is without an army of consultants.

The way to avoid complexities and to get ERP implemented on time and budget is to ignore its broader capabilities and deploy as plain vanilla a version as you can get away with. Flexibility and the ability to customise might be very seductive at sales time, but they are the worst enemy of implementation and are certain to chew up resource, time and money. Instead the secret is to focus on the ways in which Finance in your organisation is the same as it most other organisations. Once you have this sorted out and a basically successful system in place, you can then think about bells and whistles. Of course by this time, you will probably be focused on upgrading to the latest version of your ERP system, but let’s put this unpleasant thought to one side for the purposes of this discussion.

But this begs another question, which Thomas covers more eloquently that I could. Plain vanilla ERP implementations, where you essentially adapt what your organisation does to the system’s standard functionality, mean that:

[...] employees who actually have to use the ERP system day in, day out will not only dislike the fact that you’re changing their technology interface, but now you’re going to allow the technology system to dictate to them how they should perform their job, with the new business processes.

Hercules and the Hydra - Antonio del Pollaiolo

However, even if we can suppress this second inconvenient truth about ERP, a further one arises – the area is indeed hydra-like. If the best practice for ERP implementations is to customise them as little as possible – shortening projects, reducing costs and simplifying upgrades – then why is there such a large price tag for all of the bells and whistles that it is impractical to actually use?

As Mr Wailgum says in closing:

But, perhaps, [CEOs and CFOs] have been making these decisions without knowing all the facts about the long-term costs associated with ERP systems, that the upfront “sticker price” is almost meaningless.

Which brings us right back to why CFOs and CEOs hate IT.

 

 
Starting in 1987 with CIO magazine, CIO’s portfolio of properties has grown to provide technology and business leaders with insight and analysis on information technology trends and a keen understanding of IT’s role in achieving business goals. The magazine and website have received more than 160 awards to date, including two Grand Neal Awards from the Jesse H. Neal National Business Journalism Awards and two National Magazine of the Year awards from the American Society of Publication Editors.
 

tweet this Tweet this article on twitter.com
Bookmark this article with:
Technorati | del.icio.us | digg | Reddit | NewsVine

 

16 Comments | business, erp, international projects, project management, technology | Tagged: , , | Permalink
Posted by Peter Thomas

Synthesis

18 April 2009

RNA Polymerase producing mRNA from a double-stranded DNA template

  synthesis /sinthisiss/ n. (pl. syntheses /-seez/) 1 the process of building up separate elements, esp. ideas, into a connected whole, esp. a theory or system. (O.E.D.)  

Yesterday’s post entitled Recipes for success? seems to have generated quite a bit of feedback. In particular I had a couple of DMs from people I know on twitter.com (that’s direct messages for the uninitiated) and some e-mails, each of which asked me why I was so against business books. One person even made the assumption that I was anti-books and anti-learning in general.

I guess I need to go on a course designed to help people to express themselves more clearly. I am a bibliophile and would describe myself as fanatically pro-learning. As I mentioned in a comment on the earlier article, I was employing hyperbole yesterday. I would even go so far as to unequivocally state that some business books occasionally contain a certain amount of mildly valuable information.

Of course, when someone approaches a new area, I would certainly recommend that they start by researching what others have already tried and that they attempt to learn from what has previously worked and what has not. Instead, the nub of my problem is when people never graduate beyond this stage. More specifically, I worry when someone finds a web-article listing “10 steps that, if repeated in the correct sequence, will automatically lead to success” and then uncritically applies this approach to whatever activity they are about to embark on.

Assuming that the activity is something more complicated than assembling Ikea furniture, I think it pays to do two further things: a) cast your net a little wider to gather a range of opinions and approaches, and b) assemble your own approach, based borrowing pieces from different sources and sprinkling this with your own new ideas, or maybe things that have worked for you in the past (even if these were in slightly different areas). My recommendation is thus not to find the methodology or design that most closely matches your requirements, but rather to roll your own, hopefully creating something that is a closer fit.

This act of creating something new – based on research, on leveraging appropriate bits of other people’s ideas, but importantly adding your own perspective and tweaking things to suit your own situation – is what I mean by synthesis.

Of course maybe what you come up with is not a million miles from one of the existing prêt-à-porter approaches, but it may be an improvement for you in your circumstances. Also, even if your new approach proves to be suboptimal, you have acquired something important; experience. Experience will guide you in your next attempt, where you may well do better. As they saying maybe ought to go – you learn more from your own mistakes than other people’s recipes for success.
 

 
Addendum

The WordPress theme I use for this blog – Contempt – was written by Michael Heilemann a self-styled “Interface Designer, Web Developer, former Computer Game Developer and Film Lover”. Michael also writes a blog, Binary Bonsai and I felt that his article, George Lucas stole Chewbacca, but it’s OK, summed up (if you can apply the concept of summation to so detailed a piece of writing) a lot of what I am trying to cover in this piece. I’d recommend giving it a read, even if you aren’t a Star Wars fan-boy.
 

tweet this Tweet this article on twitter.com
Bookmark this article with:
Technorati | del.icio.us | digg | Reddit | NewsVine

 

10 Comments | Biology, business, management, project management, technology | Tagged: , , | Permalink
Posted by Peter Thomas

Recipes for success?

17 April 2009

I should acknowledge that I am indebted to a conversation that I had with John Collins on his blog, Views from the Bridge, for some of the themes I discuss in this article.
 
Recipe for Success?
 
Introduction

Towards the end of a recent article on perseverance I referred to people’s desire to find recipes for success. Here’s what I said:

Sometimes we want to find a magic recipe for success, or – to mix the metaphor – a silver bullet. We want to discover a series of defined steps to take that, if repeated religiously, will guarantee that we get to the desired goal each and every time. That’s why articles entitled “The 5 ways to [...]” and “My top tips for [...]” are so well-read on the web.

As well as my examples of internet top tips (see any number of articles claiming to tell you how to use twitter successfully to get the idea), this phenomenon is also a major factor behind the enduring popularity of celebrity business books. As far as I can see, these fall into two categories.
 
 
1. The Ex-CEO

This is where the extremely successful and well-known Mr Brown (and sadly it is still mostly Mr, rather than Ms Brown), now retired but previously President and CEO of Big Company Inc., writes (or more likely has some one ghost-write) a memoir explaining the secrets of his success. While the book may dwell on their upbringing, education, role models, or character-forming events in their lives, much of the work will probably focus on them just being much smarter, more risk-taking, or having greater insight than the competition (most likely all of these). Of course there may well be some interesting tit-bits amongst the reams of self-aggrandisement, but it is worth questioning just how applicable these might be to your own situation.

Are the things that Mr Brown ascribes his success to really what led to his glittering career? Are there perhaps other factors that are not captured in the memoir, but which, if absent in another organisation, would render implementing Mr Brown’s explicit recommendations valueless? Did Mr Brown’s greatest achievements actually have a big slice of luck attached to them (stumbling upon a market or a product by accident, a major competitor losing their way, events beyond anyone’s control shaping matters and so on)? Would the things that Big Company Inc. did under Mr Brown’s esteemed leadership actually work in another company, in a different market or country and with a distinctive business culture?

Put it this way, if you work in Financial Services, would copying what worked in Retail be a good idea? Alternatively, if two companies are both in Retail, does it make sense for a less successful company to slavishly adopt the strategy of the market leader – wouldn’t it be more sensible if they tried to develop a different strategy in order to differentiate their brand?

Of course there is always value in learning from the mistakes and successes of others, but surely there is a limit to how useful a business memoir can be in forming a business strategy.
 
 
2. The Academic Expert

Here Professor Green (probably still male), has a long and distinguished career in academia, reading and deconstructing the memoirs of Mr Brown and his peers, identifying common themes between them, doing primary research and constructing recherché models of business strategy development and execution. If there is a new management fad out there, Professor Green is sure to know about it – in fact it may well be based on an article of his that appeared in HBR.

Well there is certainly some value in trying to tease out commonalities between successful companies, but this is probably a lot harder than it might seem. While there may be some recurring themes, maybe many of our champions of business are one offs, successful for reasons other than their business models or strategies. In fact they may well be as unique as the people who lead them. Maybe there is no equivalent of the standard model of quantum mechanics (to say nothing of a deeper grand unified theory) that underpins business success – perhaps the science of business is different from the more reductionist sciences, such as physics. Maybe there isn’t a formula for business success; perhaps it is more like Darwinian natural selection (I’ll come back to this idea later).

Whichever way you look at it, again there is probably a limit to how much insight you can glean from this type of book.
 
 
Other genres

Of course this phenomenon extends into many other areas of human activity. As a youth I can remember only too well poring over cricket manuals in an (ultimately fruitless) attempt to improve my batting or wicket-keeping. My father, at the age of 72, still does the same with golf manuals.

The endless array of cooking books also in the same category and where would we be without the panoply of self-help books such as The Seven Habits of Annoyingly Organised People? All of which goes to show that reliance on recipes for success is a deeply ingrained human trait.
 
 
Recipes for success in IT

Having established that people like turning to both “My top tips for [...]” and “Mr Brown’s Glittering Career” (available at all good booksellers) how does this aspect of human nature impinge on one of my main areas of endeavour, IT?

Well it has a major impact in my opinion. In fact it is difficult to think of an area of life more obsessed with frameworks, blue-prints, road-maps, procedures, best practices and methodologies (to say nothing of ontologies and taxonomies). All of these are intended to take the risk out of activities – well at least to provide the people following them with the ability to say “well I did what the methodology told me to do”. Of course IT projects and IT development are very complex things and standards of design, coding and behaviour of systems are of paramount importance; but it still seems that IT people have a more visceral relationship with the above-stated areas than would be dictated solely by ticking the necessary boxes.

Nevertheless, having been personally responsible for instigating a thoroughgoing process of standardisation and quality control in a software house (and thereby obtaining an ISO accreditation), it would be churlish of me to argue that that there is no benefit in rigorously applying methodologies in IT.

When it comes to some aspects of project management and to change management in particular, some of the scepticism that I exhibited about celebrity business books returns. It’s not so much that a methodology or even a list of items to tick is not valuable, but that it cannot be an end in itself. The important thing is the thinking that goes into drawing up what you need to do and how you are going to do it, not the method that you use to record these and monitor progress. Sometimes these crucial ingredients get lost. Indeed there does seem to be an entire class of people who focus just on managing lists, rather than the ideas behind them, or the people actually doing the work.
 
 
The benefits of a Darwinian approach

Charles Darwin

I raised the idea of a Darwinian approach to business strategy earlier in this article. There do seem to be some crossovers with how we observe businesses in operation. We are familiar with the image of companies competing with each other for limited resources (our wallets, mine being very limited at present). We understand the pressure that organisations are under to come up with better, cheaper, more functional and sexier products (that are now carbon neutral and ethically-sourced as well).

The language of business is suffused by jungle analogies. The adaptation of Tennyson’s “Nature, red in tooth and claw” to capitalism being just one of the most well-known examples. The companies that are best at this game survive and thrive, those that are not fail and are forgotten. Companies in more mature markets are even often referred to as dinosaurs or fossils. The idea of never-ending refinement and progress pushing on is an essential part of business.

However, perhaps this evolutionary approach, so evident at the macro-level can also work on a micro-scale. Maybe, rather than relying on the thoughts of Mr Brown or Professor Green, a better approach would be come up with some ideas of our own, test them, discard the bad ones and nurture the less bad ones. In time, with appropriate development and alteration, the less bad may become good and then even great (hang on, I seem to have found my way back to business books with that phrase!).

To me, such an approach is more likely to result in something novel and valuable. Following a recipe for success can only ever be as good as the recipe itself. Thinking for yourself can transcend these limitations and I would argue that the downside is no greater than attempting to ape someone else’s ideas. In both cases the worst that can happen is only extinction.
 
 
Disclaimer – sort of

Of course this article has a degree of self reference. Relying upon your own intellect (hopefully refined and improved by other people’s input) is of course another recipe for success. However I hope it is a less proscriptive one. I recommend giving it a try.
 

 
Continue reading about this area in: Synthesis.
 

tweet this Tweet this article on twitter.com
Bookmark this article with:
Technorati | del.icio.us | digg | Reddit | NewsVine

 

22 Comments | Biology, business, change management, management, project management, technology | Tagged: , , , , , , , , , | Permalink
Posted by Peter Thomas

Perseverance

31 March 2009

This blog is generally focused on topics in business, technology and change; often all three at the same time. However, from time to time, a personal post leaks in. This is one such post… or is it? Read to the end and then I will leave you to make up your own mind about this question.
 
 
Introduction

Over the years I have played many sports. For example, both cricket and rugby union consumed much of my youth. I have also recently got into mountain biking and really enjoy it. However, the activity that I am most engaged in currently is rock climbing, something that I alluded to at the beginning of a blog post yesterday. Rock climbing forms a very broad church and I have taken part in many aspects of it. However, for a number of reasons, I have gravitated to the sub-genre of bouldering over the last few years.

For the uninitiated, bouldering is climbing un-roped, often on actual boulders, but also on small outcrops and generally going no more than 5-6m (15-20 ft) off the ground. You carry around crash-pads (bouldering mats) with you to hopefully take the brunt of any falls. Indeed the idea with bouldering is to fall… to try again… and to fall again. In fact maybe Beckett had bouldering in mind when he wrote:


Ever tried. Ever failed. No matter. Try again. Fail again. Fail better.

The whole point is that, because bouldering is relatively (and I stress the word relatively) safe, you can try to make moves that are at the limit of your ability; moves that would not be terribly sensible to even contemplate making on a longer, higher, roped climb. In fact bouldering climbs are so difficult that they are generally described as “problems”; an apt name that also conveys the fact that sometimes you have to use extreme subtlety and finesse as well as brute strength to get up them.

People often literally spend years attempting to complete a problem, particularly if it represents a new level of climbing for them, or if no one else has climbed the line before. Because of this, such unclimbed lines are often called projects. It’s common to ask a fellow climber about how their current project is progressing. This choice of name perhaps begins to give some indication of why I am sharing my experiences in bouldering with you today.

Having said that most boulder problems are short, some hardy souls also embrace high-ball bouldering which, as the name suggests, takes you a lot further off the ground. The following video shows one of the world’s best climbers, Chris Sharma, bouldering in Bishop, California. It segues to him and another top climber, Ethan Pringle, attempting a high-ball problem that weighs in at around 11-12m (35-40 ft).

Note 1: Ethan issues an expletive under his breath towards the end of the clip. I might well have been tempted to do so myself in similar circumstances, but count yourselves warned.

Note 2: As will be apparent if you try to click on this video, it is sadly no longer available, probably to do with copyright issues. Instead I would recommend that you take a look at the bouldering section of Dead Point Magazine’s site.

Copyright notice. This piece is taken from the DVD King Lines which features Chris Sharma climbing all over the world. The copyright holder is BigUp Productions, a world-renowned and award-winning producer of climbing DVDs.
 
 
So what does this have to do with the price of fish?

Please substitute “the price of eggs” if you are in the US

Green Wall Essential (V2). The Buttermilks, Bishop, CA

Green Wall Essential (V2). The Buttermilks, Bishop, CA

I have recently taken to showing the above photograph at the mid-point of my public speaking about business intelligence and change management. Generally I have introduced it with the comment that I wanted to relieve the audience’s boredom by showing them some of my holiday snaps.

As in the above video, this climb is also in Bishop, California, a world-class bouldering venue. The problem is called Green Wall Essential and its grade of difficulty is V2. Without going into enormous detail about the different grading systems for boulder problems, I’ll simply say that V2 is towards the easier end of the spectrum; V15/16 is the hardest that people have climbed.

The reason that I share this image with business/technology audiences is related to the number of times that I tried (and failed) to climb it. Here are some statistics:

  • More than 80 attempts
  • On 4 different days
  • During 2 separate visits to Bishop
  • Spread over 8 months

I mentioned the term project above; Green Wall Essential became my project and my obsession. The above statistics represent more effort than I have ever put into climbing anything else. The quartz monzonite rock is hard and crystalline. It digs into your fingers and peels off your skin leaving the rock stained with your blood (you can see the tape holding the tips of my fingers together in the photograph). Your muscles and tendons ache from trying to push yourself just that little bit harder in order to attain success. You endlessly try different foot holds and body positions. You try to be slow and precise. When that doesn’t work you try to be aggressive and dynamic. When that doesn’t work… and so on and so on.

Now in order to put in that much effort over that much time, and to put up with that much pain and that much failure, you have to really want to do the problem. You have to be persistent, despite set backs. You have to continue to keep a positive mind-set, to believe that you can be successful, even when you have just failed for the 80th time.

In my experience, that is precisely the same mind-set that you need to be successful with major projects, particularly in the business of change management. Hopefully your fingers will bleed less, but it will not be easy. There will be set-backs. Progress may sometimes seem glacially slow, but if you persevere then the goal is worth it.

Sometimes we want to find a magic recipe for success, or – to mix the metaphor – a silver bullet. We want to discover a series of defined steps to take that, if repeated religiously, will guarantee that we get to the desired goal each and every time. That’s why articles entitled “The 5 ways to [...]” and “My top tips for [...]” are so well-read on the web. My take is that the secret ingredient may be very simple: plain, pig-headed perseverance.

By way of illustrating the benefits of this approach (and closing this article), here I am having achieved my own personal goal on Green Wall Essential… EVENTUALLY!!!

Me a very happy boulderer having completed my project.

Me a very happy boulderer having completed my project.

I wish you luck with your own projects, be these in business intelligence, other areas of IT, change management, or even bouldering. My own “Top tip” – if at first you don’t succeed, persevere.
 

 
If I have whetted anyone’s appetite about bouldering, you can take a look at my partner’s bouldering blog, which contains bouldering photos and videos, together with her musings on what motivates her to climb.
 

tweet this Tweet this article on twitter.com
Bookmark this article with:
Technorati | del.icio.us | digg | Reddit | NewsVine

 

15 Comments | business, business intelligence, change management, cultural transformation, project management, technology | Tagged: , , , | Permalink
Posted by Peter Thomas

Tactical Meandering

25 March 2009

Meanders
 

  tactics /táktiks/ n.pl. 2 a the plans and means adopted in carrying out a scheme or achieving some end. (O.E.D.)  
 
  meander /miándər/ n. 1 a a curve in a winding river etc. b a crooked or winding path or passage. (O.E.D.)  

 
I was reminded of the expression “tactical meandering”, which I used to use quite a bit, by a thread on the LinkedIn.com Business Intelligence Group forum. The title of this was Is BI recession-proof? (as always, you need to be a member of both LinkedIn.com and the group to view this).

The conversation on the thread turned to the fact that, in the current economic climate, there may be less focus on major, strategic BI initiatives and more on quick, tactical ones that address urgent business needs.

My take on this is that it is a perfectly respectable approach, indeed it is one that works pretty well in my experience regardless of the economic climate. There is however one proviso, that the short-term work is carried out with one eye on a vision of what the future BI landscape will look like. Of course this assumes that you have developed such a vision in the first place, but if you haven’t why are you talking about business intelligence when report writing is probably what you are engaged in (regardless of how fancy the tools may be that you are using to deliver these).

I talked about this specific area extensively in my earlier article, Holistic vs Incremental approaches to BI and also offered some more general thoughts in Vision vs Pragmatism. In keeping with the latter piece, and although the initial discussions referred to above related to BI, I wanted to use this article to expand the scope to some other sorts of IT projects (and maybe to some non-IT projects as well).

Some might argue (as people did on the LinkedIn.com thread) that all tactical work has to be 100% complementary to you strategic efforts. I would not be so absolute. To me you can wander quite some way from your central goals if it makes sense to do so in order to meet pressing business requirements in a timely and cost-effective manner. The issue is not so much how far you diverge from your established medium-term objectives, but that you always bear these in mind in your work. Doing something that is totally incompatible with your strategic work and even detracts from it may not be sensible (though it may sometimes still be necessary), but delivering value by being responsive to current priorities demonstrates your flexibility and business acumen; two characteristics that you probably want people to associate with you and your team.

Tactical meandering sums up the approach pretty well in my opinion. A river can wander a long way from a line drawn from its source to its mouth. Sometimes it can bend a long way back on itself in order to negotiate some obstacle. However, the ultimate destination is set and progress towards it continues, even if this is sometimes tortuous.

Oxbow Lake Formation

Oxbow Lake Formation

Expanding on the geographic analogy, sometimes meanders become so extreme that the river joins back to its main course, cutting off the loop and leaving an oxbow lake on one side. This is something that you will need to countenance in your projects. Sometimes an approach, or a technology, or a system was efficacious at a point in time but now needs to be dropped, allowing the project to move on. These eventualities are probably inevitable and the important thing is to flag up their likelihood in advance and to communicate clearly when they occur.

My experience is that, if you keep you strategic direction in mind, the sum of a number of tactical meanders can advance you quite some way towards your goals; importantly adding value at each step. The quickest path from A to B is not always a straight line.
 

tweet this Tweet this article on twitter.com
Bookmark this article with:
Technorati | del.icio.us | digg | Reddit | NewsVine

 

5 Comments | bi and the economic crisis, business, business intelligence, it business alignment, project management, technology | Tagged: , , , , , , , , | Permalink
Posted by Peter Thomas

A more appropriate metaphor for business intelligence projects

18 March 2009
A traditional metaphor for IT projects

A traditional metaphor for IT projects

IT people are familiar with a number of metaphors for their projects. The most typical relates to building; IT projects are compared to erecting a skyscraper. The IT literature is suffused with language derived from this metaphor. We build systems. We develop blueprints for them. We design architectures (two-for-one there). This analogy has some strength and there are indeed superficial similarities between the two areas. However, as with most metaphors, if over-extended their applicability often breaks down. I recall one CEO in particular who was obsessed by the “building team” moving on to the next “site”; regardless of the current one requiring further work and dedicated maintenance. One of his predecessors often referred to wanting a “diesel submarine” built, as opposed to a “nuclear one”. Before I fall into the same trap of over-exploiting the metaphor, let’s move hurriedly on.

As I mention above, aside from the occasional misapplication, the building analogy works reasonably well for many IT projects; does it also work for business intelligence? I think that there are some problems in applying the metaphor. Building tends to follow a waterfall project plan (as do many IT projects). Of course there may be some iterations, even many of them, but the idea is that the project is made up of discrete base-level tasks whose duration can be estimated with a degree of accuracy. Examples of such a task might include writing a functional specification, developing a specific code module, or performing integration testing between two sub-systems. Adding up all the base-level tasks and the rates of the people involved gets you a cost estimate. Working out the dependencies between the base-level tasks gets you an overall duration estimate.

The problem with BI projects is that some of the base-level tasks are a bit different. An example might be: develop an understanding of a legacy data table, how it relates to other legacy data sets and to more modern systems (this sits under area two of my model of BI development – see BI implementations are like icebergs). This is not an exercise that is very easy to estimate in advance. Indeed it may not be possible to produce an adequate estimate until a substantial amount of work has been done. Even at a late stage in the task, something may be discovered which expands the work required dramatically; surprises may lurk round every corner.

Why is this? Well with legacy data, the people who developed the system may have done so many years ago. Since then, they may have left the company or moved on to other areas, taking their knowledge with them. Their place may have been taken by successive tranches of new staff. Perhaps poor initial documentation meant that later workers did not fully understand the full nature of the system, but nevertheless did their best to build upon it. Perhaps the documentation was good at first, but has not been kept up-to-date and now describes a system that no longer exists.

By definition, legacy systems will have been around for some time and layers of changes will have accumulated on top of each other. Maybe, as a company has expanded, new data has been interfaced to the system from different business units and territories; perhaps each of these cases has its own dedicated interface code, each subtly different from those of other systems. Even where people exist in an organisation who preserve an “oral tradition” about the system, handed down to them over generations; these people may not appreciate how their data interacts with other data – even if the person who looks after another legacy system sits in the adjacent cubicle.

Waterfall Project Plan (intentionally blurred somewhat)

Although these challenges can also occur when trying to understand the data in more modern systems, they are particularly acute with older ones. For a start, the people who designed these systems are more likely to be around. Also legacy systems often sit at the centre of a Byzantine web of inter-connections, batch-processes, over-night jobs and the occasional more modern service. It can be a real mess and this is a situation with which any data analyst with a reasonable amount of experience will be very familiar.

The difficulty of estimating the duration of tasks such as properly analysing legacy data makes overall estimation of BI projects more of an art than a science. Of course techniques such as time-boxing tasks can be applied, but these are not always 100% appropriate. A 75% analysed data source (even assuming that the estimate that only 25% work is left is accurate) is not an analysed data source. Leaving dark corners of knowledge is likely to be reflected in BI cubes and reports that do not reconcile back to their sources. Probably the best way to deal with this problem is to be extremely open about the challenges up-front with executive sponsors and when submitting estimates. It helps to also stress the level of uncertainty in progress reports. The more honest you are initially, the better you will be able to explain any overruns and the more likely it is that you will be believed.

These types of issues mean that the – hopefully more orderly – process of constructing a building is not a fully accurate way to describe a BI project. That is unless the metaphor is extended to include an occurance that is all too common during construction in The City of London. Given the age of Londinium, whenever ground is broken on a new project, it is more likely than not that a mediaeval, Anglo-Saxon or Roman site is unearthed (often all three). These finds, while of enormous interest to academics, can result in projects being put on hold (sometimes for years) while the dig is fully assessed, artefacts are carefully removed and catalogued by experts and so on. Sometimes the remains are of such importance that a structure preserving and protecting them (and even allowing public viewing) has to be made part of the design of the foundations of new building. Many office blocks in The City have such viewing galleries in their basements. Such eventualities can create massive and unexpected overruns in central London building projects.

So this leads me to suggest a different metaphor for BI projects. Major elements of them are much more like archaeological digs than traditional building. The extent and importance of a dig is very difficult to ascertain before work starts and both may change during the course of a project. It is not atypical that an older site is discovered underneath an initial dig, doubling the amount of work required.

So, my belief is that BI professionals should not be likened to architects or structural engineers. Instead the epithet of archaeologist is much more appropriate. And if the fedora fits, wear it!

Fortune and glory in BI?

Fortune and glory in BI?


 

 
Apologies for the initial typo’ in the article heading, which persists in the perma-link. Of course I have the odd error scattered around the place, but in a heading! Maybe I need to employ a proof-reader.
 

tweet this Tweet this article on twitter.com
Bookmark this article with:
Technorati | del.icio.us | digg | Reddit | NewsVine

 

9 Comments | business, business intelligence, management information, project management, technology | Tagged: , , , , , , , , , , | Permalink
Posted by Peter Thomas

Is outsourcing business intelligence a good idea?

14 March 2009

Outsourcing
 
Introduction

The phrase IT outsourcing tends to provoke strong reactions. People either embrace it as a universal panacea capable of addressing any business problem, or recoil in horror at the very sound of it. Just for a change, I am somewhere in the middle; to me it is another tool at the disposal of businesses which can either be used wisely or poorly (much like IT itself you might say). As always the difference between the two extremes comes down to how well the project is led. Regardless of this, there are some benefits and some disbenefits associated with IT outsourcing and this article will explore the case for applying outsourcing to business intelligence.
 
 
Benefits of general IT outsourcing

Before I plunge into the world of BI, it is perhaps worth revisiting the general reasons for IT outsourcing, some of the most regularly quoted are as follows:

1. Reduction in costs

The provider of outsourcing (I’m just going to say “the provider” from now on to save typing) can carry out the same tasks at a cheaper cost to the client organisation (while still presumably turning a profit). There can be a number of bases for this; the one that generally comes to mind is wage arbitrage between different economies. However, it could also be that the provider has economies of scale; for instance, less people being required to run the consolidated data centres of several companies, than is required to run each separately. Also the provider may have staff who are more productive than at the client.

2. Ability to scale-up and scale down resource

The nature of business is such that sometimes all hands are required on the IT deck and at others there is spare capacity (this is something I address in my two articles on Problems associated with the IT cycle and Mitigating problems with the IT cycle). Now IT departments are normally quite good at finding (hopefully) useful things for people to do, but the issue remains. The promise of an outsourcing arrangement is that the tap of resource can be adjusted to meet demand without having to either fire and rehire staff, or rely on bringing in expensive contract resource. It is often hoped that this feature of outsourcing will also help to speed IT products to market.

3. Making IT provision a contractual relationship

An arrangement with a provider, depending on how the contract is drafted, can make the provision of IT services subject to penalties and claw-backs when service levels drop below those that have been agreed. While there are clearly some sanctions that can be applied to underperformance by internal IT departments, the financial benefit to the organisation is likely to be less (unless your CIO is a multi-billionaire of course). Companies are used to these contractual relationships, they are often the lifeblood of business, and it is a more familiar way of dealing with issues for them.

4. Access to skills

The nature of IT is that it does tend to evolve, sometimes quickly, sometimes slowly. For organisations this means keeping their IT people’s skills up to date though courses, or continually looking to bring people with new skills into an organisation (such people generally not being the cheapest). The idea with an outsourcing arrangement is that these issues become the headache of the provider, not the client. This area can be particularly pertinent when there is a technology change or a significant upgrade; these are times at which the prospect of being shot of IT worries may seem very attractive. The effort and cost of, as it were, upgrading your in-house IT staff may seem prohibitive in these circumstances.

5. Focus on core competencies

This has been a business mantra for many years, why should a company engaged in a wholly separate area of human endeavour want to become experts in building and supporting complex IT systems, when they can get a specialist organisation to do this for them? This moves towards the idea of a lean, or even virtual, organisation.

6. Failure of in-house IT

It is sad to have to add this item, but it is often the implicit (and sometimes even the explicit) driver of a desire to outsource. CEOs, COOs or CFOs may be so fed up with the performance of their IT people that they feel that surely someone else could not be worse. There is an adage that you don’t outsource a problem, but this is often honoured more in breech than observance.

I am sure that there are other advantages, claimed or real, for IT outsourcing, but the above list at least covers many of the normal arguments. At this stage a fully-balanced article would probably present arguments against IT outsourcing. However, my objective here is not to provide a critique of IT outsourcing in general, but to see whether the above benefits apply to business intelligence. Because of this, and I should stress purely for the purposes of this article, I am going to accept that all of the above gains are both realisable and desirable for general IT. There will therefore you will find no comments here about arbitrage (of its very nature) resulting in differentials of pricing closing over time.

The only benefit that I am going to rule out is the final one; addressing failed IT departments. Applying outsourcing in these cases is only likely to make things worse, and probably more expensive. Far better in my opinion to work out why IT is failing (most typically due to poor leadership it has to be said, see also my article: Some reasons why IT projects fail) and draw up plans for addressing this. If outsourcing is a strong element of this, then so be it, but thinking that it will resolve this type of issue is probably naive in most circumstances.

So, as always seems to be the case in these types of articles, we have five potential benefits against which to assess outsourcing BI. Before I look at each in turn, I wanted to make some general observations.
 
 
Things that are different about BI

The main fly in the ointment with respect to outsourcing business intelligence is the fact that good BI is reliant upon four things (see also BI implementations are like icebergs):

A. An in-depth understanding of business requirements, developed by close collaboration with a wide range of business managers. In particular, what is necessary is understanding what questions the business wants to ask and why (see Scaling-up Performance Management and Developing an international BI strategy)
B. An extensive appreciation of the data available in different business systems, its accuracy and how data in different places is related to each other.
C. Developing creative ways of transforming the available data into the required information and presenting this in an easy-to-understand and use manner.
D. A focus on change management that includes business-focussed marketing, training and follow-up to ensure that the work carried out in the first three areas results in actual business adoption and thereby the creation of value (see my collection of articles focussed on cultural transformation).

With the possible exception of item C., which is more technical, the above are best carried out in a symbiotic relationship with the business. Ideally what develops is a true IT / business hybrid team, where, though people have clear roles, the differences between these blur into each other. In turn, building thus type of team is predicated on developing strong relationships between the IT and business members and establishing high levels of trust and respect.

Also with item C., this is not precisely a stand-alone activity. It is one best carried out collaboratively by technically-aware business analysts and business-aware data analysts, ETL programmers and OLAP designers. Once again, distinctions blur somewhat during this work and a different type of hybrid team appears.

I have tried to illustrate the way that these tasks and teams should overlap in the following diagram.

bi-venn-w300

Clearly it is not impossible to achieve what I have described above in an outsourced environment, but it seems that it might be rather tougher to do this. One key point is that the type of skills that are necessary for success in BI are cross-over business / IT skills and these are generally less easy to buy off the shelf. Another is that the type of intellectual property that a BI team will build up (basically extensive knowledge of what makes the organisation tick) is precisely the sort that you would want to retain within an organisation.

I would suggest that if an organisation wants to outsource BI, then they should start that way. Once a BI team has gone through tasks A. to D. above then I can’t see how it would be cost-effective to subsequently outsource. The transfer of knowledge would take too long and be too costly.

To provide some context to this let me share some non-confidential details of a study I performed recently comparing the efficiency of a well-established BI team in a developed country with a less mature BI team in a lower-cost location. Rather than considering relative costs, I looked at relative productivity. A simple way to do this is to get quotes for carrying out a certain type of work from both teams (though I also applied some other techniques, which I won’t go into here). My main finding was that the ostensibly high cost team was more than twice as productive as the allegedly low-cost team. Just to be clear, if the “high-cost” team quoted $X for a piece of work, the “low-cost” team quoted over $2X,because they required much more resource and/or time to carry out the same work.

So, in what follows, I will assume that a decision is taken to outsource at the inception of a project. With this assumption and the previous background, let’s go back and look at the five benefits of outsourcing from the beginning.
 
 
Matching the benefits to BI

1. Reduction in costs

It will take external BI resource at least as long as internal BI resource to understand business requirements and available data. In fact internal staff probably have something of an advantage as they should already have an appreciation of what the organisation does and how IT systems support this. The external resource also has the disadvantage of it probably being more difficult for them to build business relationships, this can be exacerbated if there are personnel changes during the project; something that is perhaps more likely to happen with an external provider. If the provider is located in another country, then this raises even more challenges and inefficiencies (and leads to travel expense).

It will take an external BI team at least as long as an internal one to dig into the available data and how the various systems inter-relate. Again, having some familiarity with the existing systems’ landscape would be an advantage for an in-house team.

If an external team can get to the position where they understand the business needs and the available data really well in a reasonable period of time, then they could possibly have an advantage in the arena of transforming data into information. Something that may mitigate this however is that fact that most BI development is iterative and that a rolling set of prototypes needs to be reviewed closely with the business. This element introduces the same challenges as were apparent with defining business requirements above.

Similar arguments as were made about the business requirements phase apply to deployment and follow-up.

2. Ability to scale-up and scale down resource

While it may be possible (subject to contract) to scale-down resource with a provider (though perhaps tougher to get them back when you need them), scaling-up is just as hard as it is in-house at it means more staff at the provider going through the learning curve about the organisations business needs and data.

4. Access to skills

This is the crux of the matter. The skills in question are not Java programming (or even Cobol), they are business knowledge. ETL and OLAP skills are important, but only if they are applied by people who understand what they are doing and to what purpose. These skills are not just lying around in the market place; they are acquired through hard work and dedication.

3. Making IT provision a contractual relationship

Clearly this is a benefit of outsourcing. However, given that the contract is there for when things go awry, it is worth asking the question “are things more or less likely to go wrong with a provider?”

5. Focus on core competencies

While it is quite easy to argue that building e-commerce systems is not necessarily a core competency, good BI is about understanding what is necessary to best run the business. If that is not a core competency of any organisation, then I struggle to think of what would be.
 
 
Summary

My main argument is that BI is different to general IT projects (an assertion to which I will return in a forthcoming article). Having successfully run both, I am confident in this statement. I also think that you need different types of people with different skills in BI projects. These facts, plus the closeness of business / IT relationships which are necessary in the area mean that outsourcing is less likely to be effective. I am sure that an outsourcing arrangement can work well for some organisations in some circumstances, but I would argue strongly against it being best practise for most organisations most of the time.
 

 
After penning this article, a further problem with outsourcing business intelligence came to my mind; security. On part of most BI systems is a facility to analyse the organisation’s results. Ideally the BI system will have these figures in place very soon after the end of a financial closing. Such data is market sensitive and there may be concerns with trusting an external provider with both producing this and ensuring that it remains confidential until market announcements are made. I am not suggesting that providers are unethical, just that companies may not wish to take a chance in this area.
 
I should also credit a thread on the LinkedIn.com EPM – Business Intelligence group, which got me thinking about this area (as ever, you need to be a member of LinkedIn.com and the group to view this)
 

tweet this Tweet this article on twitter.com
Bookmark this article with:
Technorati | del.icio.us | digg | Reddit | NewsVine

 

23 Comments | business, business intelligence, enterprise performance management, it business alignment, management information, outsourcing, technology | Tagged: , , , , , , | Permalink
Posted by Peter Thomas

« Previous Entries
Next Entries »
Follow

Get every new post delivered to your Inbox.

Join 3,026 other followers