When I recently published the latest edition of The Data & Analytics Dictionary, I included an entry on Charts which briefly covered a number of the most frequently used ones. Given that entries in the Dictionary are relatively brief  and that its layout allows little room for illustrations, I decided to write an expanded version as an article. This will be published in the next couple of weeks.
One of the exhibits that I developed for this charts article was to illustrate the use of Bubble Charts. Given my childhood interest in Astronomy, I came up with the following – somewhat whimsical – exhibit:
Bubble Charts are used to plot three dimensions of data on a two dimensional graph. Here the horizontal axis is how far each of the gas and ice giants is from the Sun , the vertical axis is how many satellites each planet has  and the final dimension – indicated by the size of the “bubbles” – is the actual size of each planet .
Anyway, I thought it was a prettier illustration of the utility of Bubble Charts that the typical market size analysis they are often used to display.
However, while I was doing this, my older daughter wandered into my office and said “look at the picture I drew for you Daddy” . Coincidentally my muse had been her muse and the result is the Data Visualisation appearing at the top of this article. Equally coincidentally, my daughter had also encoded three dimensions of data in her drawing:
She also started off trying to capture relative size. After a great start with Mercury, Venus and Earth, she then ran into some Data Quality issues with the later planets (she is only four).
Here is an annotated version:
I think I’m at least OK at Data Visualisation, but my daughter’s drawing rather knocked mine into a cocked hat . And she included a comet, which makes any Data Visualisation better in my humble opinion; what Chart would not benefit from the inclusion of a comet?
For me at least that is.
Actually the measurement is the closest that each planet comes to the Sun, its perihelion.
This may seem a somewhat arbitrary thing to plot, but a) the exhibit is meant to be illustrative only and b) there does nevertheless seem to be a correlation of sorts; I’m sure there is some Physical reason for this, which I’ll have to look into sometime.
Bubble Charts typically offer the option to scale bubbles such that either their radius / diameter or their area is in proportion to the value to be displayed. I chose the equatorial radius as my metric.
It has to be said that this is not an atypical occurence.
For at least the four rocky planets, it might have taken a while to draw all 79 of Jupiter’s moons.
I often check my prose for phrases that may be part of British idiom but not used elsewhere. In doing this, I learnt today that “knock into a cocked hat” was originally an American phrase; it is first found in the 1830s.
After a hiatus of a few months, the latest version of the peterjamesthomas.com Data and Analytics Dictionary is now available. It includes 30 new definitions, some of which have been contributed by people like Tenny Thomas Soman, George Firican, Scott Taylor and and Taru Väre. Thanks to all of these for their help.
If you would like to contribute a definition, which will of course be acknowledged, you can use the comments section here, or the dedicated form, we look forward to hearing from you .
If you have found The Data & Analytics Dictionary helpful, we would love to learn more about this. Please post something in the comments section or contact us and we may even look to feature you in a future article.
The Data & Analytics Dictionary will continue to be expanded in coming months.
Please note that any submissions will be subject to editorial review and are not guaranteed to be accepted.
The above infographic is the work of Management Consultants Oxbow Partners and employs a novel taxonomy to categorise data teams. First up, I would of course agree with Oxbow Partners’ statement that:
Organisation of data teams is a critical component of a successful Data Strategy
Indeed I cover elements of this in two articles . So the structure of data organisations is a subject which, in my opinion, merits some consideration.
Oxbow Partners draw distinctions between organisations where the Data Team is separate from the broader business, ones where data capabilities are entirely federated with no discernible “centre” and hybrids between the two. The imaginative names for these are respectively The Burger, The Smoothie and The Jam Doughnut. In this article, I review Oxbow Partners’s model and offer some of my own observations.
The Burger – Centralised
Having historically recommended something along the lines of The Burger, not least when an organisation’s data capabilities are initially somewhere between non-existent and very immature, my views have changed over time, much as the characteristics of the data arena have also altered. I think that The Burger still has a role, in particular, in a first phase where data capabilities need to be constructed from scratch, but it has some weaknesses. These include:
The pace of change in organisations has increased in recent years. Also, many organisations have separate divisions or product lines and / or separate geographic territories. Change can be happening in sometimes radically different ways in each of these as market conditions may vary considerably between Division A’s operations in Switzerland and Division B’s operations in Miami. It is hard for a wholly centralised team to react with speed in such a scenario. Even if they are aware of the shifting needs, capacity may not be available to work on multiple areas in parallel.
Again in the above scenario, it is also hard for a central team to develop deep expertise in a range of diverse businesses spread across different locations (even if within just one country). A central team member who has to understand the needs of 12 different business units will necessarily be at a disadvantage when considering any single unit compared to a colleague who focuses on that unit and nothing else.
A further challenge presented here is maintaining the relationships with colleagues in different business units that are typically a prerequisite for – for example – driving adoption of new data capabilities.
The Smoothie – Federated
So – to address these shortcomings – maybe The Smoothie is a better organisational design. Well maybe, but also maybe not. Problems with these arrangements include:
Probably biggest of all, it is an extremely high-cost approach. The smearing out of work on data capabilities inevitably leads to duplication of effort with – for example – the same data sourced or combined by different people in parallel. The pace of change in organisations may have increased, but I know few that are happy to bake large costs into their structures as a way to cope with this.
The same duplication referred to above creates another problem, the way that data is processed can vary (maybe substantially) between different people and different teams. This leads to the nightmare scenario where people spend all their time arguing about whose figures are right, rather than focussing on what the figures say is happening in the business . Such arrangements can generate business risk as well. In particular, in highly regulated industries heterogeneous treatment of the same data tends to be frowned upon in external reviews.
The wholly federated approach also limits both opportunities for economies of scale and identification of areas where data capabilities can meet the needs of more than one business unit.
Finally, data resources who are fully embedded in different parts of a business may become isolated and may not benefit from the exchange of ideas that happens when other similar people are part of the immediate team.
So to summarise we have:
The Jam Doughnut – Hybrid
Which leaves us with The Jam Doughnut, in my opinion, this is a Goldilocks approach that captures as much as possible of the advantages of the other two set-ups, while mitigating their drawbacks. It is such an approach that tends to be my recommendation for most organisations nowadays. Let me spend a little more time describing its attributes.
I see the best way of implementing a Jam Doughnut approach is via a hub-and-spoke model. The hub is a central Data Team, the spokes are data-centric staff in different parts of the business (Divisions, Functions, Geographic Territories etc.).
It is important to stress that each spoke satellite is not a smaller copy of the central Data Team. Some roles will be more federated, some more centralised according to what makes sense. Let’s consider a few different roles to illustrate this:
Data Scientist – I would see a strong central group of these, developing methodologies and tools, but also that many business units would have their own dedicated people; “spoke”-based people could also develop new tools and new approaches, which could be brought into the “hub” for wider dissemination
Analytics Expert – Similar to the Data Scientists, centralised “hub” staff might work more on standards (e.g. for Data Visualisation), developing frameworks to be leveraged by others (e.g. a generic harness for dashboards that can be leveraged by “spoke” staff), or selecting tools and technologies; “spoke”-based staff would be more into the details of meeting specific business needs
Data Engineer – Some “spoke” people may be hybrid Data Scientists / Data Engineers and some larger “spoke” teams may have dedicated Data Engineers, but the needle moves more towards centralisation with this role
Data Architect – Probably wholly centralised, but some “spoke” staff may have an architecture string to their bow, which would of course be helpful
Data Governance Analyst – Also probably wholly centralised, this is not to downplay the need for people in the “spokes” to take accountability for Data Governance and Data Quality improvement, but these are likely to be part-time roles in the “spokes”, whereas the “hub” will need full-time Data Governance people
It is also important to stress that the various spokes should also be in contact with each other, swapping successful approaches, sharing ideas and so on. Indeed, you could almost see the spokes beginning to merge together somewhat to form a continuum around the Data Team. Maybe the merged spokes could form the “dough”, with the Data Team being the “jam” something like this:
I label these types of arrangements a Data Community and this is something that I have looked to establish and foster in a few recent assignments. Broadly a Data Community is something that all data-centric staff would feel part of; they are obviously part of their own segment of the organisation, but the Data Community is also part of their corporate identity. The Data Community facilities best practice approaches, sharing of ideas, helping with specific problems and general discourse between its members. I will be revisiting the concept of a Data Community in coming weeks. For now I would say that one thing that can help it to function as envisaged is sharing common tooling. Again this is a subject that I will return to shortly.
I’ll close by thanking Oxbow Partners for some good mental stimulation – I will look forward to their next data-centric publication.
It is peterjamesthomas.com’s policy to disclose any connections with organisations or individuals mentioned in articles.
Oxbow Partners are an advisory firm for the insurance industry covering Strategy, Digital and M&A. Oxbow Partners and peterjamesthomas.com Ltd. have a commercial association and peterjamesthomas.com Ltd. was also engaged by one of Oxbow Partners’ principals, Christopher Hess, when he was at a former organisation.
Though the author might have had a minor role in developing some elements of it as well.
As part of my consulting business, I end up thinking about Data Capability Frameworks quite a bit. Sometimes this is when I am assessing current Data Capabilities, sometimes it is when I am thinking about how to transition to future Data Capabilities. Regular readers will also recall my tripartite series on The Anatomy of a Data Function, which really focussed more on capabilities than purely organisation structure .
Detailed frameworks like the one contained in Anatomy are not appropriate for all audiences. Often I need to provide a more easily-absorbed view of what a Data Function is and what it does. The exhibit above is one that I have developed and refined over the last three or so years and which seems to have resonated with a number of clients. It has – I believe – the merit of simplicity. I have tried to distil things down to the essentials. Here I will aim to walk the reader through its contents, much of which I hope is actually self-explanatory.
The overall arrangement has been chosen intentionally, the top three areas are visible activities, the bottom three are more foundational areas , ones that are necessary for the top three boxes to be discharged well. I will start at the top left and work across and then down.
Collation of Data to provide Information
This area includes what is often described as “traditional” reporting , Dashboards and analysis facilities. The Information created here is invaluable for both determining what has happened and discerning trends / turning points. It is typically what is used to run an organisation on a day-to-day basis. Absence of such Information has been the cause of underperformance (or indeed major losses) in many an organisation, including a few that I have been brought in to help. The flip side is that making the necessary investments to provide even basic information has been at the heart of the successful business turnarounds that I have been involved in.
The bulk of Business Intelligence efforts would also fall into this area, but there is some overlap with the area I next describe as well.
Leverage of Data to generate Insight
In this second area we have disciplines such as Analytics and Data Science. The objective here is to use a variety of techniques to tease out findings from available data (both internal and external) that go beyond the explicit purpose for which it was captured. Thus data to do with bank transactions might be combined with publically available demographic and location data to build an attribute model for both existing and potential clients, which can in turn be used to make targeted offers or product suggestions to them on Digital platforms.
It is my experience that work in this area can have a massive and rapid commercial impact. There are few activities in an organisation where a week’s work can equate to a percentage point increase in profitability, but I have seen insight-focussed teams deliver just that type of ground-shifting result.
Control of Data to ensure it is Fit-for-Purpose
This refers to a wide range of activities from Data Governance to Data Management to Data Quality improvement and indeed related concepts such as Master Data Management. Here as well as the obvious policies, processes and procedures, together with help from tools and technology, we see the need for the human angle to be embraced via strong communications, education programmes and aligning personal incentives with desired data quality outcomes.
The primary purpose of this important work is to ensure that the information an organisation collates and the insight it generates are reliable. A helpful by-product of doing the right things in these areas is that the vast majority of what is required for regulatory compliance is achieved simply by doing things that add business value anyway.
Data Architecture / Infrastructure
Best practice has evolved in this area. When I first started focussing on the data arena, Data Warehouses were state of the art. More recently Big Data architectures, including things like Data Lakes, have appeared and – at least in some cases – begun to add significant value. However, I am on public record multiple times stating that technology choices are generally the least important in the journey towards becoming a data-centric organisation. This is not to say such choices are unimportant, but rather that other choices are more important, for example how best to engage your potential users and begin to build momentum .
Having said this, the model that seems to have emerged of late is somewhat different to the single version of the truth aspired to for many years by organisations. Instead best practice now encompasses two repositories: the first Operational, the second Analytical. At a high-level, arrangements would be something like this:
The Operational Repository would contain a subset of corporate data. It would be highly controlled, highly reconciled and used to support both regular reporting and a large chunk of dashboard content. It would be designed to also feed data to other areas, notably Finance systems. This would be complemented by the Analytical Repository, into which most corporate data (augmented by external data) would be poured. This would be accessed by a smaller number of highly skilled staff, Data Scientists and Analytics experts, who would use it to build models, produce one off analyses and to support areas such as Data Visualisation and Machine Learning.
It is not atypical for Operational Repositories to be SQL-based and Analytical Repsoitories to be Big Data-based, but you could use SQL for both or indeed Big Data for both according to the circumstances of an organisation and its technical expertise.
Data Operating Model / Organisation Design
Here I will direct readers to my (soon to be updated) earlier work on The Anatomy of a Data Function. However, it is worth mentioning a couple of additional points. First an Operating Model for data must encompass the whole organisation, not just the Data Function. Such a model should cover how data is captured, sourced and used across all departments.
Second I think that the concept of a Data Community is important here, a web of like-minded Data Scientists and Analytics people, sitting in various business areas and support functions, but linked to the central hub of the Data Function by common tooling, shared data sets (ideally Curated) and aligned methodologies. Such a virtual data team is of course predicated on an organisation hiring collaborative people who want to be part of and contribute to the Data Community, but those are the types of people that organisations should be hiring anyway .
Our final area is that of Data Strategy, something I have written about extensively in these pages  and a major part of the work that I do for organisations.
It is an oft-repeated truism that a Data Strategy must reflect an overarching Business Strategy. While this is clearly the case, often things are less straightforward. For example, the Business Strategy may be in flux; this is particularly the case where a turn-around effort is required. Also, how the organisation uses data for competitive advantage may itself become a central pillar of its overall Business Strategy. Either way, rather than waiting for a Business Strategy to be finalised, there are a number of things that will need to be part of any Data Strategy: the establishment of a Data Function; a focus on making data fit-for-purpose to better support both information and insight; creation of consistent and business-focussed reporting and analysis; and the introduction or augmentation of Data Science capabilities. Many of these activities can help to shape a Business Strategy based on facts, not gut feel.
More broadly, any Data Strategy will include: a description of where the organisation is now (threats and opportunities); a vision for commercially advantageous future data capabilities; and a path for moving between the current and the future states. Rather than being PowerPoint-ware, such a strategy needs to be communicated assiduously and in a variety of ways so that it can be both widely understood and form a guide for data-centric activities across the organisation.
As per my other articles, the data capabilities that a modern organisation needs are broader and more detailed than those I have presented here. However, I have found this simple approach a useful place to start. It covers all the basic areas and provides a scaffold off of which more detailed capabilities may be hung.
The framework has been informed by what I have seen and done in a wide range of organisations, but of course it is not necessarily the final word. As always I would be interested in any general feedback and in any suggestions for improvement.
In passing, Anatomy is due for its second refresh, which will put greater emphasis on Data Science and its role as an indispensable part of a modern Data Function. Watch this space.
Though nowadays you hear “traditional” Analytics and “traditional” Big Data as well (on the latter see Sic Transit Gloria Magnorum Datorum), no doubt “traditional” Machine Learning will be with us at some point, if it isn’t here already.
This is the second year in which I have produced a retrospective of my blogging activity. As in 2017, I have failed miserably in my original objective of posting this early in January. Despite starting to write this piece on 18th December 2018, I have somehow sneaked into the second quarter before getting round to completing it. Maybe I will do better with 2019’s highlights!
Anyway, 2018 was a record-breaking year for peterjamesthomas.com. The site saw more traffic than in any other year since its inception; indeed hits were over a third higher than in any previous year. This increase was driven in part by the launch of my new Maths & Science section, articles from which claimed no fewer than 6 slots in the 2018 top 10 articles, when measured by hits . Overall the total number of articles and new pages I published exceeded 2017’s figures to claim the second spot behind 2009; our first year in business.
As with every year, some of my work was viewed by tens of thousands of people, while other pieces received less attention. This is my selection of the articles that I enjoyed writing most, which does not always overlap with the most popular ones. Given the advent of the Maths & Science section, there are now seven categories into which I have split articles. These are as follows:
In each category, I will pick out one or two pieces which I feel are both representative of my overall content and worth a read. I would be more than happy to receive any feedback on my selections, or suggestions for different choices.
Two Forbes articles argue different perspectives about the role of Chief Data Officer. The first (by Lauren deLisa Coleman) stresses its importance, the second (by Randy Bean) highlights some of the challenges that CDOs face.
Many companies want to become data driven, but getting started on the journey towards this goal can be tough. This article offers a framework for building momentum in the early stages of a Data Programme.
The number π is surrounded by a fog of misunderstanding and even mysticism. This article seeks to address some common misconceptions about π, to show that in many ways it is just like any other number, but also to demonstrate some of its less common properties.
One of the more recent chapters in my forthcoming book on Group Theory and Particle Physics. This focuses on the seminal contributions of Mathematician Emmy Noether to the fundamentals of Physics and the connection between Symmetry and Conservation Laws.
The peterjamesthomas.com Data and Analytics Dictionary is an active document and I will continue to issue revised versions of it periodically. Here are 20 new definitions, including the first from other contributors (thanks Tenny!):
People are now also welcome to contribute their own definitions. You can use the comments section here, or the dedicated form. Submissions will be subject to editorial review and are not guaranteed to be accepted.
Today I am talking to Christopher Bannocks, who is Group Chief Data Officer at ING. ING is a leading global financial institution, headquartered in the Netherlands. As stressed in other recent In-depth interviews , data is a critical asset in banking and related activities, so Christopher’s role is a pivotal one. I’m very glad that he has been able to find time in his busy calendar to speak to us.
Hello Christopher, can you start by providing readers with a flavour of your career to date and perhaps also explain why you came to focus on the data arena.
Sure, it’s probably right to say I didn’t start out here, data was not my original choice, and for anyone of a similar age to me, data wasn’t a choice, when I started out, in that respect it’s a “new segment”. I started out on a management development programme in a retail bank in the UK, after which I moved to be an operations manager in investment banking. As part of that time in my career, post Euro migration and Y2K (yes I am genuinely that old, I also remember Vinyl records and Betamax video!)  I was asked to help solve the data problem. What I recognised very quickly was this was an area with under-investment, that was totally central the focus of that time – STP (Straight Through Processing). Equally it provided me with much broader perspectives, connections to all parts of the organisation that I previously didn’t have and it was at that point, some 20 years ago, that I decided this was the thing for me! I have since run and driven transformation in Reference Data, Master Data, KYC , Customer Data, Data Warehousing and more recently Data Lakes and Analytics, constantly building experience and capability in the Data Governance, Quality and data services domains, both inside banks, as a consultant and as a vendor.
I am trying to get a picture of the role and responsibilities of the typical CDO (not that there appears to be such a thing), so would you mind touching on the span of your work at ING? I know you have a strong background in Enterprise Data Management, how does the CDO role differ from this area?
I guess that depends on how you determine the scope of Enterprise Data Management. However, in reality, the CDO role encompasses Enterprise Data Management, although generally speaking the EDM role includes responsibility for the day to day operations of the collection processes, which in my current role I don’t have. I have accountability for the governance and quality through those processes and for making the data available for downstream consumers, like Analytics, Risk, Finance and HR.
My role encompasses being the business driver for the data platform that we are rolling out across the organisation and its success in terms of the data going onto the platform and the curation of that data in a governed state, depending on the consumer requirements.
My role today boils down to 4 key objectives – data availability, data transparency, data quality and data control.
I know that ING consists of many operating areas and has adopted a federated structure with respect to data matters. What are the strengths of this approach and how does it work on a day-to-day basis?
This approach ensures that the CDO role (I have a number of CDOs functionally reporting to me) remains close to the business and the local entity it supports, it ensures that my management team is directly connected to the needs of the business locally, and that the local businesses have a direct connection to the global strategy. What I would say is that there is no “one size fits all” approach to the CDO organisation model. It depends on the company culture and structure and it needs to fit with the stated objectives of the role as designed.
On a day to day basis, we are aligned with the business units and the functional units so we have CDOs in all of these areas. Additionally I have a direct set of reports who drive the standard solutions around tooling, governance, quality, data protection, Data Ethics, Metadata and data glossary and models.
Helping organisations become “data-centric” is a key part of what you do. I often use this phrase myself; but was recently challenged to elucidate its meaning. What does a “data-centric” organisation look like to you? What sort of value does data-centricity release in your experience?
Data centric is a cultural shift, in the structures of the past where we have technology people and process, we now have data that touches all three. You know if you have reached the right place when data becomes part of the decision making process across the organisation, when decisions are only made when data is presented to support it and this is of the requisite quality. This doesn’t mean all decisions require data, some decisions don’t have data and that’s where leaderships decisions can be made, but for those decisions that have good data to support them, these can be made easily and at a lower level in the organisation. Hence becoming data centric supports an agile organisation and servant / leadership principles, utilising data makes decisions faster and outcomes better.
I am on record multiple times  stating that technology choices are much less important than other aspects of data work. However, it is hard to ignore the impact that Big Data and related technologies have had. A few years into the cycle of Big Data adoption, do you see the tools and approaches yielding the expected benefits? Should I revisit my technology-agnostic stance?
I have also been on record multiple times saying that every data problem is a people problem in disguise. I still hold that this is true today although potentially this is changing. The problems of the past and still to this day originate with poor data stewardship, I saw it happening in front of my eyes last week in Heathrow when I purchased something in a well known electronics store. Because I have an overseas postcode the guy at the checkout put dummy data into all the fields to get through the process quickly and not impact my customer experience, I desperately wanted to stop him but also wanted to catch my plane. This is where the process efficiency impacts good data collection. If the software that supports the process isn’t flexible, the issue won’t be fixed without technology intervention, this is often true in data quality problems which have knock on effects to customers, which at the end of the day are why we are all here. This is a people problem (because who is taking responsibility here for fixing it, or educating that guy at the checkout) AND it’s a technology problem, caused by inflexible or badly implemented systems.
However, in the future, with more focus on customer driven checkout, digital channels and better customer experience, better interface driven data controls and robotics and AI, it may become further nuanced. People are still involved, communication remains critical but we cannot ignore technology in the digital age. For a long time, data groups have struggled with getting access to good tools and technology, now this technology domain is growing daily, and the tools are improving all the time. What we can do now with data at a significantly lower cost than ever before is amazing, and continues to improve all the time. Hence ignoring technology can be costly when extending capabilities to your stakeholders and could be a serious mistake, however focusing only on technology and ignoring people, process, communication etc is also a serious mistake. Data Leaders have to be multi-disciplinary today, and be able to keep up with the pace of change.
I have heard you talk about “data platforms”, what do you mean by this and how do these contrast with another perennial theme, that of data democratisation? How does a “data platform” relate to – say – Data Science teams?
Data democratisation is enabled by the data platform. The data platform is the technology enablement of the four pillars I mentioned before, availability, transparency, quality and control. The platform is a collection of technologies that standardise the approach and access to well governed data across the organisation. Data Democratisation is simply making data available and abstracting away from siloed storage mechanisms, but the platform wraps the implementation of quality, controls and structure to the way that happens. Data Science teams then get the data they need, including data curation services to find the data they need quickly, for governed and structured data, Data Science teams can utilise the glossary to identify what they need and understand the level of quality based on consumer views, they also have access to metadata in standard forms. This empowers the analytics capability to move faster, spend less time on data discovery and curation, structure and quality and more time on building analytics.
I mentioned the federated CDO team at ING above and assume this is reflected in the rest of the organisation structure. ING also has customers in 40 countries and I know first-hand that a global footprint adds complexity. What are the challenges in being a CDO in such an environment? Does this put a higher premium on influencing skills for example?
I am not sure it puts a higher premium on influencing skills, these have a high premium in any CDO role, even if you don’t have a federated structure, the reality is if you are in a data role you have more stakeholders than anyone else in the company, so influencing skills remain premium.
A global footprint means complexity for sure, it means differences in a world where you are trying to standardise and it means you have to be tuned in to cultural differences and boundaries. It also means a great deal of variety, opportunities to learn new cultures and approaches, it means you have to listen and understand and flex your style and it means pragmatism plays an important part in your decision making process.
At ING we have an amazing team of people who collaborate in a way I have never experienced before, supported by a strong attachment and commitment to the success of the business and our customers. This makes dealing with the complexity a team effort, with great energy and a fantastic working environment. In an organisation without the drive and passion we have here it would present challenges, with the support of the board and being a core part of the overall strategy, it ensures broad alignment to the goal, which makes the challenge easier for the organisation to solve, not easy, but easier and more fun.
Building on the last point, every CDO I have interviewed has stressed the importance of relationships; something that chimes with my own experience. How do you go about building strong relationships and maintaining them when inevitable differences of opinion or clashes in interests arise?
I touched on this a little earlier. Pragmatism over purism. I see purist everywhere in data, with views that are so rigid that the execution of them is doomed because purism doesn’t build relationships. Relationships are built based on what you bring and give up, on what you can give, not on what you can get. I try every day to achieve this, but I am human too, so I don’t always get it right, I hope I get it right more than I get it wrong and where I get it wrong I hope I can be forgiven for my intention is pure. We owe it to our customers to work together for their benefit, where we have differences the customer outcomes should drive our decisions, in that we have a common goal. Disagreements can be helped and supported by identifying a common goal, this starts to align people behind a common outcome. Individual interests can be put aside in preference of the customer interest.
I know that you are very interested in data ethics and feel that this is an important area for CDOs to consider. Can you tell the readers a bit more about data ethics and why they should be central to an organisation’s approach to data?
In an increasingly digital world, the use of data is becoming widespread and the pace at which it is used is increasing daily, our compute power grows exponentially as does the availability of data. Given this, we need an ethical framework to help us make good decisions with our customers and stakeholders in mind. How do you ensure that decisions in your organisation about how you use data are ethical? What are ethical decisions in your organisation and what are the guiding principles? If this isn’t clear and communicated to help all staff make good decisions, or have good discussions there is a real danger that decisions may not be properly socialised before all angles are considered.
Just meeting the bar of privacy regulation may not be enough, you can still meet that bar and do things that your customers may disagree with of find “creepy” so the correct thought needs to be applied and the organisation engaged to ensure the correct conversations take place, and there is a place to go to discuss ethics.
I am not saying that there is a silver bullet to solve this problem, but the conversation and the ability to have the conversation in a structured way helps the organisation understand its approach and make good decisions in this respect. That’s why CDOs should consider this an important part of the role and a critical engagement with users of data across the organisation.
Finally, I have worked for businesses with a presence in the Netherlands on a number of occasions. As a Brit living abroad, how have you found Amsterdam. What – if any – adaptations have you had to make to your style to thrive in a somewhat different culture?
Having lived in India, I thought my move to the Netherlands could only be easy. I arrived thinking that a 45 minute flight could not possibly provide as many challenges as an 11 hour flight, especially from a cultural perspective. Of course I was wrong because any move to a different culture provides challenges you could never have expected and it’s the small adjustments that take you by surprise the most. It’s always a hugely enjoyable learning experience though. London is a more top down culture whereas in the Netherlands it’s a much flatter approach, my experience here is positive although it does require an adjustment. I work in Amsterdam but live in a small village, chosen deliberately to integrate faster. It’s harder, more of a challenge but helps you understand the culture as you make friends with local people and get closer to the culture. My wife and I have never been a fan of the expat scene, we prefer to integrate, however more difficult this feels at first, it’s worth it in the long run. I must admit though that I haven’t conquered the language yet, it’s a real work in progress!
Christopher, I really enjoyed our chat, which I believe will also be of great interest to readers. Thank you.
Disclosure: At the time of publication, neither peterjamesthomas.com Ltd. nor any of its Directors had any shared commercial interests with Christopher Bannocks, ING or any entities associated with either of these.
If you are a Chief Data Officer, a Chief Analytics Officer, a Director of Data, or hold some other “Top Data Job” and would like to share your thoughts with the readers of this site in an interview like this one, please get in contact.
The interviews that I conduct with leaders in their fields as part of my “In-depth” series have hopefully brought a new and interesting aspect to this site. However, often the boot is on the other foot and I am the person being interviewed about my experience and expertise in the data field and related matters . Maybe interviewing other people helps me when I am in turn interviewed, maybe it’s the other way round. Whatever the case, I enjoyed recording the two conversations appearing below (thanks to the interviewers in both cases) and hope that the content is of interest to readers.
In both instances a link to the site originally publishing the interview is followed by a locally hosted version of the audio track and then a download option. I’d encourage readers to explore the other excellent interviews contained on both sites.
Enterprise Management 360° Podcast – 31st July 2018
If you would like to interview me for your site or periodical, of if you are just interested in further exploring some of the themes I discuss in these two interviews, then please feel free to get in contact.
Work by the inimitable Randall Munroe, author of long-running web-comic, xkcd.com, has been featured (with permission) multiple times on these pages . The above image got me thinking that I had not penned a data visualisation article since the series starting with Hurricanes and Data Visualisation: Part I – Rainbow’s Gravity nearly a year ago. Randall’s perspective led me to consider that staple of PowerPoint presentations, the humble and much-maligned Pie Chart.
While the history is not certain, most authorities credit the pioneer of graphical statistics, William Playfair, with creating this icon, which appeared in his Statistical Breviary, first published in 1801 . Later Florence Nightingale (a statistician in case you were unaware) popularised Pie Charts. Indeed a Pie Chart variant (called a Polar Chart) that Nightingale compiled appears at the beginning of my article Data Visualisation – A Scientific Treatment.
I can’t imagine any reader has managed to avoid seeing a Pie Chart before reading this article. But, just in case, here is one (Since writing Rainbow’s Gravity – see above for a link – I have tried to avoid a rainbow palette in visualisations, hence the monochromatic exhibit):
The above image is a representation of the following dataset:
The Pie Chart consists of a circle divided in to five sectors, each is labelled A through E. The basic idea is of course that the amount of the circle taken up by each sector is proportional to the count of items associated with each category, A through E. What is meant by the innocent “amount of the circle” here? The easiest way to look at this is that going all the way round a circle consumes 360°. If we consider our data set, the total count is 18,000, which will equate to 360°. The count for A is 4,500 and we need to consider what fraction of 18,000 this represents and then apply this to 360°:
So A must take up 90°, or equivalently one quarter of the total circle. Similarly for B:
Or one sixth of the circle.
If we take this approach then – of course – the sum of all of the sectors must equal the whole circle and neither more nor less than this (pace Randall). In our example:
So far, so simple. Now let’s consider a second data-set as follows:
What does its Pie Chart look like? Well it’s actually rather familiar, it looks like this:
This observation stresses something important about Pie Charts. They show how a number of categories contribute to a whole figure, but they only show relative figures (percentages of the whole if you like) and not the absolute figures. The totals in our two data-sets differ by a factor of over 2,100 times, but their Pie Charts are identical. We will come back to this point again later on.
Pie Charts have somewhat fallen into disrepute over the years. Some of this is to do with their ubiquity, but there is also at least one more substantial criticism. This is that the human eye is bad at comparing angles, particularly if they are not aligned to some reference point, e.g. a vertical. To see this consider the two Pie Charts below (please note that these represent a different data set from above – for starters, there are only four categories plotted as opposed to five earlier on):
The details of the underlying numbers don’t actually matter that much, but let’s say that the left-hand Pie Chart represents annual sales in 2016, broken down by four product lines. The right-hand chart has the same breakdown, but for 2017. This provides some context to our discussions.
Suppose what is of interest is how the sales for each product line in the 2016 chart compare to their counterparts in the right-hand one; e.g. A and A’, B and B’ and so on. Well for the As, we have the helpful fact that they both start from a vertical line and then swing down and round, initially rightwards. This can be used to gauge that A’ is a bit bigger than A. What about B and B’? Well they start in different places and end in different places, looking carefully, we can see that B’ is bigger than B. C and C’ are pretty easy, C is a lot bigger. Then we come to D and D’, I find this one a bit tricky, but we can eventually hazard a guess that they are pretty much the same.
So we can compare Pie Charts and talk about how sales change between two years, what’s the problem? The issue is that it takes some time and effort to reach even these basic conclusions. How about instead of working out which is bigger, A or A’, I ask the reader to guess by what percentage A’ is bigger. This is not trivial to do based on just the charts.
If we really want to look at year-on-year growth, we would prefer that the answer leaps off the page; after all, isn’t that the whole point of visualisations rather than tables of numbers? What if we focus on just the right-hand diagram? Can you say with certainty which is bigger, A or C, B or D? You can work to an answer, but it takes longer than should really be the case for a graphical exhibit.
There is a further point to be made here and it relates to what we said Pie Charts show earlier in this piece. What we have in our two Pie Charts above is the make-up of a whole number (in the example we have been working through, this is total annual sales) by categories (product lines). These are percentages and what we have been doing above is to compare the fact that A made up 30% of the total sales in 2016 and 33% in 2017. What we cannot say based on just the above exhibits is how actual sales changed. The total sales may have gone up or down, the Pie Chat does not tell us this, it just deals in how the make-up of total sales has shifted.
Some people try to address this shortcoming, which can result in exhibits such as:
Here some attempt has been made to show the growth in the absolute value of sales year on year. The left-hand Pie Chart is smaller and so we assume that annual sales have increased between 2016 and 2017. The most logical thing to do would be to have the change in total area of the two Pie Charts to be in proportion to the change in sales between the two years (in this case – based on the underlying data – 2017 sales are 69% bigger than 2016 sales). However, such an approach, while adding information, makes the task of comparing sectors from year to year even harder.
The general argument is that Nested Bar Charts are better for the type of scenario I have presented and the types of questions I asked above. Looking at the same annual sales data this way we could generate the following graph:
While Bar Charts are often used to show absolute values, what we have above is the same “percentage of the whole” data that was shown in the Pie Charts. We have already covered the relative / absolute issue inherent in Pie Charts, from now on, each new chart will be like a Pie Chart inasmuch as it will contain relative (percentage of the whole) data, not absolute. Indeed you could think about generating the bar graph above by moving the Pie Chart sectors around and squishing them into new shapes, while preserving their area.
The Bar Chart makes the yearly comparisons a breeze and it is also pretty easy to take a stab at percentage differences. For example B’ looks about a fifth bigger than B (it’s actually 17.5% bigger) . However, what I think gets lost here is a sense of the make-up of the elements of the two sets. We can see that A is the biggest value in the first year and A’ in the second, but it is harder to gauge what percentage of the overall both A and A’ represent.
To do this better, we could move to a Stacked Bar Chart as follows (again with the same sales data):
Once more, we are dealing with how proportions have changed – to put it simply the height of both “skyscrapers” is the same. If we instead shifted to absolute values, then our exhibit might look more like:
The observant reader will note that I have also added dashed lines linking the same category for each year. These help to show growth. Regardless of what angle to the horizontal the lower line for a category makes, if it and the upper category line diverge (as for B and B’), then the category is growing; if they converge (as for C and C’), the category is shrinking . Parallel lines indicate a steady state. Using this approach, we can get a better sense of the relative size of categories in the two years.
However, here – despite the dashed lines – we lose at least some of of the year-on-year comparative power of the Nested Bar Chart above. In turn the Nested Bar Chart loses some of the attributes of the original Pie Chart. In truth, there is no single chart which fits all purposes. Trying to find one is analogous to trying to find a planar projection of a sphere that preserves angles, distances and areas .
Rather than finding the Philosopher’s Stone  of an all-purpose chart, the challenge for those engaged in data visualisation is to anticipate the central purpose of an exhibit and to choose a chart type that best resonates with this. Sometimes, the Pie Chart can be just what is required, as I found myself in my article, A Tale of Two [Brexit] Data Visualisations, which closed with the following image:
Or, to put it another way:
You may very well be well bred
Chart aesthetics filling your head
But there’s always some special case, time or place
To replace perfect taste
Never cry ’bout a Chart of Pie
You can still do fine with a Chart of Pie
People may well laugh at this humble graph
But it can be just the thing you need to help the staff
Never cry ’bout a Chart of Pie
Though without due care things can go awry
Bars are fine, Columns shine
Lines are ace, Radars race
Boxes fly, but never cry about a Chart of Pie
With apologies to the Disney Corporation!
It was pointed out to me by Adam Carless that I had omitted the following thing of beauty from my Pie Chart menagerie. How could I have forgotten?
It is claimed that some Theoretical Physicists (and most Higher Dimensional Geometers) can visualise in four dimensions. Perhaps this facility would be of some use in discerning meaning from the above exhibit.
No this article has not escaped from my Maths & Science section, it is actually about data matters. But first of all, channeling Jennifer Aniston , “here comes the Science bit – concentrate”.
The Theory of Common Descent holds that any two organisms, extant or extinct, will have a common ancestor if you roll the clock back far enough. For example, each of fish, amphibians, reptiles and mammals had a common ancestor over 500 million years ago. As shown below, the current organism which is most like this common ancestor is the Lancelet .
To bring things closer to home, each of the Great Apes (Orangutans, Gorillas, Chimpanzees, Bonobos and Humans) had a common ancestor around 13 million years ago.
So far so simple. As one would expect, animals sharing a recent common ancestor would share many attributes with both it and each other.
Convergent Evolution refers to something else. It describes where two organisms independently evolve very similar attributes that were not features of their most recent common ancestor. Thus these features are not inherited, instead evolutionary pressure has led to the same attributes developing twice. An example is probably simpler to understand.
The image at the start of this article is of an Ichthyosaur (top) and Dolphin. It is striking how similar their body shapes are. They also share other characteristics such as live birth of young, tail first. The last Ichthyosaur died around 66 million years ago alongside many other archosaurs, notably the Dinosaurs . Dolphins are happily still with us, but the first toothed whale (not a Dolphin, but probably an ancestor of them) appeared around 30 million years ago. The ancestors of the modern Bottlenose Dolphins appeared a mere 5 million years ago. Thus there is tremendous gap of time between the last Ichthyosaur and the proto-Dolphins. Ichthyosaurs are reptiles, they were covered in small scales . Dolphins are mammals and covered in skin not massively different to our own. The most recent common ancestor of Ichthyosaurs and Dolphins probably lived around quarter of a billion years ago and looked like neither of them. So the shape and other attributes shared by Ichthyosaurs and Dolphins do not come from a common ancestor, they have developed independently (and millions of years apart) as adaptations to similar lifestyles as marine hunters. This is the essence of Convergent Evolution.
That was the Science, here comes the Technology…
A Brief Hydrology of Data Lakes
From 2000 to 2015, I had some success  with designing and implementing Data Warehouse architectures much like the following:
Overlapping with the above, from around 2012, I began to get involved in also designing and implementing Big Data Architectures; initially for narrow purposes and later Data Lakes spanning entire enterprises. Of course some architectures featured both paradigms as well.
One of the early promises of a Data Lake approach was that – once all relevant data had been ingested – this would be directly leveraged by Data Scientists to derive insight.
Over time, it became clear that it would be useful to also have some merged / conformed and cleansed data structures in the Data Lake. Once the output of Data Science began to be used to support business decisions, a need arose to consider how it could be audited and both data privacy and information security considerations also came to the fore.
Next, rather than just being the province of Data Scientists, there were moves to use Data Lakes to support general Data Discovery and even business Reporting and Analytics as well. This required additional investments in metadata.
The types of issues with Data Lake adoption that I highlighted in Draining the Swamp earlier this year also led to the advent of techniques such as Data Curation . In parallel, concerns about expensive Data Science resource spending 80% of their time in Data Wrangling  led to the creation of a new role, that of Data Engineer. These people take on much of the heavy lifting of consolidating, fixing and enriching datasets, allowing the Data Scientists to focus on Statistical Analysis, Data Mining and Machine Learning.
All of which leads to a modified Big Data / Data Lake architecture, embodying people and processes as well as technology and looking something like the exhibit above.
This is where the observant reader will see the concept of Convergent Evolution playing out in the data arena as well as the Natural World.
Lest it be thought that I am saying that Data Warehouses belong to a bygone era, it is probably worth noting that the archosaurs, Ichthyosaurs included, dominated the Earth for orders of magnitude longer that the mammals and were only dethroned by an asymmetric external shock, not any flaw their own finely honed characteristics.
Also, to be crystal clear, much as while there are similarities between Ichthyosaurs and Dolphins there are also clear differences, the same applies to Data Warehouse and Data Lake architectures. When you get into the details, differences between Data Lakes and Data Warehouses do emerge; there are capabilities that each has that are not features of the other. What is undoubtedly true however is that the same procedural and operational considerations that played a part in making some Warehouses seem unwieldy and unresponsive are also beginning to have the same impact on Data Lakes.
If you are in the business of turning raw data into actionable information, then there are inevitably considerations that will apply to any technological solution. The key lesson is that shape of your architecture is going to be pretty similar, regardless of the technical underpinnings.
The two of us are constantly mistaken for one another.
To be clear the common ancestor was not a Lancelet, rather Lancelets sit on the branch closest to this common ancestor.
Ichthyosaurs are not Dinosaurs, but a different branch of ancient reptiles.
This is actually a matter of debate in paleontological circles, but recent evidence suggests small scales.