The revised and expanded Data and Analytics Dictionary

The Data and Analytics Dictionary

Since its launch in August of this year, the peterjamesthomas.com Data and Analytics Dictionary has received a welcome amount of attention with various people on different social media platforms praising its usefulness, particularly as an introduction to the area. A number of people have made helpful suggestions for new entries or improvements to existing ones. I have also been rounding out the content with some more terms relating to each of Data Governance, Big Data and Data Warehousing. As a result, The Dictionary now has over 80 main entries (not including ones that simply refer the reader to another entry, such as Linear Regression, which redirects to Model).

The most recently added entries are as follows:

  1. Anomaly Detection
  2. Behavioural Analytics
  3. Complex Event Processing
  4. Data Discovery
  5. Data Ingestion
  6. Data Integration
  7. Data Migration
  8. Data Modelling
  9. Data Privacy
  10. Data Repository
  11. Data Virtualisation
  12. Deep Learning
  13. Flink
  14. Hive
  15. Information Security
  16. Metadata
  17. Multidimensional Approach
  18. Natural Language Processing (NLP)
  19. On-line Transaction Processing
  20. Operational Data Store (ODS)
  21. Pig
  22. Table
  23. Sentiment Analysis
  24. Text Analytics
  25. View

It is my intention to continue to revise this resource. Adding some more detail about Machine Learning and related areas is probably the next focus.

As ever, ideas for what to include next would be more than welcome (any suggestions used will also be acknowledged).
 


 

From: peterjamesthomas.com, home of The Data and Analytics Dictionary

 

The peterjamesthomas.com Data and Analytics Dictionary

The Data and Analytics Dictionary

I find myself frequently being asked questions around terminology in Data and Analytics and so thought that I would try to define some of the more commonly used phrases and words. My first attempt to do this can be viewed in a new page added to this site (this also appears in the site menu):

The Data and Analytics Dictionary

I plan to keep this up-to-date as the field continues to evolve.

I hope that my efforts to explain some concepts in my main area of specialism are both of interest and utility to readers. Any suggestions for new entries or comments on existing ones are more than welcome.
 

 

An in-depth Interview with Allan Engelhardt about Analytics

Cybaea

Allan Engelhardt

PJT Today’s interview is with Allan Engelhardt, co-founder and principal of insights and analytics consultancy Cybaea. Allan and I know each other from when we both worked at Bupa. I was interested to understand the directions that he has been pursuing in recent years.
PJT Allan, we know each other well, but could you provide a pen picture of your career to date and the types of work that you have been engaged in?
AE I started out in experimental physics working on (very) big data from CERN, the large research lab near Geneva, and worked there after getting my degree. Then, like many other physicists, I was recruited into financial services, in my case to do risk management. From there to a consultancy helping business make use of bleeding edge technology and then on to CRM and customer loyalty. This last move was important for me, allowing me to move beyond the technology to be as much about commercial business strategy and operations.

In 2002 a couple of us left the consultancy to help customers move beyond transactional infrastructure, which is really what ‘CRM’ was about at the time, to create high value solution on top, and to create the organizational and commercial ownership of the customer needed to consistently drive value from data, inventing the concept of Customer Value Management which is now universally implemented by telcos across the world and increasingly adopted by other industries.

PJT There is no ISO definition of either insight or analytics. As an expert in these fields, can I ask you to offer your take on the meaning of these terms?
AE To me analytics is about finding meaning from information and data, while insights is about understanding the business opportunities in that meaning. But different people use the terms differently.
PJT I must give you an opportunity to both explain what Cybaea does and how the name came about.
AE At Cybaea we are passionate about value creation and commercial results. We have been called ‘Management consultants with a black belt in data’ and we help organizations identify and act upon data driven opportunities in the areas of:

Cybaea offering

  1. Customer Value Management (CVM), including acquisition, churn, cross-sell, segmentation, and more, across online and offline channels and industries, both B2C and B2B.
  2. Customer Experience and Advocacy, including Net Promoter System and Net Promoter Economics, customer journey optimization, and customer experience.
  3. Innovation and Growth, including data-driven product and proposition development, data monetisation, and distribution and sales strategy.

For our customers, CVM projects typically deliver additional 5% EBITDA growth annually, which you can measure very robustly because much of it is direct marketing. Experience and Advocacy projects typically deliver in the region of 20% EBITDA improvement to our clients, but it is harder to measure accurately because you must go above the line for this level of impact. And for Innovation and Growth, the sky is the limit.

As for the name, we founded the company in 2002 and wanted a short domain name that was a real word. It turned out to be difficult to find an available, short ‘.com’ at the peak of the dot-bomb era! We settled on ‘cybaea’ which my Latin dictionary translated as ‘trading vessel’; historically, it was a type of merchant ship of Greek origin, common in the Mediterranean, which Cicero describes as “most beautiful and richly adorned”. We always say we want to change the name, but it never happens; I guess if it was good enough for Cicero, then it is good enough for us.

PJT While at Bupa you led work that was very beneficial to the organisation and which is now the subject of a public Cybaea case study, can you tell readers a bit more about this?
AE Certainly, and the case study is available at for anyone who wants to read more.

This was working with Bupa Global; a Bupa business unit that primarily provides international private medical insurance for 2 million customers living in over 195 different countries. Towards the end of 2013, Bupa Global set out on a strategic journey to deliver sustained growth. A key element of this was the design and launch of a completely new set of products and propositions, replacing the existing portfolio, with the objective of attracting and servicing new customer segments, complying with changing regulation and meeting customer expectations.

The strategic driver was therefore very much in the Innovation and Growth space we outlined above, and I joined Bupa’s global Leadership Team to create and lead the commercial insights function that would support this change with deep understanding of the target customers and the markets in which they live. Additionally, Bupa had very high ambitions for its Net Promoter programme (Experience and Advocacy) where we delivered the most advanced installation across the global business, and for Customer Value Management we demonstrated nearly 2% reduction in the Claims line (EBITDA) from one single project.

For the new propositions, we initially interviewed over 3,000 individuals on five continents to understand value- and purchase drivers, researched 195 markets to size demand across all customer segments, and further deep-dived into key markets to understand the competitors with products, features, and prices, as well as the regulatory environment, and distribution options. This was supported by a very practical Customer Lifetime Value model, which we developed.

Suffice to say that in two years we had designed and implemented a completely new set of propositions and taken them live in more than twenty priority markets where they replaced the old products.

The strategic and commercial results were clearly delivered. But when I asked our CEO what he thought was the main contribution of the team and the new insights function, he focused on trust: “Every major strategic decision we made was backed by robust data and deep insights in which the executive team had full confidence.”

In a period of change, trust is perhaps the key currency. Trust that you are doing the right things for the right reasons, and the ability to explain why that is. This is key to get everybody behind the changes that need to happen. This is what the scientific method applied to data, analytics, and insights can bring to a commercial organization, and it inspires me to continue what we are doing.

PJT We have both been engaged in what is now generally called the Data arena for many years, some aspects of the technology employed have changed a lot during this time. What do you think modern technology enables today that was harder to achieve in the past and are there any areas where things are much the same as they were a decade or more ago?
AE Ever since the launch of the Amazon EC2 cloud computing service in late 2006 [1], data storage and processing infrastructure has been easily and cheaply available to everybody for most practical workloads. So, for ten years you have not had any excuse for not getting your data in order and doing serious analysis.

The main trend that excites me now is the breakthroughs happening in Deep Learning and Natural Language Processing, expanding the impact of data into completely new areas. This is great for consumers and for those companies that are at the leading edge of analytics and insights. For other organizations, however, who are struggling to deliver value from data, it means that the gap between where they are versus best practice is widening exponentially, which is a big worry.

PJT Taking technology to one side, what do you think are the main factors in successfully generating insight and developing analytical capabilities that are tightly coupled with value generation?
AE Two things are always at the forefront of my mind. The first is kind of obvious, namely to start with the business value you are trying to create and work backwards from that. Too often we see people start with the data (‘I got to clean all the data in my warehouse first!’), the technology (‘We need some Big Data infrastructure!’), or the analytics (‘We need a predictive churn model!’). That is cart before the horse. Not that these things are not important; rather, that there are almost certainly a lot of opportunities you could execute right now to generate real and measurable business value and drive a faster return on your investments.

The second is to not under-estimate the business change that is needed to exploit the insights. Analytical leaders have appetite for change and they plan and resource accordingly. Data and models are only part of the project to deliver the value and they are really clear on this.

PJT Looking at the other side of the coin, what at the pitfalls to look out for and do you have any recommendations for avoiding them?
AE The flip-side of the two points previously mentioned are obvious pitfalls: not starting from the business change and value you are trying to create. And it is not easy: great data scientists are not always great commercially-minded business people and so you need the right kind of skills to bridge that gap. McKinsey talks of ‘business translators who combine data savvy with industry and functional expertise’, which is a helpful summary [2]. Less helpfully they also note that these people are nearly impossible to find, so you may need to find or grow them internally.

Which gets to a second pitfall. When thinking about generating value from data, many want to do it all themselves. And I understand why: after all, data may well be a strategic asset for your organization.

But when you recruit, you should be clear in your mind if you are recruiting to deliver the change of creating the first models and changed business processes, or if you are recruiting to sustain the change by keeping the models current and incrementally improving the insights and processes. These two outcomes require people with quite different skills and vastly different temperaments.

We call them Explorers versus Farmers.

For the first, you want commercially-focused business people who can drive change in the organization; who can make things work quickly, whether that is data, analytics, or business processes, to demonstrate value; and who are supremely comfortable with uncertainties and unknowns.

For the second, you want people who are technically skilled to deliver and maintain the optimal stable platform and who love doing incremental improvements to technology, data, and business processes.

Explorers versus Farmers. Call them what you will, but note that they are different.

PJT Many companies are struggling with how to build analytical teams. Do they grow their own talent, do they hire numerate graduates or post graduates, do they seek to employ highly skilled and experienced individuals, do they form partnerships with external parties, or is a mixture of all of these approaches sensible? What approaches do you see at Cybaea clients adopting?
AE We are mostly seeing one of two approaches: one is to do nothing and soldier on as always relying on traditional business intelligence while the other is to hire usually highly technical people to build an internal team. Neither is optimal in getting to the value.

The do-nothing approach can make sense. Not, however, when it is adopted because management fears change (change will happen, regardless) or because they feel they don’t understand data (everybody understands data if it is communicated well). Those companies are just leaving money on the table: every organization have quick wins that can deliver value in weeks.

But it may be that you have no capacity for change and have made the informed decision that data and analytics must wait, reflecting the commercial reality. The key here is ‘informed’ and the follow-on question is if there are other ways that the company can realise some of the value from data right now.

The second approach at least recognises the value potential of data and aims to move the organization towards realising that value. But it is back to those ‘business translator’ roles we discussed before and making sure you have them, as well as making sure the business is aligned around the change that will be needed. Making money from data is a business function, not a technical one, and the function that drives the change must sit within the commercial business, not in IT or some other department that is still an arms-length support function.

We see the best organizations, the analytical leaders, employing flexible approaches. They focus on the outcomes and they have a sense of urgency driven from the top. They make it work.

PJT I know that a concept you are very interested in is Analytics as a Service (AaaS). Can you tell readers some more about what this means and also the work that Cybaea is doing in this area?
AE There is a war on analytical talent and a ‘winner takes it all’ dynamic is emerging with medium-sized enterprises especially losing out. Good people want to work with good people which generates a strong network effect giving advantage to large organizations with larger analytical teams and more variety of applications. Leading firms have depth of analytical talent and can recruit, trial, and filter more candidates, leaving them with the best talent.

Our analytics-as-a-service offering is for organizations of any size who want to realise value from data and insights right now, but who are not yet ready to build their own internal teams. We partner with the commercial teams to be their (commercial) insights function and deliver not just reports but real business change. Customers can pay monthly, pay for results, or we can do a build-operate-transfer model.

One of our first projects was with a small telco. They were too small to maintain a strong analytical team in-house, purely because of scale. We set up a monthly workshop with the commercial Marketing team. We analysed their data offline and used the time for a structured conversation about the new campaigns and the new changes to the web site they should implement this month. We would point them to our reports and dashboards which had models, graphs, t-tests, and p-values in abundance, but would focus the conversation on moving the business forward.

The following month we would repeat and identify new campaigns and new changes. After six months, they had more than 20 highly effective and precisely targeted campaigns running, and we handed over the maintenance (‘farming’) of the models to their IT teams. It is a model that works well across industries.

PJT Do you have a view on how the insights and analytics field is likely to change in coming years? Are there any emerging areas which you think readers should keep an eye on?
AE Many people are focused on the data explosion that is often called the ‘Internet of Things’ but more broadly means that more data gets generated and we consume more data for our analytics. I do think this opens tremendous opportunities for many businesses and technically I am excited to get back to processing live event streams as they happen.

But practically, we are seeing more success from deep learning. We have found that once an organization successfully implements one solution, whether artificial intelligence or complex natural language processing, then they want more. It is that powerful and that transformational, and breakthroughs in these fields are further expanding the impact into completely new area. My advice is that most organizations should at least trial what these approaches can do for them, and we have set up a sister-organization to develop and deliver solutions here.

PJT What are your plans for Cybaea in coming months?
AE I have two main priorities. First, I have our long-standing partner from India in London for a couple of months to figure out how we scale in the UK. This is for the analytics as a service but also for fast projects to deliver insights or analytical tools and applications.

Second, I am looking to identify the right partners and associates for Cybaea here in the UK to allow us to grow the business. We have great assets in our methodologies, clients, and people, and a tremendous opportunity for delivering commercial value from data, so I am very excited for the future.

PJT Allan, I would like to thank you for sharing with us the benefit of your experience and expertise in data matters, both of which have been very illuminating.

Allan Engelhardt can be reached at Allan.Engelhardt@cybaea.net. Cybaea’s website is www.cybaea.net and they have social media presence on LinkedIn and Google+.
 


 
Disclosure: Neither peterjamesthomas.com Ltd. nor any of its directors have any direct financial interest in either Cybaea or any of the other organisations mentioned in this article.
 
 
Notes

 
[1]
 
https://aws.amazon.com/about-aws/whats-new/2006/08/24/announcing-amazon-elastic-compute-cloud-amazon-ec2—beta/
 
[2]
 
McKinsey report The Age of Analytics, dated December 2016, http://www.mckinsey.com/business-functions/mckinsey-analytics/our-insights/the-age-of-analytics-competing-in-a-data-driven-world


 

 

25 Indispensable Business Terms

Business School Reunion - BTVS-style. Back row: Tara (Amber Benson), Willow (Alyson Hannigan), Xander (Nicholas Brendon), Anya (Emma Caulfield), Wesley (Alexis Denisof), Cordelia (Charisma Carpenter), Oz (Seth Green), Angel (David Boreanaz), Spike (James Masters), Dawn (Michelle Trachtenberg) & Joyce (Kristine Sutherland). Front row: Creator/Writer/Director (Joss Whedon) & Buffy (Sarah Michelle Gellar). Absent Giles (Anthony Stewart Head)

The first episode of Buffy the Vampire Slayer aired on 10th March 1997. To commemorate its 20th anniversary – and of course to celebrate 1st April 2017 – peterjamesthomas.com is pleased to present this comprehensive – and wholly indispensable – illustrated list of business terminology, Slayer-style:
 
 
1. Stakeholder [1]

Stakeholder

“The Freshman” – Season 4, Episode 1.

2. Stakeholder Management [2]

Stakeholder Management

Promotional shot.

3. Stakeholder Engagement

Stakeholder Engagement

“Something Blue” – Season 4, Episode 9.

4. Cross-functional

Cross functional

“Consequences” – Season 3, Episode 15.

5. Cross-functional Team

Cross-functional Team

“The Wish” – Season 3, Episode 9.

6. Institutionalise

Institutionalise

“Normal Again” – Season 6, Episode 17.

7. Presentation Deck

Presentation Deck

“First Date” – Season 7, Episode 14.

8. Playing Hardball

Playing Hardball

“The Gift” – Season 5, Episode 22.

9. Key Player

Key Player

“Buffy vs Dracula” – Season 5, Episode 1.

10. Platform

Platform

“The Gift” – Season 5, Episode 22.

11. Think Outside the Box

Think outside the box

“Bargaining, Part 1” – Season 6, Episode 1.

12. Vision Statement

Vision statement

“Restless” – Season 4, Episode 22.

13. Machine Learning

Machine Learning

“Intervention” – Season 5, Episode 18.

14. Doing the Heavy Lifting

Doing the Heavy Lifting

“The Gift” – Season 5, Episode 22.

15. Town Hall Meeting

Town Hall Meeting

“Band Candy” – Season 3, Episode 6.

16. Empower

Empower

“Chosen” – Season 7, Episode 22.

17. Drinking the Kool Aid

Drinking the Kool Aid

“Destiny” – Angel, Season 5, Episode 8 [3].

18. Bleeding Edge

Bleeding Edge

“This Year’s Girl” – Season 4, Episode 15.

19. Best Practice

Best Practice

“Once More with Feeling” – Season 6, Episode 7.

20. Animal Spirits

Animal Spirits

“The Pack” – Season 1, Episode 6.

21. Change Management

Change Management

“Checkpoint” – Season 5, Episode 12.

22. Face Time

Face Time

“Who Are You?” – Season 4, Episode 16.

23. Scalable

Scalable

“The Gift” – Season 5, Episode 22.

24. Take Offline

Take offline

“I, Robot… You, Jane” – Season 1, Episode 8.

25. Wow Factor

Wow Factor

“Tabula Rasa” – Season 6, Episode 8.

 
With inspiration drawn from Business Buzzword Bingo and Forbes Most Annoying Business Jargon, as well as the author’s own experience. With love (and of course enormous apologies) to everyone who worked at, or for, Mutant Enemy.

Except for the banner photo, which is © Entertainment Weekly / Time Inc., all images most likely © Warner Brothers Entertainment, but sourced from all over.
 


 
Notes

 
[1]
 
I must admit that it was the word “stakeholder” which first planted the seed that grew into this article. Whenever I hear the rather vapid term in a business context, an image of Sarah Michelle Gellar wafts unbidden into my consciousness; which is no doubt what the person using the term intended all along of course.
 
[2]
 
This image already crept into the notes section of Themes from a Chief Data Officer Forum – the 180 day perspective.
 
[3]
 
OK, I may have cheated here, though it could be argued that Angel also “started” on 10th March 1997 rather than 5th October 1999. I’d welcome any suggestions for a more BTVS-themed image.

 

 

An in-depth Interview with Blockchain luminary Gary Nuttall

Distlytics Ltd.

Gary Nuttall

PJT Today I am speaking with Gary Nuttall, Managing Director of Distlytics Ltd. Distlytics is a ground-breaking consultancy implementing Distributed Ledger
Technology (aka “Blockchain”) within Financial Services. I know Gary from when we both worked in the Business Intelligence space and wanted to get his perspective on both the current state of this area and its future possibilities.

Gary, would you mind telling readers a little about yourself and your journey prior to Distlytics?

GPN My career spans multiple industries – I began in retail 25+ years ago working on Management Information, Decision Support and Expert Systems. Moved on to Pharmaceuticals (implementing a greenfield BI/DW platform), then the wine industry, commodities trading and then commercial insurance.

The strand that has remained central to my career has been business intelligence and analytics – using data to improve business processes and enabling better informed decision making.

PJT Can you expand on what Distlytics does and how it does it?
 
GPN I’ve been interested in Blockchain for several years and saw that many developments are implemented using a shared Distributed Ledger architecture. This made me think, how do you perform analytics on a distributed ledger? Lots of people are exploring using a blockchain-based solution to improve business performance or create new products and services but nobody was thinking about the analytics layer. So: Distributed Ledger + Analytics = Distlytics.

The original focus therefore was to identify how an analytics layer could be introduced, what the best architecture would be, were there suitable existing technologies or would something unique need to be developed etc. All very interesting stuff and right up my street as it involved new and emerging technology, business improvement and BI.

It quickly became clear however that I was ahead of the game and that many organisations were only just at the “what’s blockchain and how can we use it?” phase. Therefore I started with providing consultancy and running Proof of Concepts to explore the technology and its suitability in the London Commercial Insurance Market.

PJT Given your work over the last few years, you would seem to have an ideal perspective from which to comment on the future adoption of blockchain technology and how it is evolving. Even in early 2017, it seems that the word blockchain is likely to cause furrowed brows and a change of topic to something less challenging than public keys, distributed databases and hash values.

Before talking to us about how you have seen blockchain used, can you try to provide a brief business-centric overview of what a blockchain is and how it works?

GPN I have a fairly simple, albeit technical, definition of what it is: It’s a Write-only, cryptographically secured, distributed, programmable, database. It sounds quite boring and nothing massively new, in terms of what it is; and that criticism is fair. The magic happens when you look not at what it is but rather on what it enables “out of the box”. Each of the individual features can be delivered by existing technologies but it would be very difficult (and expensive) to provide them all as standard.

Part of what I do as a consultant is to get people excited about what the technology enables, not how it does it. Fundamentally, blockchain is a protocol (it describes how something is done). In the 1980s there was a protocol introduced called TCP with the IP layer added to give us TCP/IP. Most non-technical people haven’t heard of TCP/IP and techie people would struggle to explain how it works. However, everyone agrees that “The Internet” has changed the World and that’s what TCP/IP is used for. I usually try to move conversations on from what it is/how it works to what it enables and what benefits it brings (more of which later).

PJT It seems that sometimes while people might grasp the essentials of blockchain there is then a “so what?” moment.

In your experience, are there any key facts, examples of usage, or even just anecdotes that help the penny to drop?

GPN The major consultancies appear to be in the game of “who can publish the biggest number” currently when talking about potential savings that organisations could achieve. PwC suggested Reinsurers could save $5-$10Bn in reduced expenses. Accenture indicated banks could save $8-$12Bn p.a. and McKinsey suggested up to $110Bn saving in Financial Services over three years.

So, some attention grabbing numbers about the potential. Nobody has, Bitcoin aside, actually implemented anything at scale and delivered the magnitude of benefits proposed.

PJT Can you tell me a bit more about how Distlytics recommends blockchain is used and the advantages that this confers?
 
GPN The starting point is to emphasise that blockchain is not necessarily the right answer. There are occasions when a traditional database is better, or a change to a business process would deliver more benefit at lower risk. With each use case it’s important to examine key requirements and to map them against what blockchain can offer.

At one extreme, if the problem is around maintaining a central master list that teams within a single organisation can use then something like Master Data Services (or for a small firm, a central spreadsheet!) suffices. At the other end of the spectrum, if there’s a need for multiple parties to access a common data repository that is write-only (thereby providing good in-built audit), is distributed (and so cyber resilient), cryptographically stored (so cyber resistant), and would benefit from multiple parties having access to “their” data then a blockchain begins to look like a potential solution.

PJT What about the broader market, what compelling blockchain stories have emerged over the last 12 months?
 
GPN I’ve been attending blockchain conferences for several years (and have presented at quite a few). The technology seems to be following a familiar path – we’re not actually moving forwards that much. There’s increasing investment, lots of hype but very few examples of anybody moving beyond Proof of Concept. Whilst 2016 was the year of PoC’s, it’s hoped that 2017 will be the year of pilots and 2018 the year of productionisation.

There are however two notable exceptions. First, Everledger is putting diamonds on a blockchain to prove origination and authenticity as so reduce fraud and blood diamond trade. Second, Estonia, as a nation, is the first to put many of its digital services onto blockchain and it already provides a global digital identity scheme. There’s likely to be a big change in 2017 as projects move out of stealth mode (several financial trading exchanges are moving from PoC to pilot). Watch this space.

PJT Are there any newer features or capabilities, either existing or pending, which you see as providing greater utility based on blockchain foundations?
 
GPN The Bitcoin protocol code was released in 2009 (at that time the word blockchain wasn’t even used). Since then we’ve seen the launch of numerous other protocols (e.g. Ripple, Ethereum and Eris/Monax). There have been major developments in scalability (e.g. BigChainDB) and performance (SETL.IO) and the market is rapidly evolving with new protocols being developed and existing ones maturing. It is however still a fairly new technology.
 
PJT What challenges do you see to the wider adoption of blockchain, be these regulatory, legal, technical, to do with privacy (on both sides of the argument) or relating to people’s understanding of the technology and what it can do?
 
GPN I’m going to stick my neck out and say that regulators shouldn’t try to regulate blockchain. Just like how they don’t regulate relational databases or spreadsheets. What they do regulate is organisations and processes and how the technology is applied. So, regulators will take a great interest in cryptocurrencies and how smart contracts are used to auto-execute trades, etc. They’re unlikely to attempt to regulate the protocols themselves.

Privacy is going to be interesting with the upcoming EU GDPR. I’m writing a paper on this as there’s a range of issues to address (how do you delete a record from a write-only database?) As mentioned earlier, people don’t really need to understand how the technology works, they need to understand how it can be used. Likewise the regulators will need to develop their awareness. Kudos to the FCA who are running their Innovation Sandbox with 14 companies pushing the regulatory boundaries – of which eight of the projects are blockchain-based.

PJT Concerns around blockchain have sometimes centred on possible issues such as the enforceability of smart contracts in law, the potential for specific miners to monopolise a market (negating the benefits of the distributed model) and the time taken to mine new chain links not being compatible with some types of transactions. What do you say to allay the fears of people who raise these issues?
 
GPN It’s now generally accepted that “Smart Contracts” was a poor choice of words as they’re neither smart not contracts. It does sound more innovative that “computer programmes” which is what they actually are! There are several approaches to resolve the enforceability issue – The simplest is to retain existing, legally recognised contracts and reference them to the smart contract. The smart contract performs the execution of the contract and the original provides the legal wrapper. Another is to write smart contracts in a form that is legally binding and accepted. This is an opportunity that lawyers who want to write software are getting excited about (it gives them work!)

Miner monopolisation really depends upon which protocol is being used and whether the implementation is a public or private blockchain. Bitcoin is currently dominated by the computing power controlled by Chinese miners – and that’s making some people (and Governments) nervous.

Time taken to mine is, again, protocol dependent. Bitcoin takes around 10 minutes for a transaction to be confirmed, Ethereum is at least twice as fast, so neither are suited for low latency volume processing. By comparison, Symbiont claims 87,000 transactions per second and that should suffice for most processing requirements. Ethereum is working on its Raiden Network to offer very high speed processing too.

For those who have concerns about legality and performance I’d say that this is a rapidly evolving technology. In just a few years we’ve seen orders of magnitude performance improvements. On the legal side, there remains uncertainty and that’s why I suspect there’ll be a blend of old and new ways run in parallel until new precedents have been established in law.

PJT Many people will associate blockchain with its most well known implementation, Bitcoin. Not everyone will immediately have a positive reaction to this association. How do you address concerns that might arise relating to everything from the volatility of Bitcoins, to news stories of Bitcoin theft, to the perennial linkage with money laundering and the dark economy?

Given this less than propitious environment, how do you provide an alternative and more positive message?

GPN Every Bitcoin “failure” has been at the application, not protocol, level. Bitcoin thefts (e.g. Mt Gox) are rather like a bank being broken into – it doesn’t reflect a failure of pounds (or dollars). Most blockchain protocols are open source and the associated cryptocurrencies are worth millions. This means that they’re highly susceptible to attack and that’s actually what strengthens the protocol. Human beings develop immunity to disease through exposure to it and it’s the same with Bitcoin.

As for Bitcoin’s associated with Dark Web and drug dealing and other illicit activities, it’s true! However, dollars and pounds have been used to fund illicit activities too but that doesn’t erode people’s confidence in traditional currencies. Interestingly, crime prevention agencies actually like cryptocurrencies such as Bitcoin! Don’t tell the criminals but cryptocurrencies such as Bitcoin aren’t completely anonymous. In fact a complete ledger of every transaction is publicly available. This means the lineage of payments is easily traceable. As recent convictions of Danish drug dealers has proved!

PJT What industry sectors or business processes do you think blockchain is likely to have the greatest impact on in coming months? Are there any areas which you see as crying out for such a new approach?
 
GPN Imagine we’re in the 1980’s and the same question was asked a lot about the Internet! Financial Services have spent over $1Bn figuring out how to use the new technology but there’s a lot of work going on in Public Sector and third sector (i.e. charities) too. Having spoken with a wide range of people doing fascinating work, I reckon that it’ll end up transforming areas that we’re not even thinking about – there’s so much going on with copyright, media protection, music distribution, charity donations, etc. that we’ll probably be surprised by how widespread its adoption is.
PJT We have both spent considerable time working in insurance and reinsurance. An increasing number of commentators, including yourself, have suggested that blockchain can play a pivotal role in driving change and reducing costs in this sector. There has even been talk of alternative models, such as peer-to-peer insurance and of the possible disintermediation of brokers. What are your views on the potential of blockchain in Insurance?
 
GPN One of the powerful features of blockchain is that it provides an opportunity to fundamentally disrupt any business model that requires an intermediary. The insurance value chain currently involves multiple intermediaries, each of whom believe they add value (rather than cost). The work I’ve been doing in the London Commercial Insurance Market is paving the way for radical new approaches and could see the value chain between a client with an insurable risk and a capital provider underwriting the risk being dramatically shortened.

Brokers talk about removing the need for Underwriters and Underwriters question the need for Brokers in a future model. Blockchain enables both approaches and we could see radical changes in operating models as well as new products and services being developed. It’s possible that insurance may be augmented (or replaced) by alternative financial instruments that can be developed using blockchain. As an example, think of an insurance contract sliced into individual components that can then be traded in a marketplace – a new derivatives marketplace. Other financial sectors have Swaps, Options, etc. and this could extend to insurance as alternative mechanisms for risk mitigation.

PJT Are there any other aspects of blockchain technology, current or future, which you feel it would be helpful for readers to know about?
 
GPN Things are changing so fast, it’s likely that if I were to recommend something then it would be out of date before the interview is published. I would suggest that readers try to keep a watching brief on some of the bigger things – protocols such as Bitcoin, Ethereum, Ripple and Monax as well as technology consortia such as Hyperledger and also consortia specific to their sector (e.g. R3 I banking and B3I in Insurance).
 
PJT What is next for Distlytics?
 
GPN If a week is a long time in politics then it’s an age in blockchain. There is so much happening around the World. I’m working with a number of fellow consultants to build a Global capability, known as “Team Blockchain”, to help company board executives to better understand the risks and opportunities that the technology offers. I’ll continue to offer bespoke consultancy through Distlytics and will continue research into Distributed Ledger Analytics.
 
PJT What is next for Gary Nuttall and do you see blockchain as being at the centre of your future endeavours?
 
GPN I’m not a surfer but I think that blockchain is a huge wave of opportunity. It’s all about timing it right and choosing the time to surf it and, importantly, realise when it’s time for the next wave. There’s plenty to do in the blockchain space for a few years I reckon and then there’ll be another wave – Artificial Intelligence, Robo-Process Automation, Internet of Things are all growing (and in many ways complement blockchain nicely). Meanwhile I’ve become an advisor for Blocksure who’re developing a (General) Insurance platform. Having met many startups who’re working on prospective industry solutions, these guys are worth watching!

I’ll be monitoring how long the blockchain wave continues to grow and provide opportunities whilst watching the other waves. Of course, the really exciting stuff is when big waves converge and technology is no different – Blockchain, AI and IoT all provide massive opportunities. Now if we link them together then the opportunities become paradigm shifting.

PJT Gary, thank you for your time and the insights and information you have provided.
 

Gary Nuttall can be reached at gnuttall@distlytics.com. Distlytics’s website is www.distlytics.com and Gary regularly tweets with the @gpn01 hashtag.
 


 
Disclosure: Neither peterjamesthomas.com Ltd. nor any of its directors have any direct financial interest in either Distlytics or any of the other companies or organisations mentioned in this article.
 

 

Ideas for avoiding Big Data failures and for dealing with them if they happen

Avoid failure

In August 2016, I read an article by Paul Barsch (@paul_a_barsch), who at the time was Teradata‘s Marketing Director for Big Data Consulting Services [1]. I have always had a lot of time for Paul’s thoughts; and of course anyone who features the Mandelbrot Set so prominently in his work deserves a certain amount of kudos.

Paul Barsch

The title of the article in question was Big Data Projects – When You’re Not Getting the ROI You Expect and the piece appeared on Paul’s personal blog, Just Like Davos. Something drew me back to this article recently, maybe some of the other writing I have done around Big Data [2], but most likely my recent review of areas in which Data Programmes can go wrong [3]. Whatever the reason, I also ended up taking a look at his earlier piece, 3 Big Data Potholes to Avoid (December 2015). This article leverages material from each of these two posts on Paul’s blog. As ever, I’d encourage readers to take a look at the source material.

I’ll kick off with some scare tactics borrowed from the earlier article (which – for good reasons – are also cited in the later one):

[According to Gartner] “Through 2017, 60% of big data projects will fail to go beyond piloting and experimentation and will be abandoned.”

As most people will be aware, rigorous studies have shown that 82% of statistics are made up on the spur of the moment [4], but 60% is still a scary number. Until that is you begin to think about the success rate of most things that people try. Indeed, I used to have the following stats as part of my deck that I used internally in the early years of this decade:

“Data warehouses play a crucial role in the success of an information program. However more than 50% of data warehouse projects will have limited acceptance, or will be outright failures”

– Gartner 2007

“60-70% of the time Enterprise Resource Planning projects fail to deliver benefits, or are cancelled”

– CIO.com 2010

“61% of acquisition programs fail”

– McKinsey 2009

So a 60% failure rate seems pretty much par for the course. The sad truth is that humans aren’t very good at doing some things and complex projects with many moving parts and lots of stakeholders, each with different priorities and agendas, are probably exhibit number one of this. Of course, looking at my list above, if any of the types of work described is successful, then benefits will accrue. Many things in life that would be beneficial are hard to achieve and come with no guarantee of success. I’m pretty sure that the same observation applies to Big Data.

If an organisation, or a team within it, is already good at getting stuff done (and, importantly, also has some experience in the field of data – something we will come back to soon), then I think that they will have a failure rate with Big Data implementations significantly less than 60%. If the opposite holds, then the failure rate will probably exceed 60%. Given that there is a continuum of organisational capabilities, a 60% failure rate is probably a reasonable average. The key is to make sure that your Big Data project falls in the successful 40%. Here another observation from Paul’s December 2015 article is helpful.

If you build your big data system, chances are that business users won’t come. Why? Let’s be honest—people hate change. […] Big data adoption isn’t a given. It’s possible to spend 6-12 months building out a big data system in the cloud or on premise, giving users their logins and pass-codes, and then seeing close to zero usage.

I like the beginning of this quote. Indeed, for many years my public speaking deck included the following image [5]:

Field of Dreams

I used to go on to say some variant of the following:

Generally if you only build it, they (being users) are highly unlikely to come. You need to go and get them. Why is this? Well first of all people may have no choice other than to use a transaction processing system, they do however choose whether or not to use analytical capabilities and will only do so if there is something in it for them; generally that they can do their job faster, better, or ideally both.

Second answering business questions is only part of the story. The other element is that these answers must lead to people taking action. Getting people to take action means that you are in the rather messy world of influencing people’s behaviour; maybe something not many IT types are experts in. Nevertheless one objective of a successful data programme must be to make the facilities it delivers become as indispensable a part of doing business as say e-mail. The metaphor of mildly modifying an organisation’s DNA is an apt one.

Paul goes on to stress the importance of Executive sponsorship, which is obviously a prerequisite. However, if Executive support forms the stick, then the Big Data team will need to take responsibility for growing some tasty carrots as well. It is one of my pet peeves when teams doing anything with a technological element seem to think that is up to other people (including Executive Sponsors) to do the “wet work” of influencing people to embrace the technology. Such cultural transformation should be a core competency of any team engaged in something as potentially transformational as a Big Data implementation [6]. When this isn’t the case, then I think that the likelihood of a Big Data project veering towards the unsuccessful 60% becomes greater.

Einstein on Experience

Returning to Paul’s more recent article, two of the common mistakes he lists are [7]:

  • Experience – With millions of dollars potentially invested in a big data project, “learning on the job” won’t cut it.
     
  • Team – Too many big data initiatives end up solely sponsored by IT and fail to gain business buy-in.

It was at this point that echoes from my recent piece on the risks impacting data programmes became a cacophonous clamour. My risk number 4 was:

Risk Potential Impact
4. Staff lack skills and prior experience of data programmes. Time spent educating people rather than getting on with work. Sub-optimal functionality, slippages, later performance problems, higher ongoing support costs.

And my risk number 16 was:

Risk Potential Impact
16. In the absence of [up-front focus on understanding key business decisions], the programme becoming a technology-driven one. The business gets what IT or Change think that they need, not what is actually needed. There is more focus on shiny toys than on actionable information. The programme forgets the needs of its customers.

It’s always gratifying when two professionals working in the same field [8] reach similar conclusions.

It is one thing to list problems, quite another to offer solutions. However Paul does the latter in his August 2016 article, including the following advice:

Every IT project carries risk. Open source projects, considering how fast the market changes (the rise of Apache Spark and the cooling off of MapReduce comes to mind), should invite even more scrutiny. Clearly, significant cost rises in terms of big data salaries, vendor contracts, procurement of hard to find skills and more could throw off your business value calculations. Consider a staged approach to big data as a potential panacea to reassess risk along the way and help prevent major financial disasters.

Thomas Edison

Having highlighted both the risk of failure and some of the reasons that failure can occur, Paul ends his later on a more up-beat tone:

One thing’s for sure, if you decide to pull the plug on a specific big data initiative, because it’s not delivering ROI it’s important to take your licks and learn from the experience. By doing so, you will be that much smarter and better prepared the second time around. And because big data has the opportunity to provide so much value to your firm, there certainly will be another chance to get it right.

The mantra of “fail fast” has wormed its way into the business lexicon. My critique of an unthinking reliance on this phrase consists of the comment that failing fast is only useful if you succeed every now and again. I think being aware of the issues that Paul cites and listening to his guidance should go some way to ensuring that one of your attempts at Big Data implementation will end up in the successful category. Based on the Gartner statistic, then if you do 5 Big Data projects, your chances of all of them being unsuccessful is only 8% [9]. To turn this round there is a 92% chance that at least one of the 5 will end in success. While this sounds like a more healthy figure, the key, as Paul rightly points out, is to make sure you cut your losses early when things go badly and retain some budget and credibility to try again.

Samuel Beckett

Back in March 2009, when I wrote Perseverance, I included a quote that a colleague of mine loved to make in a business context:

Ever tried. Ever failed. No matter. Try again. Fail again. Fail better. [10]

I think that the central point that Paul is making is that there are steps you can take to guard against failure, but that if – despite these efforts – things start to go awry with you Big Data project, “it takes leadership to make the right decision”; i.e. to quit and start again. Much as this runs against the grain of human nature, it seems like sound advice.
 


 
Notes

 
[1]
 
He has since moved on to EY.
 
[2]
 
Including:

  1. The Big Data Universe
  2. Do any technologies grow up or do they only come of age?

And some pieces scheduled to be published during the rest of February and March.

 
[3]
 
20 Risks that Beset Data Programmes.
 
[4]
 
Seemingly you can find most percentages quoted somewhere, but the following is pretty definitive:

https://www.google.co.uk/search?q=82+of+statistics+are+made+up

 
[5]
 
I would be remiss if I didn’t point out that the actual quote from Field of Dreams is “If you build it HE will come”. Who “he” refers to here is pretty much the whole point of the film.

 
[6]
 
Once more I would direct readers to my, now rather venerable, trilogy of articles devoted to this area (as well as much of the other content of this site):

  1. Marketing Change
  2. Education and cultural transformation
  3. Sustaining Cultural Change
 
[7]
 
I have taken the liberty of swapping the order of Paul’s two points to match that of my list of risks.
 
[8]
 
Clearly a corn [maize] field in the context of this article.
 
[9]
 
7.78% is a more accurate figure (and equal to 60%5 of course).
 
[10]
 
Samuel Beckett, Worstward Ho (1983).

 

 

Solve if u r a genius

Solve if u r a genius - Less than 1% can do it!!!

I have some form when it comes to getting irritated by quasi-mathematical social media memes (see Facebook squares “puzzle” for example). Facebook, which I find myself using less and less frequently these days, has always been plagued by clickbait articles. Some of these can be rather unsavoury. One that does not have this particular issue, but which more than makes up for this in terms of general annoyance, is the many variants of:

Only a math[s] genius can solve [insert some dumb problem here] – can u?

Life is too short to complain about Facebook content, but this particular virus now seems to have infected LinkedIn (aka MicrosoftedIn) as well. Indeed as LinkedIn’s current “strategy” seems to be to ape what Facebook was doing a few years ago, perhaps this is not too surprising. Nevertheless, back in the day, LinkedIn used to be a reasonably serious site dedicated to networking and exchanging points of view with fellow professionals.

Those days appear to be fading fast, something I find sad. It seems that a number of people agree with me as – at the time of writing – over 9,000 people have viewed a LinkedIn article I briefly penned bemoaning this development. While some of the focus inevitably turned to general scorn being heaped on the new LinekdIn user experience (UX), it seemed that most people are of the same opinion as I am.

However, I suspect that there is little to be done and the folks at LinkedIn probably have their hands full trying to figure out how to address their UX catastrophe. Given this, I thought that if you can’t beat them, join them. So above appears my very own Mathematical meme, maybe it will catch on.

It should be noted that in this case “Less than 1% can do it!!!” is true, in the strictest sense. Unlike the original meme, so is the first piece of text!
 


Erratum: After 100s of views on my blog, 1,000s of views on LinkedIn and 10,000s of views on Twitter, it took Neil Raden (@NeilRaden) to point out that in the original image I had the sum running from n=0 as opposed to n=1. The former makes no sense whatsoever. I guess his company is called Hired Brains for a reason! This was meant to be a humorous post, but at least part of the joke is now on me.

– PJT