The Big Idea

Why should you read this book?


Big data alone cannot lead or manage a business. Imagine you have a big data system right now – what would you do with it?


A lot of businesses are ramping up big data systems. The sad thing is that some of these implementations will never pay for themselves, and the vast majority will not garner nearly the benefit that was promised by a vendor or consultant. Perhaps you are considering starting such a project, or perhaps you are saddled with one that is not performing to your expectations. This book is for you!


The piece that is missing is the business itself. Just gathering a lot of data is unlikely to answer your questions. It is like pointing a telescope at some random part of the sky and then trying to see Jupiter. You might have accidentally caught it in your field of view, but it is not really all that likely. Similarly, knowledge about the business is required to focus the data collection and analysis in those areas that are relevant to your business.
The problem is that most businesses also do a pretty terrible job about knowing how the business itself works. So before we can talk about your big data system, we have to talk about your business.


Leading a business is about envisioning a new destination and enabling the business to go there. Managing a business is about making decisions that allow your people to do what needs to be done to make that business successful. Leading a business is executing change, and is documented in a strategic plan. Managing a business uses data to make decisions using a decision support system. Both of these functions require measuring things – data in other words. Most businesses have a lot of metrics, but very few in our experience have metrics that are actually designed to measure what is important to lead and manage a company. They are not integrated across the company to accomplish the objectives of the company. Most big data efforts going on now are tacked onto existing, non-integrated metrics, and so you have a recipe to measure the wrong things faster and in more detail than ever before. That leads to suboptimal, or even outright damaging, management decisions for which you have paid a lot of money.


The book is based on bringing together two areas of knowledge that have not been effectively brought together before – leading and managing a business and big data.


Of course, such a blending has been attempted before, but one is always sacrificed on the altar of the other. The approach we outline is more of a marriage than a capitulation.


The first part of the book is based on a few premises:

  • Building and managing a business is about people first
  • People determine why the business exists
  • The aspirations of the business can be turned into metrics that measure if it is fulfilling its purpose
  • Incorrect or incomplete metrics drive incorrect behavior and there is no list of the “right” metrics for any type of business
  • Business-level metrics can be translated into metrics for everyone in the business
  • The day-to-day metrics are managed and improved by every level in the organization
  • Just managing day-to-day means that you fall behind your competition. You need to also plan to close gaps in business-level metrics. This is called a strategic plan.
  • Closing gaps (breakthroughs) requires a different type of work than continuous improvement (kaizen)
  • Therefore, a strategic plan will be achieved by deploying projects to affect those metrics throughout the business that are related to the changes needed while maintaining performance in those areas that are not.

The first part of the book describes how the leadership of an organization sets the direction of the company, translates those words into metrics, cascades these metrics throughout the organization, determines the needs for improvement with a gap analysis, and deploys a strategic plan to close those gaps while still doing the day-to-day activities that keep the lights on.


This leads you to the right questions to ask and the people’s best guess as to what metrics would help answer those questions. This process works well by itself, however people are slow and prone to missing important connections. In today’s business world, it is often not about getting the best answer, but about getting a “good enough” answer to the right questions more quickly than your competitors. That is where the second part of the book comes in.


The second part of the book shows you how to design a system to gather data to validate those metrics and answer those questions more completely and more quickly, while providing insights that are relevant to your organization.


Something revolutionary is happening in the world of data. Going all the way back to Galileo (and beyond!) data was very expensive to generate. If you were running a business and wanted to use data to drive your decisions, you probably had to hire or engage experts in how to properly design, measure, collect, and analyze the processes. This is the era of conventional data analysis. This meant using tools and techniques to maximize your ability to make a decision while minimizing the cost of doing so. In the recent past, such things as experimental design and Six Sigma were critical tools to figure out how to extract the absolute most amount of information from the least amount of data. In many situations, these tools are still essential for businesses today, and if that is where your business you would do well to be thinking in this way. Big data should not be the next stop on your journey.


However, for the first time in human history, we are now seeing data that is cheap and everywhere. Companies now have processes that are monitored every tenth of a second, 24/7 with all the data being deposited into a data lake. Other businesses track millions of customers as they browse and buy products. This is the era of big data. These companies have a wealth of data – but the massive data sets tend to be happenstance data, rather than collected by design, and it may be of low quality as well. The challenge in these situation is in taking massive amounts of data and turning it into knowledge that can be used to improve the performance of the business. This means that you have two difficult problems to solve: asking the right questions from your big data and how to get the answers. The former is a question for the business, the latter is a question for data science.


What the authors have noticed is that businesses generally fall into the following categories:

The Main Sequence (- -): These are businesses that have neither a unified idea of what the business is intended to achieve nor an integrated way of making management decisions. These companies go in all directions at once, meaning that they don’t really go much of anywhere at all. This is by far the most common type of business. Adding big data systems into these businesses tends to just further add confusion to the mix. These companies need to unify the business behind a common vision, then begin to ask the data how to get there. They can then move into the next classification…


White Dwarfs (+ -): These are rare companies that have a unified vision and the management systems to support it, but have conventional types of data: expensive and rare. They can deploy their resources to strategically close gaps, however, they may find that they are falling behind their competitors who may be making decisions faster using big data. The good news is that they can easily use their systems to transition into the realm of big data should they want it.


Red Giants (- +): These are companies who lack a unified vision or the structures to support a unified vision, but have been convinced to invest in a big data system. These businesses are at real risk of blowing up into a “supernova,” since they have just spent a lot of money to get access to a type of data that they are not well-equipped to utilize. These businesses need to build management systems to align everyone in the company behind a unified vision, and then create the business processes to achieve that before they can get much use out of their big data system.


Magnetars (+ +): These are the extremely rare companies who have a well-defined vision, management systems to accomplish the vision, and big data helping them find good solutions inside and outside of the company faster than their competition. These organizations are the most powerful in the business cosmos, and it is the goal of this book to help you map a path towards this ideal.

The clever reader will have noticed these business categories are based on two factors:

  • a unified vision and business processes to get there
  • the presence or absence of a big data system

These are indicated by the + and – following the name. You might think of it as the human business side and the big data business side. Together, these two factors interact to allow businesses to achieve their human objectives faster and better than ever before.


Add comment

We are all here to learn something, and we know we don't have all the answers so we would love to hear what you think. If what you say causes us to change the content, we will credit you in the text. Oh, and let's keep it civil and helpful!


Security code
Refresh