By Scott Rose, Vice President of Services, Think Big |
In my previous blog, I talked about the Internet of Things (IoT) and challenges related to implementation, in particular the technical and business problems organizations face launching and scaling IoT programs. One of those challenges is what I call ‘working in the long tail’.
Historically businesses have focused on optimizing “normal” operations, processes that addressed their customers’ predominant needs. So, for example, they managed supply chain and manufacturing to get high quality products at the lowest cost into the right markets at the right time, and sales and service to build and maintain profitable relationships with as many customers as possible. Analytics systems were designed to recognize exception conditions within the scope of these normal operations and to answer questions such as; “How are sales against plan for this product in the US?”
However, not many companies are operating in the long tail—capturing data at the edges of the enterprise and acting on exception conditions that are too few and far between to notice.
Let me give you an example. For one of our customers—a global telecommunications company—normal operations are built around average and peak network traffic (i.e. an average Monday on the
Internet, and Black Friday). This company designed their technical infrastructure and operations to handle the left side of the graph below, operating at nearly 100% uptime.
However to the right of the graph, in the long tail, is where this telecommunications company had the most opportunity to innovate their business, simply because they had never before been able to capture data about and act on business events that were a tiny fraction of their normal operations. To do this we helped the client build a big data platform to capture and analyze all network transactions. It was in the long tail that we found some amazing insights in areas such as cyber security. For instance, imagine finding tiny clusters of communication; a handful of transactions within trillions that identified a cyber-threat waiting to strike—many months sooner than they could previously identify that threat.
But it is one thing to identify insights in the long tail and another to operate there. Even though this telecommunications company could now identify individual network transactions that represented threats to their customers’ businesses, doing something about it, reliably and at scale, was a much greater challenge. To drive the business change operating in the long tail required, they made the transition a Board of Directors and CEO level commitment.
This transition to analyzing and acting on long tail data will challenge organizations in almost every industry. In another example, we are working with a storage manufacturer to make machine log data available for wider analytics. This manufacturer has hundreds of machines involved in the assembly and testing of products, with plants all over the world. Each day the company collects over 10 billion machine logs, hundreds of millions of records from operational data stores, and over a hundred million log files from devices in the field. It is a complex system, but the impact to the business from improved yield and faster triage is material and justified investment in an IoT solution.
One of the early wins for the system was an engineer diagnosing a single testing machine, out of 400, that was out of spec on a production line due to a faulty five dollar Ethernet cable. Prior to the new analytics solution, best case, they would have driven down yield by incorrectly failing product. Worst case, the diagnostic machine would generate a false positive and the company would ship bad product to customers, leading to millions in wasted product and loss of customer goodwill. Without the implemented IoT solution, this manufacturer would not have found this manufacturing anomaly until the next scheduled maintenance window.
For this manufacturer, to operationalize this scenario at scale requires significant business change. They will have to re-design operating metrics that have been core to their business for decades. They will need to re-engineer processes for quarterly production line maintenance and re-calibration to work in real-time. They will need to re-train plant managers and machine operators and maintenance staff and roll these changes out to plants globally. These kinds of changes will take investment and commitment, but the upside is significant competitive advantage.
Clearly, the biggest opportunities for companies embracing IoT reside in the long tail. Imagine the power to recognize new patterns at the edge, and forming new insights to improve your business based on a full record of your physical world. Seizing that opportunity and capitalizing on it is another challenge, what I call ‘path to production’, which I’ll discuss in my next blog.