In an always-on digital economy, the shelf life of customer and market insights is constantly shrinking. To uncover and act on trends quickly enough to generate value, many companies are moving aggressively to gather, analyze and generate insights from data that is gathered immediately — or very soon after it is created. Whether it’s a customer seeking a higher credit limit, an out-of-stock item in a store or a spike in a patient’s blood pressure, acting on insights in the moment can improve the customer experience, increase revenue, or even save a life.
However, all these benefits come at a cost. Gathering, analyzing and acting on real-time data requires specialized infrastructure, skills and processes that include not only technology, but analytical skills and changes to business processes, security, privacy and governance. The cost of real-time data can rise exponentially the closer to real-time the analytics process becomes. Real-time analytics systems must also adapt to rapid and ongoing changes in the source data.
On the technology front, we find the systems that generate real-time data often wait to transmit it in batch processes for later analysis. Transmitting that data immediately may require greater computing power to continually identify changed records and map them to the target database. Providing that scalable computing power may require redesigning the application in a microservices architecture, or the use of middleware (such as message queues) that require new licenses and skills. Enabling decisions based on real-time data at the edge of the network, such as in production equipment or a retail store, also requires changes to applications and network architectures.
Making use of real-time analytics also requires changes to business processes and managing the effects of those changes on employees and business partners. For example, using real-time data such as traffic patterns, weather or nearby sporting events to determine food stocks and food preparation schedules in individual restaurants within a chain may require managers to make more frequent changes in everything from food preparation to schedules for their wait staff.
Because real-time analysis requires far more data, and far more types of data than ever before, it also raises new and more complex security, privacy and compliance issues. If, for example, a company uses information posted by a consumer’s social media account while she is on vacation in another country, might that data fall under that country’s data protection rules?
Enterprises that gather, manage and analyze more real-time data than they need can turn what should be quick-reaction analytic projects into expensive, lengthy efforts that do little to drive the business forward.
As with any other business tool, using real-time data effectively requires understanding its true value, including the cost of lost opportunities if the real-time data is not available, and exactly where it can best help the business.
Considering the effort and expense required to use real-time data effectively and safely, we recommend a holistic evaluation that focuses on fast data use cases that will drive the most value most quickly for the business.
Here are six areas in which we have found fast data efforts have delivered business value:
A large US-based insurance company found its digital campaigns were not delivering the offers customers wanted. We reviewed its lead- management process and found customer segments were too fragmented and driven by a one-size-fits-all approach. We identified the most useful real-time data from the insurer’s customer relationship management (CRM) system and used it to train its machine learning (ML) algorithms to provide better real-time customer profiling, micro segmentation, and real-time multichannel campaign delivery. The result was a 20% improvement in lead conversion rates, and a 32% upturn in campaign click rates.
A major convenience store chain faced rising food waste costs because store managers were preparing food each day based on outdated sales information. We worked with the client to understand what subset of its real-time sales data would most likely yield useful insights. The result was a focus on the inventory of perishable food such as sandwiches and donuts, minimizing waste by ordering only enough as was likely to sell by the end of the day. The ability to fine-tune such ordering has saved the chain millions of dollars a year.
More complex regulations governing everything from financial transactions to data privacy require organizations to quickly find and fix violations. But not every type of data must be gathered, analyzed and acted on in real time for compliance purposes. A bank may need real-time access to data that would identify criminal activity in the making and prevent money laundering, but not to data required for monthly or quarterly reports.
B2C companies have traditionally grouped customers into broad markets, creating marketing campaigns for each that can take weeks or months to execute. That is far too long to influence the customer at the time they’re making a decision, whether in a brick-and-mortar store or online. We helped a retailer map its customers’ needs to real-time conditions, such as their location, the local weather and the customer’s physical or virtual browsing history, to increase sales through incentives such as an instant discounts.
We helped a major wireless provider reduce losses from fraudulent claims for phone upgrades by building a real-time data hub that provides staff with more real-time data, such as a customer’s contract type and payment history, and then apply business rules to approve or deny applications for an upgrade. This real-time fraud detection system also provides 90% of approvals in less than one second.
We used real-time data to help Grudfos, a global manufacturer of water management equipment, reduce the cost and time required to repair water pumps. We created a virtual agent that diagnoses problems using data from Internet of Things components such as pumps and presents potential solutions to technicians via a Microsoft HoloLens augmented reality dashboard and a chatbot. As a result, non-experts at remote facilities can handle complex repairs on their own, with guidance from Grundfos.
We also helped an elevator manufacturer reduce maintenance cost and effort by providing repair personnel a natural language chatbot into which they could enter information about the problem, such as symptoms and error codes, and receive recommendations based on real-time data about failures in similar equipment.
As an organization embraces real-time data, here are fundamentals on which to focus to achieve timely benefits.
Just collecting more real-time data does not make an enterprise digital. Carefully choosing the right business cases and factoring in the full range of technical and non-technical requirements drives the fastest and most long-lasting return on investment from fast-data efforts.