In an always-on digital economy, the shelf life of customer and market insights is constantly shrinking. To uncover and act on trends quickly enough to generate value, many companies are moving aggressively to gather, analyze and generate insights from data that is gathered immediately — or very soon after it is created. Whether it’s a customer seeking a higher credit limit, an out-of-stock item in a store or a spike in a patient’s blood pressure, acting on insights in the moment can improve the customer experience, increase revenue, or even save a life.
However, all these benefits come at a cost. Gathering, analyzing and acting on real-time data requires specialized infrastructure, skills and processes that include not only technology, but analytical skills and changes to business processes, security, privacy and governance. The cost of real-time data can rise exponentially the closer to real-time the analytics process becomes. Real-time analytics systems must also adapt to rapid and ongoing changes in the source data.
On the technology front, we find the systems that generate real-time data often wait to transmit it in batch processes for later analysis. Transmitting that data immediately may require greater computing power to continually identify changed records and map them to the target database. Providing that scalable computing power may require redesigning the application in a microservices architecture, or the use of middleware (such as message queues) that require new licenses and skills. Enabling decisions based on real-time data at the edge of the network, such as in production equipment or a retail store, also requires changes to applications and network architectures.
Making use of real-time analytics also requires changes to business processes and managing the effects of those changes on employees and business partners. For example, using real-time data such as traffic patterns, weather or nearby sporting events to determine food stocks and food preparation schedules in individual restaurants within a chain may require managers to make more frequent changes in everything from food preparation to schedules for their wait staff.
Because real-time analysis requires far more data, and far more types of data than ever before, it also raises new and more complex security, privacy and compliance issues. If, for example, a company uses information posted by a consumer’s social media account while she is on vacation in another country, might that data fall under that country’s data protection rules?
Enterprises that gather, manage and analyze more real-time data than they need can turn what should be quick-reaction analytic projects into expensive, lengthy efforts that do little to drive the business forward.
As with any other business tool, using real-time data effectively requires understanding its true value, including the cost of lost opportunities if the real-time data is not available, and exactly where it can best help the business.