When I speak with clients about their data management architecture — and the accessibility, availability and flow of their business data, both internally and externally — the conversation often turns to data hubs, data lakes, data marts and data warehouses.
The problem with this type of discussion is it detracts from the main issue regarding clients’ most prominent data pain point: the lack of real-time accessibility and flow of data among multiple systems in response to business or customer events or transactions.
After all, one of the cornerstones of a digital business is making data available to those who need it at the right time. Modern businesses are driven by real-time business insights, decisions and reporting.
The real conversation
What we should be talking about is the role an integration data hub can play in addressing most of the real-time data challenges clients experience. That’s because very often, these challenges are the result of using traditional point-to-point batch-mode integration among systems for operational data. The issue with point-to-point integration is that it gets complex quickly: If a company has 10 systems that data needs to be moved or exchanged with, there would be up to 90 bi-directional integration lines, according to the n (n-1) connection rule.
The result: complex and costly IT maintenance, challenges with managing multiple copies of data and databases, security risks, and inconsistent data definitions across the organization. Business agility and innovation also suffer.
From point-to-point to hub-and-spoke
Using an integration data hub, however, businesses can quickly and easily streamline access to operational data from systems of record. A centralized data hub is not a technology per se but a method for sharing and communicating data and connecting core IT systems in a real-time, event-driven, hub-and-spoke pattern instead of the traditional point-to-point integration approach.
A data hub enables information sharing by connecting data producers with data consumers. Systems of record publish their data in real-time to the integration data hub so applications can access, consume and use the data in real-time. The hub provides a point of mediation, governance and visibility to how data flows across the enterprise. It defines data-level access, as well as policies on how long the data is kept.
Integration data hubs come to life
Many healthcare, retail and insurance clients have achieved a high degree of success creating an integration data hub. For example, we helped a health payer make claims data from core systems available to upper-stream processes and customer-facing applications within 20 seconds of update or change in status. We also assisted a retail client to provide accurate multi-site inventory updates to its point-of-sale system within 12 seconds.
In all of these cases, the integration data hub helped streamline operational data access easily and quickly, publishing data to upper-stream operational systems and historical and analytical ones. Both are important:
- Upper-stream systems need the data to complete business transactions that cut across multiple IT systems, such as issuing payment for a processed claim or displaying real-time updates to a member portal or app. Both of these capabilities positively impact customer service and satisfaction.
- Analytics platforms use the data to create real-time business insights and reports or to train artificial intelligence/machine learning (AI/ML) models. Doing so leads to real-time, data-driven business decisioning and visibility.
Taking a holistic view
One of the critical mistakes I find clients fall into is thinking of data in isolation of business processes, and as a byproduct of a system rather than the core product. But modernizing IT demands a holistic, outside-in view of how data, applications and businesses interact to deliver value.
Real-time, streamlined accessibility of data has become a business necessity; data is valuable when it moves, not at rest. As a result, organizations that best manage their data and how it flows to the right people at the right time — quickly — will emerge highly competitive.
A data hub is one of the key solutions organizations need to achieve real-time, event-driven, distributed accessibility of data and realize business advantage from their IT modernization efforts.