Data’s traditional flow is neat and linear: It moves from a source to a data warehouse, and then into a form that readies it for reporting.
When data is used to shape and train artificial intelligence (AI), it breaks that mold.
Data moves freely across the enterprise — and that’s challenging for companies. As they get ready for AI, organizations face thorny questions about how to manage data and insights that move in multiple directions and are often generated outside the data platform.
Who’s in charge?
AI raises a host of complex issues for enterprises regarding how best to treat the huge volumes of data generated by sensors, devices, channels and external sources. Companies tackle tough questions such as should all the data generated or sourced be brought back to the data platform? How much should they retain? What portions are valuable, and how should they distribute the sourced data across the enterprise and into downstream consumption systems?
Perhaps the most important question relates to accountability. Who will be responsible for these decisions? The answer has direct implications for enabling and harvesting the intelligence created outside the data platform.
Hammering out these details is new to most companies. Typically, data-related decisions are parceled out among three teams: infrastructure, applications and data. AI systems require a more integrated process. The data organization has a natural, logical opportunity to assert ownership for enabling a data architecture that supports the required distribution mechanism. But it also has to be up for the challenge. That is, has it thought through what needs to change? More important, is its data architecture ready for the change?
The seven design principles of an AI-ready data architecture
Backing up well-planned data leadership is a data architecture that’s responsive and able to accommodate AI data’s freeform movements to wherever intelligence is required within the enterprise.
It’s a complex journey: The architecture has to bring in data quickly and present it to multiple consumption points in the enterprise. In the new digital enterprise ecosystem, consumption points run the gamut from processes and applications to edge devices, sensors and services.
We see that many companies use the cloud to replicate their existing on-premise data architecture. It’s a comfortable solution, but it falls short of unlocking the new capabilities the cloud offers.
The cloud offers organizations freedom like never before when it comes to computing, storage and operations. It expands the possibilities and sets up the enterprise for applying advanced forms of AI (i.e., machine learning) to solve complex business problems. The restrictions of traditional data models and architectures have no place in the cloud. In fact, taking advantage of AI requires a complete rethink and redesign. Your organization needs to answer honestly whether it’s ready to press the reset button and refresh.
AI raises the bar. The following seven design principles will help your organization get AI-ready and make the most of it with a data architecture that enables hyper personalization and real-time decision-making, as well as enhanced business agility and more expansive data management.
Plan for scale and elasticity.
AI is all about data, all the time. Does your IT team’s architecture enable computations to be performed on demand? Does the environment allow users the freedom to, say, apply a formula to a large data set without first asking IT to check server capacity? Scale and elasticity are at the heart of AI. A cloud-enabled data architecture offers elasticity, letting your organization scale up for the moments where additional computing horsepower is needed — or isn’t.
Shape an architecture that can ingest all types of data and measure changes over time.
With 5G moving toward widespread deployment, there’s no excuse for not getting data in real time. Yet there’s also no point in accessing richer sources of data unless you have an architecture that can consume it. An AI-ready architecture is able to address different shapes and granularities of data such as transactions, logs, geospatial information, sensors and social. In addition, real-time time-series data is key to the constant feed of input that propels data-driven devices, from smart-home appliances and health devices to self-driving cars. Make sure your AI architecture has the capability to consume different data structures in different time dimensions, especially real time.
Be metadata driven from the start.
Is your organization identifying and classifying data at the point of ingestion? Most enterprises view metadata extraction as an afterthought, typically driven by compliance. Yet metadata is much easier to manage early in the process rather than later, and it has value to organizations far beyond compliance. For example, cataloging your company’s metadata can create a library of data sets that everyone in the organization can access, thereby enabling wider use of insight-generation and AI throughout the enterprise.
Provide open access across all layers.
Platforms have three layers of data: raw, curated and consumption. Older architectures frequently grant access only to the consumption layer. That’s a problem for decision scientists, who often like to examine raw data for overlooked elements that may generate more information. Be sure all of your architecture’s layers are exposed and open for access.
Enable autonomous data integration.
Mapping to target usage environments still remains a largely manual process. As your team rethinks its AI data architecture, consider applying machine learning (ML) in data integration so that your integration layer can automatically detect changes in incoming data and adjust the integration patterns with no manual intervention. Feeding AI means a steady stream of changing source systems. If your company still requires a month to integrate a new source, it risks losing out on the continuous flow of data that makes AI so powerful.
Get feature engineering right.
ML is often underutilized in enterprises because the data isn’t ready and usable. Feature engineering offers the answer. It transforms data into consumable forms and shapes that ML models can use. Because features describe data points and serve as inputs into the learning system, they need to be as precise as possible. Careful feature engineering plays an important role in an organization’s ability to make ML accessible to everyone within the business. It’s data science freedom.
Support a unified security model for data.
If your enterprise is like most, it relies on a complex, hybrid environment that blends cloud-based and on-premise services. Data resides in scattered locations, consumed by individuals and reports as well as other applications and devices. Security is important to AI because of its many variables: consumers and third parties may or may not opt in to share their personal data, corporate application systems may apply varying levels of security on different data sets, and producers and consumers of data may be the same or different. AI’s issues of trust and ethics also influence security. A unified security approach lets you begin considering security from the point that data is produced to all points of consumption and cycles of enrichment.
Essential Components of Real and Responsible AI (Part 1)
Listen to Podcast
In this 2-part episode series, experts from Cognizant’s AI & Analytics practice, Poornima Ramaswamy, Dr. Jerry Smith & James Jeude suggest ways to get these elements right and responsibly embrace machine intelligence.
Essential Components of Real and Responsible AI (Part 2)
Listen to Podcast
In this second part of the AI podcast series, our experts Poornima Ramaswamy, Dr. Jerry Smith and James Jeude from Cognizant Digital Business’ AI & Analytics Practice share their views on how to bring AI into the business mainstream.
Like all businesses, your data organization is likely at an inflection point, and change is inevitable. The question for the data organization is does it want to lead, or to be led? The answer will determine whether your enterprise is ready to transition to a data-driven environment.