As well as speed of deployment and cost efficiency, the RAG approach can also be more flexible than a custom LLM, as its results can reflect real-time changes in data without the entire model having to be trained.
Even here, though, the quality of the model’s output, and its overall function, is constrained by the quality and accessibility of the data it has to work with. Whichever approach a business chooses for its generative AI strategy, the odds of success are tightly indexed to how well its existing data architecture performs in the following key areas:
Cataloging and metadata
Data, by itself, is meaningless—even to the sprawling neural network of a modern LLM. The first pillar of data readiness, therefore, is that data be organized in a centralized data repository with comprehensive metadata describing its source, structure, content, and, ideally, meaning. If, as predicted, LLMs will soon be functioning as ‘agents’ for human users, delivering answers and results in response to natural-language queries and instructions, the LLMs not only need access to all relevant data, but to information about the data that gives it context and meaning. Without excellent metadata management, it will be difficult or impossible for LLM agents to be effective.
Data must also be accurate—especially if, as in the Custom LLM approach outlined above, it’s to be used for training a generative AI model. Before embarking on their gen AI journeys, companies should determine how much trust their leaders and workers have in their existing analytics, reporting, and or BI (Business Intelligence) dashboards. If the answer is “not much”, the leader would do well to invest in resolving these issues, before pumping resources into training LLMs on substandard data.
The importance of data security is no secret to most modern businesses, but any weak points in existing defenses are liable to be exposed, and quickly, in the coming age of generative AI. Because the behavior of LLMs is not deterministic or precisely predictable—this being the essence of their ‘creativity’—it’s hard to game out in advance how that malicious actors might ‘trick’ an LLM into divulging proprietary data, either about a business or its customers. (This is a particular concern when using a RAG-enhanced off-the-shelf LLM, the approach outlined above. Because the model interacts more frequently with proprietary databases, the risks of a potential breach are multiplied accordingly.) While these threats are to some extent unknowable, though, it’s a safe bet that businesses whose most sensitive data is best protected today will carry this advantage into the era of generative AI.
Speed and flexibility
To maximize gen AI’s potential, firms not only need data pipelines that can provide LLMs with the raw material for processing; they need to be ready to receive and store the torrents of new data that’s produced as a result. The construction of this data architecture needs to happen before the fact, and it needs to be flexible enough to handle a flow of information that may increase linearly or even exponentially in coming years. Expensive data warehouses or even RDBMS systems may start to hinder the amount of data that can be stored and processed cost effectively. It’s not too soon to start exploring modern ‘data lakehouse’ architectures, including scalable cloud object storage systems such as S3 or GCS.
As the future unfolds, paradoxically, the need to revisit the past will become more pressing and more frequent. The ability to restore or ‘replay’ previous versions of a dataset, is literally indispensable in the training, tuning, and testing of LLMs. Even for firms considering an ‘off-the-shelf’ LLM—pre-trained, pre-tuned and pre-tested—the replay ability of their existing data systems is a useful barometer of overall ‘data readiness,’ heading into the age of generative AI.
That goes for all the metrics listed here. Those firms without good data, and just-as-good data architecture, will find themselves at a competitive disadvantage as generative AI transforms the landscape of modern business; they should consider building a solid foundation of data readiness before they invest in their first LLM.
Conversely, those companies who already have their data ducks in a row are well positioned not only to reap the rewards of generative AI, but also—probably—of subsequent next big things as yet unimagined.
To learn more, visit the Generative AI or Consulting sections of our website or contact us.
This article was written by Jonathan Sims, Consultant in Cognizant’s Consulting practice.