Single-platform approach may fall short for AI data management

Users should beware of the single platform approach when preparing for the demands of AI and machine learning on their data management systems, Gartner is warning.

A gamut of vendors including Snowflake, Google Cloud, Microsoft, and Databricks have spent the last couple of years trying to show that data management and analytics platforms can lay the ground for user organizations implementing AI and machine learning, which have been the subject of considerable hype.

Speaking ahead of her presentation at Gartner Symposium this week, Roxane Edjlali, senior director analyst, told The Register that while the single platform approach appealed to some users, they might need more than the solutions available on a single platform.

"The appealing part for enterprises is to say, if we take everything from the same stack, we are not going to need to assemble all of those pieces ourselves," she said. "But it might not be enough, and vendors are not at the same level of maturity across all of the different pieces that support AI-readiness of data."

Some of the major players in analytics and data management continue to make a play for AI and machine learning spending with their platform approach. For example, last year Databricks launched a complete overhaul based on technologies gained in its $1.3 billion buy of MosaicML, a generative AI startup.

Microsoft's Fabric platform supports seven core workloads: Data Factory (connectors), Synapse Data Engineering (authoring for Apache Spark), Synapse Data Science (build AI models), Synapse Data Warehousing, Synapse Real Time Analytics, Power BI, and Data Activator.

Snowflake has also launched a fully managed service designed to rid developers building LLMs into their applications of the onerous task of creating the supporting infrastructure, for example.

Edjlali said getting data management ready for AI requires three main practices, including observability, analytics, and AI governance. "These three pillars are not typically the strong suit of those data management vendors that all started from building DBMSes or data lake technologies. You can definitely see they're building it," she said.

The idea of a single data platform was also only an "aspiration" among users because the typical estate of data management technologies was so diverse.

"Most organizations have data still on-premises and in the cloud. They don't have everything on the same platform," Edjlali said. "Many organizations have multiple clouds because of mergers and acquisitions, or because large companies with many different departments may choose different solutions for different purposes, and so forth. There is an aspiration for simplification, but it is difficult for it to come true."

While Edjlali warned that getting data management technologies ready for AI was not a "built once and for all" activity, she said that AI itself could help in a number of the processes necessary.

"Not only is data management supporting AI, but data management techniques are also embedding AI to support data management activity. Those two trends are coming together to facilitate how you are interacting with data, producing that data tracking lineage. AI is increasingly embedded within those data management platforms. That is here to stay." ®