Voltron Data throws its weight behind AMD for GPU-accelerated SQL

Exclusive Support for AMD's Instinct GPUs is coming to Voltron Data's accelerated SQL engine Theseus in the latest sign Nvidia's CUDA moat is getting shallower.

Introduced in late 2023, Theseus - named for the ship of Theseus since it's constantly being stripped down and rebuilt - uses GPUs to accelerate SQL queries, enabling large volumes of data to be processed at high speed.

You may recall early last year, Voltron demonstrated its Theseus platform, completing the full TPC-H 100 terabyte-scale factor benchmark on unsorted Parquet files pulled directly from storage in less than an hour.

The ability to churn through large quantities of data quickly has become increasingly important as enterprises look to capitalize on AI. That's because data usually needs to be pre-processed before it can be used for fine-tuning or integrated into something like a retrieval augmented generation (RAG) pipeline. And, in the case of the latter, the speed at which queries are processed can make a big difference. It doesn't matter how fast your AI infrastructure can churn out tokens if it's stuck waiting on a database query to finish.

Up to this point, this kind of GPU-accelerated data process has primarily run on Nvidia accelerators. Voltron's TPC-H benchmark, for example, used about 6TB worth of Nvidia accelerators. But now the company is looking to extend this functionality to AMD's Instinct family of chips.

"Our goal is to give customers the choice to run their SQL query engines on either Nvidia or AMD architectures," Rodrigo Aramburu, co-founder and field CTO of Voltron Data, told El Reg.

AMD's Instinct accelerators have become increasingly popular as an alternative to Nvidia's GPUs among hyperscalers and cloud providers, like Meta, Oracle, and Microsoft, in the year and a half since the MI300X made its debut. That's because it's relatively cheaper to run AI inference workloads on Instinct vs the equivalent Nvidia chips. Oracle last week announced plans to deploy 131,072 of AMD's latest generation MI355X accelerators.

But while it's relatively straightforward to get large language models (LLMs) running on either vendor's GPUs, AI is only one of many workloads users may want to accelerate.

Unfortunately, the CUDA software libraries necessary to run these workloads on Nvidia hardware don't always have an AMD equivalent.

As a reminder, the term CUDA gets thrown around a lot to describe Nvidia's low-level GPU programming language, but it's really a collection of libraries and frameworks optimized for accelerating all manner of workloads. This has contributed to the perception of the so-called CUDA moat.

AMD has invested considerable resources into building out its open-source software stack in order to close this gap. In fact, Aramburu notes it was only in the past few months that the libraries necessary to get SQL databases running on AMD's CDNA architecture made their way onto GitHub.

In particular, the latest iteration of Voltron's Theseus SQL Engine builds on hipDF, AMD's equivalent to the libcuDF libraries that underpinned Nvidia's data science platform RAPIDS.

Similar to cuDF, the hipDF is GPU accelerated DataFrame library for loading, joining, aggregating, filtering, and otherwise manipulating data that's based on Apache Arrow. The hipDF was introduced as part of AMD's ROCm for Data Science libraries last month.

But just because the libraries exist doesn't mean that integrating them into an existing platform will be easy or performant. The good news, Aramburu tells us, is that this process was much less of a headache than the engineering team had feared.

"We have been very impressed with how everything has worked," he said.

For Voltron's customers, all of this is abstracted. It doesn't matter whose GPUs you're using; your SQL databases can be accelerated by them.

The actual implementation is still early and support for Instinct Accelerators in Theseus remains in preview. However, Aramburu says that even in this early stage, the performance on AMD's platform is performing well.

"Early benchmarks show strong performance and scalability, validating our accelerator-native analytics stack across different silicon," Aramburu said.

Voltron is already planning another TPC-H run on AMD MI300s. Though this will likely come closer to general availability later this year.

Production support for the accelerators is expected to roll out late this year. ®