Fujitsu has started selling middleware that optimizes the use of GPUs, so that those lucky enough to own the scarce accelerators can be sure they're always well-used.
The software is available immediately, according to a Tuesday announcement, which reveals the Japanese tech sumo created the GPU allocator in November 2023.
The software distinguishes between programs that require a GPU and those that will do fine with a CPU alone, and assigns the appropriate resource.
It can also allocate resources in real time "to give priority to processes with high execution efficiency, even if the GPU is running a program."
Fujitsu teased the tech in November 2023 and, at the time, said it requires programmers to use a framework of its own devising that allowed applications to work alongside a GPU allocation server.
Fujitsu's Tuesday announcement does not specify whether that requirement remains - but the released product does combine the allocator with AI processing optimization tech so that it can identify needs and allocate resources on a per-GPU basis.
The tech titan claimed the middleware exhibited a 2.25x increase in computational efficiency during trials at AI-camera solutions company AWL Inc, cloud-based AI services provider Xtreme-D and cloud-computing space operator Morgenrot Inc.
It also asserted that memory management tech in the tool allows GPUs to handle tasks that need more memory than is present on an accelerator.
Fujitsu plans to continue developing the tech so it can work across multiple GPUs installed on multiple servers. It's currently confined to single boxes - so perhaps not the sort of thing that will make a big training cluster sing, but a handy way to wring more out of a GPU-equipped server.
Two big customers have already started using the tool - namely Japanese fintech Tradom and cloud services provider Sakura Internet, which will soon begin a feasibility study for employing it in its datacenter operations.
Ever the hero, Fujitsu claimed the middleware would not only fully use the GPU, but also address global shortages.
"By addressing the challenges of GPU shortages and power consumption driven by the increasing global demand for AI, Fujitsu aims to contribute to enhanced business productivity and creativity for its customers."
Whether AI actually delivers productivity is a hotly contested idea. That there is a shortage of GPUs is not: US regulators have expressed concern that accelerator vendors make it hard for anyone other than hyperscalers to acquire them, and vendors have complained of long lead times from leading supplier Nvidia.
Speaking of Nvidia, it has tried to reassure nervous buyers that its next-gen Blackwell accelerators will soon roll off production lines in big numbers.
But even if Nvidia and its prime competitor AMD can deliver more hardware, Fujitsu is not alone in offering GPU sharing tools and services. While the AI bubble is expected to burst at some point, orgs of all sorts are rushing to put generative AI to work, creating enormous demand for GPUs and datacenters. ®
Congress urged to act before year's end to support US competitiveness
Another step toward tech independence for China - on the day Tim Cook came to town
Middleware aimed at softening the shortage of AI accelerators
From hell's heart I stab at thee
Appen points to, among other problems, a lack of high-quality training data labeled by humans
Microsoft blocks updates to avoid giving admins another headache
Unisys, Avaya, Check Point, and Mimecast settled with the agency without admitting or denying wrongdoing