Why Storage Matters for AI: Solidigm Shares POV at Tech Field Day

That’s the question Ace Stryker, Director of Market Development, and Alan Bumgarner, Director of Strategic Planning, set out to answer for industry experts and influencers at Gestalt IT’s AI Field Day on February 21, 2024.

Not surprisingly, the short answer is that it’s all about the massive amounts of data that AI models need to process in order to learn and improve.

Ace and Alan discussed why solid-state storage drives (SSDs) are particularly well suited to AI use cases like large language models (LLMs) such as Chat GPT4, recommendation engines, photo tagging, surveillance, and inventory management, as well as multimodal use cases that integrate a number of these elements.

They emphasized how Solidigm has an industry-leading storage portfolio that optimizes and accelerates AI infrastructure, scales efficiently and improves operational efficiency, whether stored and accessed in data centers or at the near edge (e.g., regional data center) or the far edge (e.g., ruggedized, in a vehicle, in the field).

Another area the Solidigm team emphasized was the value of SSDs over hard disk drive (HDD) arrays.

"When it comes to AI, SSDs outperform HDDs in every way,” said Ace. “Within the AI structure, one storage device is not doing one thing at a time. They may be hit by multiple pipelines at the same time – involving training, checkpointing, and preparation. These things happen in parallel, generating more of an I/O workload.”

He also emphasized Total Cost of Ownership (TCO) as a critical element in where SSDs shine over all-HDDs arrays. Stryker compared the number of drives, servers, racks, power density and energy costs for an all-HDD Array (3.5” 24TB) vs. an all-SDD array (Solidigm D5-P5336 61.44 TB) at 70% utilization, to demonstrate this point. This chart illustrates the comparison:

“Not only are these drives storing quite a bit more data,” said Ace. “They are also physically smaller… that has a direct impact on the number of servers you need to populate the storage you require, and how many racks are needed to house those servers. So, you can think of it almost as a cascading effect where fewer smaller drives with higher densities lead to fewer servers, fewer racks, smaller data center footprint, and power optimizations as well.” He went on to share that all of these have sustainability implications.

Alan further emphasized the focus on power with an example from Meta and Stanford University. “The number one thing that most of these folks are thinking about is power,” he said.

Alan took attendees through the five stages of AI work training: data ingest, data prep, training, checkpointing, and inference. He talked about storage challenges with each of these, the workloads they require, and how SSDs are optimal for these workloads.

In summary, Ace and Alan emphasized the critical role storage plays for AI servers: accelerating workflows and seeing them through end to end with efficiency. Said Ace, “With data sets getting bigger and bigger and the models getting more sophisticated, storage needs to be able to scale efficiently in order to keep GPUs fed, keep utilization high, and make AI servers as efficient as possible during every stage of the workflow.” He also underscored cost as an important piece of the SSD story as related to power efficiency, rack space, and data center footprint.