Marvell Unveils AI Memory Strategy to Boost Server Efficiency. What That Means for MRVL Stock.
Article excerpt
Highlighted: the sentence this signal was extracted from
Marvell Technology (MRVL) has spent the past decade building chips that move, store, and process data for the world's biggest cloud companies. Now, the chipmaker is turning its attention to a problem that is slowing down artificial intelligence platforms. As AI models grow bigger and chatbots hold longer conversations, the computers running them are running out of fast, accessible memory. Marvell says it has a plan to address this issue while accessing every layer of a data center, from a single server to an entire building. Modern AI chatbots answer questions and remember every conversation. Every word a user types gets stored temporarily in something called a KV cache, short for key-value cache, so the AI can refer back to it. The problem is that these caches are getting massive. Longer conversations, bigger AI models, and the rise of AI agents that complete multistep tasks on their own require more memory than a single server chip can hold. When a processor has to wait for data because memory is full or too far away, it sits idle. Marvell calls this a GPU stall, and it's expensive. Marvell announced new products aimed at this bottleneck, and the lineup spans three levels of a data center. This setup can offload up to 32 terabytes of cached data and deliver two to three times more AI output within the same power budget. "AI infrastructure is moving beyond isolated servers...
Keep reading with a free account
The rest of this article, and every signal for Marvell, is in your free account.
/AI%20(artificial%20intelligence)/Ai%20chip%20by%20Quality%20Stock%20Arts%20via%20Shutterstock.jpg)