TL;DR:
- Habana Labs’ Gaudi chips for AI are Intel’s most competitive offering in the generative AI space.
- Gaudi2 offers low-power, high-speed AI training and inference relative to the competition.
- Mobileye has deployed Gaudi2 for training custom object detection models for autonomous vehicles.
- Gaudi3, currently in manufacturing, is set to offer significant performance boosts and improvements to power efficiency.
- Habana plans to integrate Gaudi chips with accelerators emerging from Intel’s Accelerated Computing Systems and Graphics group.
- Gaudi is expected to remain the recommended solution for heavy AI workloads for the next couple of years.
- Intel is designing the next-generation accelerator, which will combine Gaudi’s capabilities with some of AXG’s capabilities.
- Expect more news on Gaudi3 in the coming quarters.
Main AI News:
Since its acquisition by Intel just four years ago, Habana Labs has grown to become the center of excellence for Intel in AI solutions. The company’s Gaudi chips for AI are now Intel’s most competitive offering in the increasingly prominent generative AI space, according to Habana COO Eitan Medina.
Last year, Habana unveiled its second-generation AI training chip, Gaudi2, which brought Gaudi from 16nm (first-gen) to 7nm (both TSMC), offering 96GB of in-package HBM2e memory and 24 Tensor processor cores. Habana and Intel have pitched the Gaudi chips’ advantages in low-power, high-speed AI training and inference relative to the competition.
Intel CEO Pat Gelsinger recently demonstrated the Gaudi2 system at the World Economic Forum, showcasing Intel’s heavily-promoted Gaudi-based architecture for dedicated deep learning solutions. Medina explained that Intel recommends the Habana solutions for AI tasks, especially for training large language models (LLMs), since the higher-end Intel GPUs are not yet available.
In March, Habana Labs and Hugging Face benchmarked Gaudi2 for inference on the 176 billion-parameter BLOOMZ LLM, comparing it to a server based on Nvidia’s A100 GPU. The results showed 1.3× faster inference on Gaudi2, with Habana projecting that multiplier to hit 1.8× on FP8 rather than BF16. Further, Gaudi2 used 22% less power in the process, highlighting the value proposition for customers.
Mobileye, an Intel company, has deployed Gaudi2 for production use, applying the chips for training custom object detection models aimed at autonomous vehicle operation. While many of Habana Labs’ deployments are in testbeds, it is quickly emerging as Intel’s top AI offering.
Habana Labs’ Gaudi chips for AI are set to get even better with the upcoming launch of Gaudi3. According to Habana COO Eitan Medina, Gaudi3 is currently in manufacturing and will be the company’s TSMC 5nm product. Medina expects Gaudi3 to offer a significant performance boost and improvements to power efficiency.
While Nvidia and AMD are also launching their next-generation solutions aimed at AI applications, Medina remains optimistic about Habana’s competitiveness. He believes Gaudi2 will be competitive on price performance, with Gaudi3 outperforming it significantly, thanks in part to the big jump to 5nm.
Medina expects Gaudi to remain the recommended solution for heavy AI workloads for the next couple of years. Beyond that, the roadmap is opaque, with planned integration between Habana Labs’ chips and the accelerators emerging from Intel’s Accelerated Computing Systems and Graphics group.
Medina revealed that Intel is already designing the next-generation accelerator, which will combine Gaudi’s capabilities with some of the AXG capabilities. The roadmap between Habana and the AXG side of the organization will be unified, and the company is working on a deeper integration.
Expect more news on Gaudi3 in the coming quarters as Habana Labs continues to lead the charge in AI solutions.
Conlcusion:
Habana Labs’ Gaudi chips for AI are emerging as Intel’s top offering in the increasingly prominent generative AI space. With the upcoming launch of Gaudi3, set to offer significant performance boosts and improvements to power efficiency, Habana is well-positioned to maintain its leadership in the market for heavy AI workloads. As the competition from Nvidia and AMD heats up, Habana’s continued innovation and integration with Intel’s Accelerated Computing Systems and Graphics group will be crucial to staying ahead in the race for AI solutions.
