- NVIDIA collaborates with Microsoft to enhance Phi-3 Mini language models.
- Integration of NVIDIA TensorRT-LLM optimizes model inference on NVIDIA GPUs.
- Phi-3 Mini offers ten times larger models than its predecessor, Phi-2.
- Workstations and PCs equipped with NVIDIA GPUs can efficiently execute the model.
- Phi-3 Mini introduces two variants catering to different token capacities.
- Integration extends beyond traditional computing, benefiting autonomous robotics and embedded systems.
- NVIDIA’s commitment to open systems includes contributions to the open-source ecosystem.
- TensorRT-LLM optimizations enhance inference throughput and reduce latency.
Main AI News:
In a strategic move, NVIDIA is bolstering Microsoft’s Phi-3 Mini open language model by seamlessly integrating NVIDIA TensorRT-LLM. This collaborative effort aims to refine the efficiency of large language model inference, particularly when leveraging NVIDIA GPUs, extending from personal computing devices to expansive cloud servers.
The Phi-3 Mini represents a significant leap forward, offering the capacity for models that are ten times larger compared to its predecessor, the Phi-2. While Phi-2 was predominantly confined to research applications, the new iteration empowers workstations armed with NVIDIA RTX GPUs and PCs featuring GeForce RTX GPUs to efficiently execute the model using either Windows DirectML or TensorRT-LLM.
Sporting an impressive 3.8 billion parameters and trained on a colossal dataset of 3.3 trillion tokens in a mere seven days across 512 NVIDIA H100 Tensor Core GPUs, Phi-3 Mini introduces two variants tailored to diverse requirements. These variants, accommodating either 4k or 128k tokens, set a groundbreaking standard for processing very lengthy contexts.
Beyond conventional computing realms, the integration of Phi-3 Mini into the NVIDIA ecosystem heralds a new era. Developers immersed in projects revolving around autonomous robotics and embedded systems can tap into a plethora of community-driven tutorials, such as those available on Jetson AI Lab, to harness the transformative potential of generative AI. Furthermore, the compact design of Phi-3 Mini, housing 3.8 billion parameters, renders it exceptionally suitable for edge devices, delivering efficiency without compromising on performance.
The tailored TensorRT-LLM, designed to support Phi-3 Mini’s extended context window, incorporates a myriad of optimizations and kernels, including LongRoPE, FP8, and inflight batching. These enhancements significantly boost inference throughput while simultaneously reducing latency. NVIDIA’s plan to make these implementations accessible via the examples folder on GitHub ensures a seamless transition for developers to the TensorRT-LLM checkpoint format optimized for inference.
Underscoring its steadfast commitment to open systems, NVIDIA reiterates its active engagement in the open-source ecosystem, actively contributing to various projects and collaborating with esteemed foundations and standards bodies. With a portfolio boasting over 500 projects released under open-source licenses, NVIDIA continues to spearhead the advancement of open technologies and standards, fostering innovation and collaboration across the industry.
Conclusion:
NVIDIA’s partnership with Microsoft to enhance Phi-3 Mini language models signifies a significant advancement in the field of large language model inference. By integrating TensorRT-LLM and extending support to NVIDIA GPUs, the collaboration not only improves efficiency but also opens doors for diverse applications across industries. This development underscores NVIDIA’s dedication to fostering innovation in open technologies, ultimately driving progress and collaboration in the market.
