TL;DR:
- MLCommons unveils new MLPerf benchmark results for Training v3.0 and Tiny v1.1.
- Training v3.0 showcases improved performance in training advanced neural networks and AI model deployment at the edge.
- MLPerf Training round witnesses wide industry participation, highlighting substantial performance gains.
- MLPerf Training adds new benchmarks for large language models and updated recommenders, aligning with industry trends.
- Over 250 performance results were submitted by 16 companies in MLPerf Training v3.0, including notable newcomers.
- MLPerf Tiny benchmark suite emphasizes “tiny” neural networks for low-power devices, with impressive results and power measurements.
- Academic, industry and national lab submissions contribute to the Tiny ML v1.1 benchmarks.
- MLPerf results indicate the accelerated pace of innovation and a diverse range of design choices.
- The advancements in AI performance showcased by MLPerf results present significant opportunities for the market.
Main AI News:
The unveiling of the latest MLPerf™ benchmark suite results by MLCommons has brought forth promising advancements in the realm of artificial intelligence. The two benchmark suites, Training v3.0 and Tiny v1.1, shed light on the remarkable performance of machine learning models and the expedited processing of data by trained neural networks on low-power devices in compact form factors.
The benchmarks demonstrate notable progress in training sophisticated neural networks and deploying AI models at the edge. Furthermore, the recent MLPerf Training round showcases extensive industry participation and highlights impressive performance gains, reaching up to 1.54 times and 33 to 49 times improvement over the initial round.
The open-source and peer-reviewed MLPerf Training benchmark suite encompasses comprehensive system tests that place emphasis on machine learning models, software, and hardware, catering to a wide range of applications.
Notably, two new benchmarks have been incorporated into the MLPerf Training suite. The first benchmark revolves around a large language model (LLM) utilizing the GPT-3 reference model, effectively capturing the growing prevalence of generative AI. The second benchmark features an updated recommender, thoughtfully modified to align with industry practices, employing the DLRM-DCNv2 reference model. These new tests are specifically designed to propel AI advancement by ensuring that industry-standard benchmarks accurately represent the latest trends in adoption, thereby assisting customers, vendors, and researchers in their endeavors.
With over 250 performance results, a 62% increase from the previous round, the MLPerf Training v3.0 round has attracted submissions from 16 distinct contributors: ASUSTek, Azure, Dell, Fujitsu, GIGABYTE, H3C, IEI, Intel & Habana Labs, Krai, Lenovo, NVIDIA, NVIDIA + CoreWeave, Quanta Cloud Technology, Supermicro, and xFusion. MLCommons extends its congratulations to CoreWeave, IEI, and Quanta Cloud Technology, who have made their first MLPerf Training submissions.
The MLPerf Tiny benchmark suite, dedicated to collecting various inference use cases involving “tiny” neural networks typically under 100 kB, showcases the ability to process sensor data, such as audio and vision, to provide endpoint intelligence for low-power devices in compact form factors. In addition to evaluating these capabilities, MLPerf Tiny also offers optional power measurement.
Within the latest round, the Tiny ML v1.1 benchmarks comprise 10 submissions from academic institutions, industry organizations, and national labs, generating an impressive 159 peer-reviewed results. Submitters include Bosch, cTuning, fpgaConvNet, Kai Jiang, Krai, Nuvoton, Plumerai, Skymizer, STMicroelectronics, and Syntiant. This round also encompasses 41 power measurements. MLCommons commends Bosch, cTuning, fpgaConvNet, Kai Jiang, Krai, Nuvoton, and Skymizer for their inaugural contributions to MLPerf Tiny.
Dr. Csaba Kiraly, co-chair of the MLPerf Tiny Working Group, commented on the comprehensive array of hardware solutions and innovative software frameworks covered by the benchmark suite. “The v1.1 release includes submissions ranging from tiny and inexpensive microcontrollers to larger FPGAs, showing a large variety of design choices,” Dr. Kiraly remarked. He further emphasized that the combined effect of software and hardware performance improvements can yield staggering advancements of up to 1000-fold in certain areas, compared to the initial reference benchmark results. These findings unequivocally reflect the rapid pace of innovation unfolding within the field.
Conclusion:
The latest MLPerf benchmark results reveal substantial progress in AI performance, particularly in training advanced neural networks and deploying AI models at the edge. The increased industry participation and notable performance gains signify a thriving market with a wide variety of hardware solutions and innovative software frameworks. The incorporation of new benchmarks reflects the industry’s adoption of generative AI and the alignment of recommender models with industry practices. These results not only provide valuable insights for customers, vendors, and researchers but also drive further advancements in machine learning and AI technologies. The market can anticipate accelerated innovation and increased opportunities as businesses harness the potential of artificial intelligence.
