TL;DR:
- SambaNova unveils SN40L chip for its full-stack AI platform.
- The SN40L supports 5 trillion-parameter models with 256k+ sequence lengths.
- It offers dense and sparse compute alongside large and fast memory.
- This breakthrough enables higher-quality models, faster inference, and lower ownership costs.
- SambaNova aims to rival GPT4 with its full-stack LLM platform.
- The chip addresses challenges in AI implementation, including cost and performance.
- SambaNova’s SN40L chip stands out with its unique memory capabilities.
- The SambaNova Suite includes advanced language models and multimodal capabilities.
- The company focuses on providing reliable access to open-source models and systems.
Main AI News:
In a groundbreaking move, SambaNova Systems, renowned for its purpose-built, end-to-end AI platform, introduces its latest innovation: the SN40L chip. This remarkable piece of technology will be the driving force behind SambaNova’s complete large language model (LLM) platform, the SambaNova Suite. Inside this revolutionary chip lies a unique combination of dense and sparse computing, alongside expansive and high-speed memory, rendering it a true “intelligent chip.”
Manufactured by TSMC, SambaNova’s SN40L boasts the capability to support a 5 trillion-parameter model, with a sequence length of 256k+ achievable on a single system node. Such a feat is only attainable through an integrated stack and signifies a monumental advancement beyond previous state-of-the-art chips. This leap in technology enables the development of higher-quality models, delivering swifter inference and training, all while maintaining a lower total cost of ownership.
Rodrigo Liang, co-founder and CEO of SambaNova Systems, affirms, “Today, SambaNova offers the only purpose-built full stack LLM platform—the SambaNova Suite—now with an intelligent AI chip; it’s a game changer for the Global 2000. We’re now able to offer these two capabilities within one chip—the ability to address more memory, with the smartest compute core—enabling organizations to capitalize on the promise of pervasive AI, with their own LLMs to rival GPT4 and beyond.“
This innovative chip is merely one facet of SambaNova’s all-encompassing LLM platform, engineered to resolve the most significant challenges faced by enterprises in deploying generative AI. Kunle Olukotun, co-founder of SambaNova Systems, points out, “We’ve started to see a trend towards smaller models, but bigger is still better, and bigger models will start to become more modular. Customers are requesting an LLM with the power of a trillion-parameter model like GPT-4, but they also want the benefits of owning a model fine-tuned on their data. With the new SN40L, our most advanced AI chip to date, integrated into a full stack LLM platform, we’re giving customers the key to running the largest LLMs with higher performance for training and inference, without sacrificing model accuracy.”
Introducing the SN40L, the Engine behind the SambaNova Suite:
- SambaNova’s SN40L can support a 5 trillion-parameter model, with a sequence length of 256k+, leading to superior model quality, quicker inference and training, all while reducing the total cost of ownership.
- Expansive memory capabilities unlock the true potential of multimodal applications for LLMs, empowering companies to effortlessly search, analyze, and generate data in various modalities.
- Enhanced efficiency in running LLM inference translates into a lower total cost of ownership for AI models.
Drawing upon six years of engineering expertise and customer insights, SambaNova’s team meticulously addressed the challenges associated with AI implementation: training and inference costs, sequence length limitations, and LLM speed (latency). The result is a fully modular and extensible LLM platform that empowers customers to incrementally incorporate new modalities, expand their expertise, and increase the model’s parameter count (up to 5T) without compromising inference performance.
Peter Rutten, Research Vice-President, Performance-Intensive Computing at IDC, underscores the uniqueness of SambaNova’s SN40L chip, stating, “SambaNova’s SN40L chip is unique. It addresses both HBM (High Bandwidth Memory) and DRAM from a single chip, enabling AI algorithms to choose the most appropriate memory for the task at hand, giving them direct access to far larger amounts of memory than can be achieved otherwise. Plus, by using SambaNova’s RDU (Reconfigurable Data Unit) architecture, the chips are designed to efficiently run sparse models using smarter compute.”
New Models and Features within the SambaNova Suite:
- Llama2 variants (7B, 70B): Cutting-edge open-source language models allow customers to adapt, expand, and deploy the best LLM models while retaining ownership.
- BLOOM 176B: The most precise multilingual foundation model in the open-source community, enabling customers to tackle a wide array of language-related challenges and extend support to low-resource languages.
- A new embedding model for vector-based retrieval augmented generation, enabling customers to embed documents into vector embeddings for retrieval during the Q&A process without generating hallucinations. The LLM then analyzes, extracts, or summarizes the information.
- A world-leading automated speech recognition model for transcribing and analyzing voice data.
- Additional multimodal and long sequence length capabilities.
- Inference-optimized systems with 3-tier Dataflow memory for uncompromised high bandwidth and capacity.
GV General Partner and SambaNova Series A Board Director Dave Munichiello emphasizes, “Recent breakthroughs have highlighted the potential for AI to rapidly improve both our lives and our businesses. While the AI hype cycle drove curiosity for AI’s potential, the Fortune 1000 is seeking more than a new capability—they are asking for predictability, dependability, and enterprise-grade availability.” SambaNova’s latest announcement underscores their commitment to delivering reliable access to top-tier open-source models on cutting-edge systems, granting customers control over their models, data, and computational resources.
Conclusion:
SambaNova’s SN40L chip signifies a pivotal advancement in the AI market. It empowers organizations to harness the potential of large language models, offering superior performance, greater efficiency, and lower costs. SambaNova’s commitment to modularity and extensibility positions them at the forefront of AI technology, promising predictability, dependability, and enterprise-grade availability for businesses in need of advanced AI solutions.
