TL;DR:
- Amazon is developing custom microchips, Inferentia and Trainium, for generative AI, offering an alternative to Nvidia GPUs.
- Amazon’s CEO emphasizes their ability to meet the growing demand for generative AI chips.
- Microsoft and Google have made significant investments in generative AI, gaining attention with products like ChatGPT and Bard.
- Amazon entered the game later with its large language models, Titan, and Bedrock service to enhance software with generative AI.
- Experts see Amazon playing catch-up in a market it’s not accustomed to.
- Amazon’s custom silicon, Nitro and Graviton, could provide an edge in the long run.
- AWS dominates the cloud market, which can be leveraged for generative AI expansion.
- AWS offers various developer tools for generative AI, catering to a wide range of industries.
- AWS CEO’s vision includes a $100 million generative AI innovation center.
- AWS customers’ familiarity with Amazon’s ecosystem gives it an advantage in the generative AI race.
Main AI News:
Inside an inconspicuous office complex nestled in Austin, Texas, a duo of modest chambers harbors a cadre of Amazon personnel, diligently crafting two variants of microchips tailored to the exigencies of training and expediting generative AI. These unique chips, christened Inferentia and Trainium, proffer an alternative avenue for AWS clientele to groom their expansive language models, circumventing the mounting hurdles and financial strain associated with procuring Nvidia GPUs.
In a June interview with CNBC, Amazon Web Services CEO Adam Selipsky ardently conveyed, “The world at large hungers for increased computational prowess to propel generative AI endeavors—be it GPUs or Amazon’s proprietary chips, the ones we’re meticulously devising.” He affirms Amazon’s unrivaled capacity to sate the collective craving of its customers.
Notwithstanding, swifter actors who have injected greater resources into capitalizing on the generative AI surge have emerged. Microsoft’s conspicuous prominence was bolstered when it launched ChatGPT in November, followed by an eye-catching $13 billion investment in OpenAI. This seismic move cemented its position as a frontrunner. Microsoft adroitly folded the generative AI models into its repertoire, seamlessly integrating them with Bing by February. In a parallel trajectory, Google unveiled its mammoth language model, Bard, and concurrently backed its ambitions with a staggering $300 million investment in OpenAI’s rival, Anthropic.
Not until April did Amazon throw its gauntlet into the ring, introducing its suite of expansive language models under the moniker “Titan.” Accompanying this paradigm shift was “Bedrock,” a service tailored to amplify software development through generative AI. However, an undercurrent of urgency prevails. “Amazon, known for trailblazing rather than trailing, finds itself in an unfamiliar position, striving to bridge the gap,” remarked Chirag Dekate, Gartner’s VP Analyst, underscoring Amazon’s atypical position.
Meta recently unleashed its proprietary LLM, christened Llama 2, capturing attention. This open-source adversary to ChatGPT is now available for testing on Microsoft’s Azure public cloud.
Emerging as a Definitive Distinctive Element
In the grand tapestry of time, Dekate believes Amazon’s bespoke silicon might ultimately bestow a distinct edge in the realm of generative AI. “The true distinction is the remarkable technical dexterity they channel,” he pronounced. “Microsoft, I emphasize, is devoid of Trainium or Inferentia,” he articulated resolutely.
The roots of AWS’ custom silicon endeavor trace back to 2013 when the seeds of innovation were sown through a specialized hardware piece christened “Nitro.” This humble beginning burgeoned into an expansive presence, with at least one Nitro chip inhabiting every AWS server—a staggering count of over 20 million.
The year 2015 marked Amazon’s acquisition of Annapurna Labs, an Israeli chip startup, further solidifying its ambitions. In 2018, the formidable Graviton, an Arm-based server chip, emerged as Amazon’s riposte to industry behemoths like AMD and Intel in the domain of x86 CPUs.
In 2018, Amazon unfurled its AI-centric chips, entering the fray two years post Google’s inaugural unveiling of the Tensor Processor Unit, better known as the TPU. While Microsoft’s collaborative venture with AMD—the Athena AI chip—lurks in the shadows, awaiting its revelation.
CNBC was granted exclusive access to Amazon’s chip research hub in Austin, Texas, where Trainium and Inferentia gestated and matured. Matt Wood, the VP of Product, expounded on their respective roles, elucidating, “Machine learning unfurls in two distinct stages: training the models and subsequently, inferring against the trained models. Trainium triumphs by delivering an approximately 50% boost in price-to-performance ratio relative to alternative methodologies for training machine learning models on AWS.”
Debuting in 2021, Trainium took its position on the stage, trailing the 2019 advent of Inferentia, now on the cusp of its second iteration.
Inferentia, as Wood explains, furnishes customers with “unparalleled cost-efficiency, high-throughput, and low-latency machine learning inference.” This precisely encapsulates the predictions emanating from generative AI models—a quintessential feature that empowers users with timely responses.
Nonetheless, Nvidia’s GPUs continue to reign supreme in the realm of training models. AWS unveiled novel AI acceleration hardware fortified by Nvidia H100s in July, underscoring Nvidia’s prevailing dominance.
Leveraging Cloud Dominance for Strategic Leverage
Amazon’s cloud dominion, however, proffers a formidable differentiator. Chirag Dekate expounds, “Amazon’s triumph doesn’t hinge upon making headlines. Their preexisting robust cloud infrastructure holds the key. Enabling extant patrons to embark upon value creation expeditions utilizing generative AI—such is the crux.”
When contemplating Amazon, Google, and Microsoft as the triad vying for generative AI supremacy, an astronomical pool of AWS clients emerges. Their existing rapport with Amazon’s ecosystem could inexorably tip the scales in its favor. Mai-Lan Tomsen Bukovec, AWS’ VP of Technology, elucidates, “Velocity is of the essence. These corporations’ agility in fostering generative AI applications rests upon harnessing the power of AWS’s data repository, coupled with the computational and machine learning tools at their disposal.“
Having clinched a resounding 40% of the cloud computing market in 2022, AWS stands as the paramount provider. While operating income has encountered a trifecta of consecutive quarterly slumps, AWS retains its supremacy, singlehandedly accounting for 70% of Amazon’s staggering $7.7 billion operating profit in Q2. The historical expanse of AWS’ operating margins far eclipses those of Google Cloud.
A Panoply of Developer Tools Steers the Course
AWS boasts an expanding array of developer tools that hone in on generative AI. Swami Sivasubramanian, AWS’ VP of Database, Analytics, and Machine Learning, affirms, “Anticipating the emergence of ChatGPT was a preemptive endeavor. Rapidly devising a chip or even conceptualizing the Bedrock service in a mere two to three months is implausible.”
Bedrock ushers AWS patrons into a realm where the models created by Anthropic, Stability AI, AI21 Labs, and Amazon’s own Titan hold sway. Sivasubramanian elucidates the rationale: “A single model isn’t ordained to reign supreme. Our clientele deserves a pantheon of avant-garde models, meticulously curated from various providers—selecting the most apt tool for each task.“
AWS’ repertoire extends to the likes of AWS HealthScribe, unveiled in July, enabling physicians to craft patient visit summaries through generative AI. SageMaker, an enclave dedicated to machine learning, bequeaths an arsenal of algorithms and models. The coding companion, CodeWhisperer, enables developers to wrap up tasks 57% more expeditiously, a testament to Amazon’s innovation. Not to be outdone, Microsoft reported productivity leaps via its own coding companion, GitHub Copilot.
June bore witness to AWS’ announcement of a $100 million generative AI innovation hub—a testament to its unrelenting commitment. AWS CEO Selipsky heralded, “Numerous patrons express a fervent desire to delve into generative AI, yet often remain baffled by its implications within their unique business contexts. Our arsenal of solutions architects, engineers, strategists, and data scientists is poised to offer personalized guidance.”
Amazon’s Horizon: Expansive Vistas in Generative AI
While thus far, AWS has leaned predominantly towards provisioning tools instead of direct rivalry with ChatGPT, internal communications have unveiled Amazon CEO Andy Jassy’s orchestration of a centralized unit dedicated to sculpting sprawling large language models.
During the second-quarter earnings call, Jassy divulged that AI now fuels a “significant chunk” of AWS’ operations, substantiated by over 20 machine learning services. This augmentation has led to transformative strides for companies like Philips, 3M, Old Mutual, and HSBC.
The flourishing AI landscape, however, has kindled apprehensions about proprietary data security. Concerns ripple through corporate corridors, wary of confidential information infiltrating the training data underpinning public large language models. Adam Selipsky assuages these anxieties, elucidating, “Numerous Fortune 500 corporations have barred ChatGPT for this very reason. Our approach to generative AI, coupled with Bedrock, ensures that anything conducted through Bedrock’s aegis remains enshrined within an isolated virtual private cloud environment, safeguarded by impenetrable encryption and AWS’ meticulous access controls.”
Presently, Amazon steers a relentless trajectory into the realm of generative AI, with “over 100,000” customers leveraging machine learning on AWS—an impressive feat, albeit a fraction of AWS’ sprawling clientele. Industry pundits posit that this equilibrium could tilt, rendering Amazon’s dominion more expansive. Chirag Dekate underscores this sentiment, affirming, “Enterprises aren’t abruptly swayed to shift infrastructure strategies toward Microsoft’s generative AI dominance. For loyal Amazon patrons, the exploratory voyage within Amazon’s ecosystem beckons.”
An Amazon employee works on custom AI chips, in a jacket branded with AWS’ chip Inferentia, at the AWS chip lab in Austin, Texas, on July 25, 2023. Source: Katie Tarasov
Conclusion:
Amazon is aggressively pursuing its ambitions in the generative AI market by developing custom chips and tools. Despite facing competition from early movers like Microsoft and Google, Amazon’s extensive cloud dominance and commitment to innovation position it as a formidable contender in the evolving landscape of AI and cloud computing.
