TL;DR:
- Mistral AI, an open-source model startup, has released its latest LLM, MoE 8x7B, via a torrent link, in contrast to Google’s more traditional Gemini release.
- MoE 8x7B is being compared to a scaled-down GPT-4, with eight experts, each having 111B parameters and 55B shared attention parameters.
- Only two experts are used for each token’s inference in MoE 8x7B.
- This unconventional release strategy generated significant attention and discussion within the AI community.
- Experts praise Mistral AI’s branding strategy, which is gaining recognition in the AI space.
- Mistral AI recently secured a $2 billion valuation and made headlines with a record-setting $118 million seed round.
- They have been actively involved in discussions related to the EU AI Act, advocating for reduced regulation of open-source AI.
Main AI News:
In a move that defies convention, Mistral AI, the open-source model startup, has set the tech world abuzz by releasing its latest Language Model (LLM) with nothing more than a torrent link. This unconventional approach stands in stark contrast to Google’s recent Gemini release, which has been criticized for its polished, rehearsed presentation.
Mistral AI’s new model, named MoE 8x7B, is being dubbed as a “scaled-down GPT-4” by some in the Reddit community. It appears to be a Mixture of Experts (MoE) with 8 experts, each with 111B parameters and 55B shared attention parameters, totaling 166B parameters per model. Notably, for the inference of each token, only 2 experts are used.
Uri Eliabayev, an AI consultant and founder of “Machine & Deep Learning Israel,” noted that Mistral has a reputation for unconventional releases, often without accompanying papers, blogs, code, or press releases. Open-source AI advocate Jay Scambler praised the move as unusual but effective in generating significant attention and discussion within the AI community.
The unconventional release strategy has garnered praise from industry experts. Eric Jang, Vice President of AI at 1X Technologies and former Google robotics research scientist, commented that Mistral’s brand is quickly gaining recognition in the AI space.
Mistral AI, based in Paris, recently secured a $2 billion valuation in a groundbreaking funding round led by Andreessen Horowitz. The company had previously made headlines with a record-setting $118 million seed round, the largest in European history. Their first large language AI model, Mistral 7B, was launched in September, and the company has also been actively involved in discussions surrounding the EU AI Act, advocating for less regulation on open-source AI.
Conclusion:
Mistral AI’s disruptive release approach for MoE 8x7B showcases its commitment to open-source innovation. This move, coupled with their substantial funding and active involvement in regulatory discussions, positions Mistral AI as a significant player in the evolving AI market, challenging traditional release norms and advocating for a more open AI ecosystem.
