TL;DR:
- Sarvam AI introduces OpenHathi-Hi-v0.1, the first Hindi large language model.
- OpenHathi rivals GPT-3.5 in Indic language proficiency while maintaining strong English capabilities.
- The model undergoes a meticulous two-phase training process, focusing on embedding alignment and cross-lingual attention.
- Performance evaluations show OpenHathi outperforms GPT-3.5 in both native and Romanized Hindi.
- Developed in collaboration with AI4Bharat and fine-tuned with KissanAI’s conversational data.
- KissanAI launches Dhenu 1.0, a specialized Agriculture Large Language Model for Indian farmers.
- Sarvam AI secures $41 million in Series A funding led by Lightspeed, with Peak XV Partners and Khosla Ventures.
Main AI News:
In a groundbreaking move, Sarvam AI, the Indian AI startup, has unveiled OpenHathi-Hi-v0.1, marking a significant milestone as the inaugural entry in the OpenHathi series of large language models. This cutting-edge model, developed on a budget-friendly platform, is an extension of the formidable Llama2-7B and proudly delivers performance akin to GPT-3.5 for Indic languages.
OpenHathi takes center stage with its impressive 48,000-token expansion of Llama2-7B’s tokenizer. Its development encompasses a meticulous two-phase training process. The initial phase is dedicated to embedding alignment, a process that strategically aligns randomly initialized Hindi embeddings. This is followed by bilingual language modeling, which educates the model on cross-lingual attention across tokens.
The result is a model that showcases remarkable proficiency across a wide range of Hindi language tasks, effectively rivaling, if not surpassing, the capabilities of GPT-3.5, all while maintaining a high level of English proficiency. Sarvam AI’s rigorous evaluation criteria encompass not only standard Natural Language Generation (NLG) tasks but also real-world, non-academic challenges. These evaluations, pitting OpenHathi against GPT-3.5 with GPT-4 as the arbiter, have consistently revealed OpenHathi’s superior performance in Hindi, both in its native script and Romanized variations.
This remarkable achievement has been made possible through a collaborative effort. Sarvam AI joined forces with academic partners at AI4Bharat, who contributed invaluable language resources and benchmarking expertise. Additionally, fine-tuning of the model was carried out in partnership with KissanAI, leveraging conversational data generated by a bot that interacts with farmers in multiple languages.
Notably, KissanAI recently unveiled Dhenu 1.0, a revolutionary Agriculture Large Language Model tailor-made to meet the specific needs of Indian agricultural practices. This bilingual marvel comprehends queries in English, Hindi, and Hinglish, effectively catering to the linguistic requirements of farmers across the nation.
The brains behind Sarvam AI, Pratyush Kumar and Vivek Raghavan, embarked on this journey in July 2023. They brought their vision to life with substantial backing, securing an impressive $41 million in Series A funding. This round of investment was led by Lightspeed, with active participation from Peak XV Partners and Khosla Ventures.
The foundation of Sarvam AI rests upon the rich backgrounds of its co-founders, who are rooted in AI research and digital infrastructure development. The startup’s mission is clear: to address India’s unique requirements by prioritizing the integration of Generative AI across diverse Indian languages and fostering strategic collaborations with enterprises to develop domain-specific AI models using their valuable data.
Conclusion:
Sarvam AI’s OpenHathi-Hi-v0.1 represents a significant leap forward in the field of language models, catering to the unique linguistic needs of the Indian market. Its superior performance in Hindi, combined with English proficiency, positions it as a game-changer for various industries, especially those seeking to tap into India’s diverse linguistic landscape. The collaboration with AI4Bharat and KissanAI further solidifies Sarvam AI’s commitment to delivering tailored solutions for the Indian market, while the substantial funding underscores the growing interest and investment in AI innovations within India.
