TL;DR:
- India utilizes an AI-driven language translation system, Bhashini, to create comprehensive language datasets.
- Villagers in Karnataka contribute native Kannada sentences to develop India’s first AI-based Tuberculosis chatbot.
- India boasts over 121 languages spoken by 10,000 or more people, but NLP predominantly serves English.
- Kalika Bali of Microsoft Research India emphasizes the need for inclusive AI tools for non-English speakers.
- Crowdsourcing speech data is an effective approach for building datasets for AI models in education, healthcare, and more.
- Initiatives like Masakhane in Africa and Jais in the UAE aim to strengthen NLP research in underrepresented languages.
- Ethical considerations and fair compensation are vital for the success of crowdsourcing initiatives.
- AI projects like Project Vaani, EkStep Foundation’s tools, and AI4Bharat’s Jugalbandi chatbot enhance speech recognition in native languages.
- Grassroots organizations like Gram Vaani employ AI chatbots to break down language barriers for welfare-related queries.
Main AI News:
In a quest to usher in a new era of digital accessibility, the Indian government has embarked on a pioneering initiative. Through the innovation of Bhashini, an AI-driven language translation system, India is making strides towards creating comprehensive language datasets. These datasets are not only reshaping how information is disseminated but also catalyzing economic opportunities for hundreds of millions of Indians.
Karnataka, a southwestern state in India, recently became the epicenter of a groundbreaking project. Villagers, speaking the native Kannada language, contributed their voices to an app. Their collective effort is an essential step in the development of the nation’s first AI-based chatbot for Tuberculosis. With over 40 million native Kannada speakers, this endeavor is a testament to India’s linguistic diversity, boasting more than 121 languages spoken by 10,000 or more people.
However, the challenge lies in extending the benefits of natural language processing (NLP) to cover this vast linguistic landscape. NLP, a subset of artificial intelligence, equips computers to comprehend both written and spoken language. Currently, it predominantly caters to English, French, and Spanish speakers, leaving a substantial portion of India’s population excluded from valuable information and economic prospects.
Kalika Bali, principal researcher at Microsoft Research India, emphasizes the need for inclusive AI tools, stating, “For AI tools to work for everyone, they need to also cater to people who don’t speak English or French or Spanish.” However, the task of collecting sufficient data in Indian languages for large language models like GPT seems daunting. Therefore, Bali suggests creating layers atop generative AI models, such as ChatGPT, to bridge this linguistic divide.
Karya, a tech firm, has harnessed the power of thousands of Indian language speakers to generate speech data. This data is then used to construct datasets for technology giants like Microsoft and Google. Simultaneously, the Indian government is actively engaged in building language datasets through Bhashini, an AI-led language translation system. This platform fosters a collaborative approach, allowing people to contribute sentences in various languages, validate transcriptions, translate texts, and label images. Tens of thousands of Indians have already joined this mission.
Pushpak Bhattacharyya, the head of the Computation for Indian Language Technology Lab in Mumbai, highlights the challenges in this endeavor, including the predominance of oral traditions, scarcity of electronic records, and code mixing. Gathering data for less common languages demands special efforts, making it a formidable undertaking.
In the world of natural language processing, only a fraction of the world’s languages are represented. English takes the lead in major NLPs, leaving a multitude of languages underrepresented. Governments and startups worldwide are striving to close this gap. Initiatives like Masakhane in Africa and Jais in the United Arab Emirates aim to strengthen NLP research in underrepresented languages.
Crowdsourcing emerges as an effective method to collect speech and language data, particularly in a country as diverse as India. It not only captures linguistic nuances but also addresses cultural and socio-economic factors. However, ethical considerations, gender, ethnic, and socio-economic bias, and fair compensation for workers are imperative to ensure the success and scalability of these initiatives.
With the rapid advancement of AI, there is a growing demand for languages previously overlooked. Karya, for instance, collaborates with non-profit organizations to empower workers from economically disadvantaged backgrounds. These workers not only earn fair wages but also own a stake in the data they generate, fostering the potential for AI products that benefit their communities.
In a country where only a small fraction of the population speaks English, speech and speech recognition have become paramount. Projects like Google-funded Project Vaani, EkStep Foundation’s AI-based translation tools, and AI4Bharat’s Jugalbandi chatbot are pivotal in making information accessible to millions in their native languages.
Gram Vaani, a social enterprise working with farmers, employs AI-based chatbots to answer welfare-related queries. Such initiatives are breaking down language barriers and making vital information more accessible, especially at the grassroots level.
As India embraces AI to democratize information access, it’s not just about technological advancements but also empowering communities. Swarnalata Nayak from Odisha, who contributes her native Odia language to Karya’s initiative, exemplifies the transformative potential of these projects, as she proudly states, “I can provide for my family through talking on the phone.”
Conclusion:
India’s concerted efforts to harness AI for linguistic inclusivity hold significant promise. By addressing the linguistic diversity of its population, India not only improves access to information but also unlocks economic opportunities. This multilingual revolution has the potential to reshape markets, particularly in sectors such as education, healthcare, and technology, by catering to a more diverse audience and bridging language gaps.
