How to get AI to work in its 22 languages

Vineet Sawant has spent the last two years riding the streets of Mumbai on a scooter as a delivery driver.

“Being on the road is always very stressful, especially in cities like Mumbai,” he says.

But when he first started, the language barrier was an additional challenge.

His mother tongue is Marathi and Mr Sawant speaks “very little” English. “I understand, but it’s very difficult to read,” he explains.

This caused problems at his new job.

He said: “It was difficult at first. Everything was in English and I can understand some things, but I feel more comfortable speaking in Marathi. I used to ask other couriers to help me figure out what to do.”

His employer Zepto promises “India's fastest online grocery delivery.” Therefore, having drivers have problems with delivery instructions was not ideal.

To make this process easier, Zepto partnered with Reverie Language Technologies a year ago to introduce a synthetic translation service for its drivers.

Since then, delivery drivers can choose between six languages ​​in the Zepto app.

“I don't have to guess anymore,” says Mr. Sawant.

“Before, it took me longer to read, and sometimes I even made mistakes. Now, if a client writes “call”, I get this instruction in Marathi. So I don't have to ask or check again. Everything is clear.”

Mr. Sawant's difficulties are common.

“India has 22 official languages ​​and hundreds of dialects,” says Professor Pushpak Bhattacharya of IIT Mumbai, one of India’s leading experts on the use of AI in Indian languages.

“Without technology that understands and speaks these languages, millions of people are excluded from the digital revolution, especially in education, government, healthcare and banking,” he says.

The introduction of new generative artificial intelligence systems such as ChatGPT has made this task more urgent.

AI uses huge amounts of data to train, such as web pages, books, or video transcripts.

In widely spoken languages ​​such as Hindi and English this is relatively easy to obtain, but in others it is more difficult.

“The main challenge in creating Indian language models is data availability. I'm talking about updated data. Coarse quality data available. But this data is not of very high quality and needs to be filtered,” says Professor Bhattacharya.

“The problem in India is that for many Indian languages, especially tribal and regional dialects, this data simply does not exist or is not digitized.”

Reverie Language Technologies is currently deploying its AI-powered translation technology to a number of Indian companies.

Co-founder Vivekananda Pani says that while translation technology will make communication easier, there is “the potential that less common dialects will be crowded out.”

“The challenge will be to ensure that the surprising benefits of artificial intelligence-driven language advances do not inadvertently reduce the rich diversity of human language.”

To help solve Professor Bhattacharya's problem contributed by Bhashinia government project to develop high-quality data sets needed for AI training.

In addition to data sets, Bhashini has created artificial intelligence language models and translation services in 22 languages.

This is a huge undertaking that started in 2022 but has already made great progress.

Bhasini currently houses 350 AI-powered language models that have processed over a billion tasks.

More than 50 government departments as well as 25 state governments are collaborating with Bhashini.

For example, Bhashini's technology is used in multilingual chatbots for government services and to translate government schemes into local languages.

“Bhashini ensures linguistic and cultural representation of India by building India-specific AI models rather than relying on global platforms,” says Amitabh Nag, CEO, Digital India, a unit of Bhashini.

He hopes that within the next two to three years, rural users will have voice access to government services, financial instruments and information systems in their native languages.

We hope that these India-specific datasets will one day give people developing AI-based models the tools to make them much easier to adapt to the entire population.

Currently, developing any artificial intelligence program to handle complex processes such as healthcare can be extremely challenging.

Kshitij Jadhav, assistant professor at the Coit Center for Digital Health at IIT Mumbai, is working on an artificial intelligence program to help people quit smoking.

He explains that people at different stages of the process need different advice, and they usually need a well-trained person to carry out such an assessment.

But there are a limited number of practitioners who can help, especially those who can work in multiple languages, so Professor Jadhav hopes his artificial intelligence model can bridge the gap.

AI “will first determine what kind of conversation a person needs and will formulate questions, show empathy, emotions accordingly,” says Professor Jadhav.

And all of this will hopefully eventually be done in 22 languages. Initial experiments are being conducted in English and Hindi.

“It will be very personalized rather than something off the shelf,” he says.

Back on the city streets, Vineet Sawant has increased the number of packages he delivers from ten to about 30 per day, helped in part by the delivery app's translation feature.

He thinks it will help more people like him.

“It makes us feel like we belong. Not everyone understands English. When an app speaks our language, we feel more confident and perform better.”