Digital Economy Blog - “Crack the voice” of poor quality French versions thanks to automated dubbing

2023-11-22 15:10:15

The month of September 2023 marked the end of summer and the beginning of dark hours for… the dubbing industry. Since mid-September, numerous videos of famous personalities conversing easily in several foreign languages have been circulating on the platforms X (new name for Twitter) and Tik Tok. No, the French did not suddenly become good at English, Italian or even Japanese, but artificial intelligence (AI) always has been.

In 2015, 70% of French people^[1] declared that they preferred to see a film in the French version rather than in the original version. This French version only existed until then thanks to dubbing authors and actors specialized in this field. But since September, the start-up Heygen has offered its users the possibility of doing this work with the help of artificial intelligence. Several unions and professional associations have raised their voices against this new use, which threatens their profession.

Heygen Ai, bad scapegoat of the press?

In recent days the press has highlighted this innovative startup a lot, as if it were the sworn enemy of the dubbing industry. By looking at the Heygen Ai site we realize that the possibility of directly dubbing a person in another language is just one of the many features offered by the platform. The start-up offers to support the user in their professional presentations. Users can also make presentation videos without having to speak in front of the screen and learn the presentation thanks to an avatar in their likeness. Finally, the Heygen Ai platform only supports up to 500MB of video, therefore videos of less than 10 minutes.

Even though Heygen has been in the news since mid-September, she probably won’t have an impact on the dubbing industry. However, some companies have been offering to translate films using AI since 2020.

©HeyGen homepage

The fear of the great replacement, unfounded or justified fear?

In 2021, the Israeli start-up Deephub allowed South American and Mexican audiences to see the horror film Everytime I die dubbed in their language using AI. But when we look a little closer at the use of AI in the world of cinema, we realize that this is not the first use of this technology. In 2020, Disney used similar processes in the series The Mandalorian by taking recordings of Mark Hamill (player of the character Luke Skywalker) when he was 40 years old. This allowed them to have the character appear as a young Jedi.

It is true that using AI helps reduce costs because a day of dubbing costs between €250 and €300. But talk of a major replacement may seem excessive at the moment. It is more difficult to convey emotion in voice with artificial intelligence. When we compare the original undubbed version and the one dubbed by an AI like Heygen’s, we note that certain details are erased: an actor chewing gum no longer seems to have any in the ‘artificial’ version. The AI translates for the moment without any particular emotion.

The debate on the disappearance of the profession of voice actor is very similar to the debate which emerged with the appearance of digital books. However, in 2022, 51.4%^[1] of book sales were for printed books compared to 7.8% for digital books. This trend is similar in many countries such as the United States (44.6% sales of paper books and 23.4% eBooks). We tend to forget that humans still like to have human and physical contact. A dubbing actor and a dubbing author will be able to grasp the subtleties of a language, make puns specific to it or make references to the popular culture of a country unlike AI which cannot, for the moment, what to propose a literal and mechanical translation. We must not forget that the dubbing author must not simply translate a text, but adapt it. At the moment the AI is not capable of doing such a thing.

SOURCES :

About Venice CORNET

Registered for the examination of complementary courses in Luxembourg law (CCDL) and student in Master 2 in Digital Economy Law at the University of Strasbourg, I am determined to implement my expertise in digital law and intellectual property within innovative legal teams.