What if meeting Dalida, Jean Gabin or Lady Di was possible in 2022? We would dream of it, Thierry Ardisson has (almost) done it. In Time Hotel, the first edition of which will be broadcast on Monday May 2 on France 3, the star presenter interviews deceased actors, singers and politicians. How is it possible ? By using one of the applications of artificial intelligence: deepfakes. More precisely, the presenter and his team relied on the technologies of two French companies, Ircam Amplify and Mac Guff, to recreate, with one, the voices of the stars who disappeared, and with the other, their faces.
Behind the algorithmic experiment, two French companies
“One of our areas of work is voice in the broad sense”, explains Nicolas Pingnelain, commercial director of Ircam Amplify. Created in 2020, the company aims to economically promote the fundamental research carried out at the Institute for Acoustic/Music Research and Coordination (Ircam). “Upstream, this means detecting emotions, markers of stress or anxiety, downstream, synthesizing a voice capable of reproducing this type of variation. The human being is even more sensitive to the voice than to the image, he continues, a trifle allows us to detect if it is false. No wonder Thierry Ardisson’s team turned to French researchers: a monotonous voice like that of our intelligent assistants to take on Dalida’s intonations would never have worked.
For the faces too, it was necessary to create the illusion of reality. The team turned to Mac Guff, a 35-year-old French studio, creator, among others, of Me, ugly and mean. “We have always had a research and development dimension, which allows us to stay at the forefront, says its president Rodolphe Chabrier. We were thus among the first to make morphingof relief…” Five years ago, the entrepreneur embarked on artificial intelligence. Mac Guff soon built two tools: the Talking Picture, to make still images “talk”, and the Face Engine, dedicated to recreating faces in 3D. Both built from deep learning models, these technologies made it possible to model the faces of stars of the past on the bodies of the actresses and actors who gave the reply to Thierry Ardisson during the filming of the docu-fiction.
How do we raise the dead?
Because Thierry Ardisson is keen on it: Time Hotel is documentary work. And it’s true: to train the algorithmic machines that reconstruct faces and voices, you have to provide them with equipment. These training datasets were made up of every possible archive on each of the celebrities the show brings to life. “Television appearances, images given by families, films in the case of actors like Jean Gabin, enumerates Rodolphe Chabrier… We take everything we find! ” For the voice, it’s a little simpler, explains Nicolas Pingnelain: ” The important thing is to have samples of all the phonemes, all the syllables that exist in the language. Afterwards, with a few tens of minutes of speech, we can already have a good result. “As for the words spoken by Dalida, Lady Di or Coluche in the show, underlines Thierry Ardisson, all were really said during the lifetime of the celebrities.
The result works quite well… But all the same, we can’t help but feel uneasy. Could this be the effect of strange valley, which makes robots trying to imitate humans a bit unpleasant to the person watching them? Is it the fear of a certain smoothing of the faces, when one lingers on the nose a little too fine of the artificial Dalida, on the photos of shootings? “We had Dalida’s voice validated by her brother Orlando barely two weeks ago and he himself cannot tell the difference between the synthesized version and the real one”, assures Nicolas Pingnelain. “When they saw the result, the Gabin sons found their father’s expressions, they were amazed! adds Rodolphe Chabrier. The production also made the decision to “magnify” the stars, he specifies, “to look for the ‘iconic’ Dalida, as Thierry says”. In this case, the uneasiness may come from ethical questions: is it acceptable to bring dead people back to life? Response from everyone who worked on hotel of time : not only is no law prohibiting it, but the rights holders have reviewed everything, the scripts, the faces, the voices, and everything has been validated.
Innovation Hotel?
“A tool has no ideology, emphasizes Thierry Ardisson, this point is very important in the face of the a priori that may exist”. Rodolphe Chabrier delivers the same message and we can’t help but smile: in tech, the argument of neutrality is a classic. In the world of artificial intelligence, more and more researchers and activists estimate however, posing the problem in terms of good or bad use is not always relevant: in some cases, it would simply be necessary to refrain from developing a tool. For hotel of timethat said, it is clear that Thierry Ardisson follows an idea that has obsessed him for a long time: in 1994, he “resurrected” John Lennon in Gone with time. In 2002, it is Victor Hugo that he invited on the set ofWe will have seen it all, alongside Renaud and Guy Bedos, alive and well. Replacing lookalikes with deepfakes, maybe it’s just keeping up with the times?
This would be to miss all the dimension of innovation that the creation of the program required: Rodolphe Chabrier like Nicolas Pingnelain underline the innovative aspect of the program and the work that it required. “60 minutes of recomposed faces over 90 minutes of program, that has never been done before! exclaims the leader of Mac Guff, if only because the technologies accessible before the company built its own algorithmic models would have made the operation much too expensive. For voice synthesis, “our researchers modified the tool less than a month ago, and they will probably do it again after the release ofhotel of time says Nicolas Pingnelain: the show used applications directly from basic research. Each scientific advance will improve the programs that have been built for the occasion.
A double cultural opportunity
In fact, maybehotel of time opens up two cultural axes at once: the ability to get to know the interviewed stars better, first. After all, Thierry Ardisson’s stated goal is to “spectacularise culture and make knowledge captivating”. It is also the opportunity to see what a still young technology can give in the field of visual and sound special effects. Deep fakes have already been used in series – by Mac Guff, for rejuvenate Mathieu Amalric in The office of legendsor by the Russian operator Megafon, to play Bruce Willis in an advertisement without him really playing.
Difficult to predict if hotel of time will itself mark a turning point. On the other hand, working on this specific case enabled the Mac Guff developers and the Ircam researchers to make progress in their respective fields. And among the applications they imagine for their new algorithmic tools, Rodolphe Chabrier and Nicolas Pingnelain are sketching out avenues as varied as voice authentication (to finally abandon passwords), dubbing (no more French voices not corresponding to American actors?) or, more classically, the retouching of faces and then of bodies. “With the Face Engine, you can recreate, rejuvenate, age, grow, transform someone into a burn victim in a very realistic way, for the cinema or video games”, illustrates Rodolphe Chabrier. Who knows which show will invent Thierry Ardisson when it will be possible to also play on bodies and environments.