“Return the age of Choi Min-sik, an actor in his 60s”
Disney+ ‘Casino’ post-production staff
AI de-aging with realism and youthful transformation
Born in 1962, 61-year-old actor Choi Min-shik took on the challenge of acting in his 30s. The Disney+ original series ‘Casino’ is a 16-part series depicting the life of Cha Moo-sik, a Korean casino godfather in the Philippines. He encompasses the present through his childhood, when he was left at a daycare center, through his student and youth days. Until the 20s, child actors and others acted, but Choi Min-sik digested the 30s and older.
After undergoing skin care and skin-tight procedures, is appearance the secret? You’re welcome. The great actor Choi Min-shik’s wrinkles are still deep as time goes by. Instead, it is the ‘face and voice de-aging’ technology using artificial intelligence (AI) that has turned back his time by nearly 30 years. It is not at the level of editing the scene for a few minutes, but in several times, freely crossing the age range.
In 2019, the production cost was lower than the Netflix movie ‘The Irishman’, which embodied the 20-somethings such as actor Robert De Niro in his 70s, and the level of implementation was also advanced. If you ask me if it was difficult to even notice what kind of technology was used, that is an extremely high praise for those who have been working on post-production for over three months. We heard the story behind the work that continued the ‘first’ attempt from the C-JeS Gulliver Studio of the visual effects (VFX) team and Supertone, who were in charge of AI voice synthesis.
‘VFX Director’ Lee Joo-won, Director of C-JeS Gulliver Studio
It was the Visual Effects (VFX) team of C-JeS Gulliver Studio that tried a new face de-aging with domestic technology in ‘Casino’. This studio previously won an Emmy Award for Special Visual Effects for Netflix’s “Squid Game,” and this time, it succeeded in boldly putting the de-aging technology that was in the research stage into practical use.
Director Lee Joo-won, who oversaw this process, said, “Although our technology level is not a complete leader in the world, it is a meaningful first step to boldly put AI de-aging to practical use.”
This de-aging was implemented through post-production using AI without using any special equipment at the filming site. For example, just as a photo editing application on a smartphone takes a picture of a person’s face and then applies a new filter, the AI-learned face of Choi Min-sik in his 30s is synthesized on top of Choi Min-sik’s face in his 60s. The production process is much simpler than in Hollywood special effects films such as ‘The Time of Benjamin Button Goes Backwards’, ‘X-Men’ and ‘Gemini Man’ in the past, where the actor had a machine attached to his head or put dots on his facial muscles and performed motion capture. done
Of course, the process of implementing this far was not easy. The drama footage is 4K super high definition, but Choi Min-sik’s footage from 30 years ago was a low-resolution videotape, so there weren’t many usable materials. Accordingly, ‘Super Resolution’ technology that raises image resolution with AI or traditional 3D capture technology was also used as a supplement.
It was a bigger task to express acting, which is the ‘realm of art’ here. “The more wrinkles are removed to make the face younger, the more the detail of the acting disappears,” he said. AI alone was not enough to bring out the emotional changes expressed by moving fine muscles, such as trembling the corners of the mouth or adjusting the power of the eyes. In response, the VFX team divided the facial contours such as the forehead, eyes, nose, and chin again on top of the video that had gone through the AI process, and modified the degree of correction one by one.
“Especially for middle-aged actors, every wrinkle on their face has evolved into a tool for acting. Also, even if you are a person of the same period, depending on the situation you are in, your complexion may be dark and wrinkles may appear. The de-aging work had to follow the emotions the actor wanted to express.”
Director Lee created the face of the ‘young Choi Min-sik’ to be realized on the screen, communicating with director Kang Yoon-seong, production crew, and actors throughout the entire period from scenario preparation, test shooting, on-site shooting, and post-production. In addition to AI work, many teams were involved in makeup, lighting, and editing.
Of course, in some works, excessive settings and effects are pointed out as elements that hinder viewer immersion. The reality itself, such as short production period and limited production cost, is a big barrier. Director Lee said, “The viewer’s evaluation standard is rising.” “In ‘Casino’, it was not a perfect 30-year-old, but there was an expected result in that it realized the point of contact between the actor’s past and current acting.”
‘AI Voice Synthesis’ Lee Seung-bok, Head of Supertone Content Business Division
It is not enough that the character’s face has become younger. If you speak in a low voice that is old and low on that face, reality will fall. Supertone, a start-up with original voice synthesis/separation technology, made the voice of Choi Min-sik in his 60s into his 30s. Previously, through a broadcast program, it drew attention by introducing a technology to restore the voices of legendary singers such as the late Kim Kwang-seok and Lim Yoon-taek, and even the voice acting beyond singing was made with AI.
Like face de-aging, voice de-aging is the first attempt in the world to be used in a video work. Lee Seung-bok, head of the Supertone Contents Business Division, said, “The previous film, The Irishman, drew attention for its de-aging technology, but it was an area like homework that could not be solved in the audio industry because only the face became younger and the voice remained the same.” When I came in, I was able to achieve results,” he said.
If the voice of an actor simultaneously recorded at the filming site is processed as noise-free and clean as possible using AI voice separation technology in ‘Gaudio Lab’, Supertone utilizes its own technologies such as ‘Age Slider’ and ‘Voice Jin’ to create ‘Cham Moo-sik in his 30s’. ‘. The public remembered Choi Min-sik’s youth, the character Choi Min-sik tried to express through this work, and the tone that was not awkward for viewers.
“I felt that Choi Min-sik was a great actor, he had many voices left in his 30s, but each character had a different acting. By referring to past works such as ‘The Moon in Seoul’ (1994), ‘Number 3’ (1997), ‘Firan’ (2001), ‘Old Boy’ (2003), and ‘War on Crime’ (2012), the AI is the most challenging. I tried to implement a suitable voice for ‘, and it took a lot of time to find the final nuance. The subsequent conversion was relatively straightforward.”
Director Lee picked the second episode, especially the scene where Mu-sik (Choi Min-sik) and his friend Dong-eok (Lee Jong-yoon), who are of the same age, have a conversation in a dorm in the Philippines as a scene where the voice synthesis was well melted. “It’s a comfortable conversation between friends, but I felt the age difference between the actors in the actual original video and audio. Still, the quality of the original audio was good and the conversion was natural, so I thought ‘it came out great’ after the scene was finished.”
Supertone aims to develop hyper-realistic AI voices that can be used in the fields of culture and art in the future as well. For example, the level of technology has risen to the point where AI can replace foreign language dubbing over videos performed by Korean actors in Korean. You can dub without language restrictions by keeping the unique tone of the actor as it is.
“Even without the original voice, new character voices for each gender, nationality, and race can come out. However, it is a part that we have to keep thinking about whether this can be seen as a unique creation and whether there will be resistance from viewers. I look forward to seeing how far the AI technology that makes imagination a reality will be commercialized.”