EMO, an AI system that makes photos and illustrations sing and talk realistically

【EMO, an AI system that can make photos and illustrations sing and talk realistically】
https://gigazine.net/news/20240301-emote-portrait-alive/

・A team from the Intelligent Computing Laboratory of Alibaba, a Chinese technology company, has announced Emote Portrait Alive (EMO), an AI system that can animate photos and images to produce moving images that appear to speak and sing in real time.

・EMO can generate smooth, expressive facial expressions and head movements that match the input audio track.

・Unlike conventional methods that rely on 3D facial models and synthesized contours to mimic facial movements, EMO generates video directly from the input audio. This enables natural video output by capturing the subtle movements and inherent habits of singing and speech.

・The researchers plan to investigate methods of detecting synthesized images in order to address problems such as the spread of misinformation and misuse of this technology for impersonation.

The above is a quote from the article

The ability to combine and produce is becoming more important.

This kind of technology has been around for some time, but it differs from the past in that it “generates video directly from audio,” and the quality is amazing.

The chatter and the singing seem quite natural.

Of course, there are concerns about misuse,

If it is possible to create such a good quality from still images, I think it will be a very effective tool for animation production in the future. (The work in the animation field seems to be very demanding.)

In the future, it will be possible not only to make still pictures and characters speak and sing, but also to move their bodies as desired.

Then, the time may come when it will be possible to easily create animations of characters created by oneself.

【A definite change in the way music is made. I was able to create an entire concept album for a fictional rock band using AI!】
https://www.techno-edge.net/article/2024/02/27/2872.html

I mentioned this on X (Twitter) the other day,

The story is that the songs for the album, the album cover, and even the members of the band were all created by AI.

I knew that we would eventually come to an era like this, but it seems that we have already entered that era.

Creativity in creating 1 from 0 is of course important,

In the future,

However, in the future, the main focus may be on creativity that extends, combines, and produces what is already there.

DJ ability (ability to combine) and producing ability are important.

Let’s hone and train such skills. (Even more so if you are going to be involved in the creative industry)

See you then

I often compose music based on existing music, and I sometimes wonder if that is much different from what AI compositions are doing. It makes me wonder what human and AI creativity is.