A team of A.I. researchers, choreographers, and VFX artists have united to create an interactive portrait that can mimic eerily humanlike emotional states and movements. Behind the interactive portrait is an A.I. cognitive model that reads the emotional state of the viewer in real time and predicts the appropriate emotional response, performing the corresponding humanlike movements required to express it. The project was debuted during Focus fair at the Saatchi Gallery in London from October 4-7.
“My initial references mostly came from art history and especially early Expressionism,” said one of the project’s leaders, Jesper Eriksson, citing Klimt, Schiele, Van Gogh, and the Post-Impressionists. “I enjoyed the philosophical switch that started at that time, the whole notion of being able to express internal states of being, thoughts, and emotions. What if we could take that one step further and create a portrait that explicitly tried to model these things in real time? Maybe we can push the notion of portraiture a little bit further.”
A.I. interactive portrait debuted at Focus fair at Saatchi Gallery, October 4-7, 2023. Photo courtesy of Jesper Eriksson.
The emotional cognitive model behind the portrait assumes that much of the way that we read emotion in others is dependent on their movements. The gestures that the portrait both recognizes and uses to respond were recorded in a lab according to a movement theory first originated by the dance artist and choreographer Rudolf von Laban. His ideas were adopted by the dancer Ya Malmgren, who combined them with Jungian psychoanalytic theories about personality types to create a new method for character development based on how we can express our inner state through movement.
“That was a fantastic key for us, because through a camera feed [inputting live footage into the algorithm] we can only read movement. Then we have to infer what that person is like, psychologically speaking,” said Eriksson. “The theory is rigorously well-defined enough that we could code it so a computer can understand it.” In this way, the model combines age-old classical theories with state-of-the-art A.I. techniques for cognitive modelling.
These movements, performed by ballet dancer Rebecca Wield, formed thousands of data points with labeled dimensions like weight, space, time, and flow that the A.I. learned on top of learning to recognize a basic type of gesture like a wave. So far, the team has focused on just one waving gesture with nearly 50 variations communicating corresponding emotional nuances. This aspect will be further developed once the team secures more funding to collect more motion capture data.
Motion capture data is recorded on the movements of ballet dancer Rebecca Wield. Photo courtesy of Jesper Eriksson.
“We’re just trying to get one thing close to feeling like human behavior,” said Eriksson. “How many variations do we need to have for people to stop trying to figure the algorithm out and just go along with the experience of it? At what point do people start feeling like the portrait is actually listening and responding to them rather than running rehearsed scripts?”
In this sense, the portrait is both a fun interactive device and an ongoing piece of research. “It’s an empirical process, which is strange to talk about in an artistic context,” he added. “But we can use art to help people to connect to it.”
In a world filled with ever more sophisticated scams, what about the possible misuses of a machine that can mimic human behavior? “Obviously if we uncover something that feels very human, it could be used for all kinds of nefarious purposes to deceive or as propaganda,” admitted Eriksson. “This is a question that our research group will not be able to answer, but it’s something that we have to invite a much bigger audience to make progress on.”
Moving the conversation swiftly on to more positive outcomes, Eriksson cited a potential benefit to the technology: the possibility of providing a responsive A.I. companion to those experiencing extreme loneliness.
More Trending Stories:
