TL;DR:
- Pixar faced a challenge in animating elemental characters for their movie, Elemental.
- Creating a believable and expressive fire character, Ember, proved difficult.
- Traditional fire simulations resulted in unsettling visuals.
- Pixar turned to neural style transfer (NST) using machine learning to tackle the issue.
- The team combined stylized flames with the original simulation to achieve the desired effect.
- The use of NST required significant computing power but resulted in remarkable visuals.
- Pixar’s collaboration with Disney Research Studios propelled the success of the project.
- The integration of art and technology showcases Pixar’s innovative approach.
- The application of AI and machine learning in animation opens new creative possibilities.
Main AI News:
The intersection of art and technology has long been a hallmark of Pixar Animation Studios. From pioneering computer-generated imagery to pushing the boundaries of storytelling, the studio has consistently embraced innovation to create groundbreaking cinematic experiences. However, even for the imaginative minds at Pixar, animating the elemental characters in their upcoming movie, Elemental, proved to be a daunting challenge. Specifically, capturing the essence of fire and water in a way that felt authentic yet visually compelling seemed to elude the studio’s animators. That is until they turned to the power of artificial intelligence (AI) and machine learning.
Ember, one of the film’s protagonists, presents a unique predicament. As a character made entirely of fire, conveying its dynamic nature while maintaining its shape and personality proved to be a formidable task. Paul Kanyuk, a crowds technical supervisor at Pixar, explains, “Our fire fluid simulations are very naturalistic and they’re designed to mimic reality. But with a character like Ember, it’s really important to concentrate on the performance of the face.” Striking the right balance between the vibrant energy of fire and the character’s expressiveness posed a significant challenge.
Initially, attempts to shape Ember’s fiery form resulted in unsettling visuals, resembling a ghostly apparition or a menacing demon. The realism of the fire simulation, when applied directly to the character, created an eerie effect that missed the mark. Director Peter Sohn emphasizes the need to make Ember recognizable as fire, while ensuring it retains a unique personality. He says, “Fire naturally is so busy, but if you slow it down, it can turn into something that looks like a plasma. It was interesting to compare it to other anthropomorphized characters, because they’re all very fantastical and you can do anything with them. If you’re drawing an emotion, there is no one-to-one, but everyone knows what fire looks like.”
To tackle this challenge, Pixar’s team explored a groundbreaking solution rooted in machine learning. Drawing inspiration from a paper published by Siggraph Asia, they turned to neural style transfer (NST), a technique commonly used in AI to apply artistic styles to images. Kanyuk, who had been researching NST’s potential for solving animation-related problems, believed it could be the key to addressing the fire conundrum. However, he acknowledged the uncertainty inherent in machine learning, estimating only a 50 percent chance of success. Nevertheless, both Kanyuk and Sohn were committed to exploring all possible avenues.
Collaborating with Disney Research Studios, known for their expertise in AI and machine learning, Pixar embarked on a transformative journey. Jonathan Hoffman, a talented Pixar artist, created a set of stylized flames referred to as the “fleur-de-lis.” These whimsical and swirly flames served as a complement to the original fire simulation, allowing the team to infuse Pixar’s signature style and control into the dynamic movement of fire. Co-character supervisor Jeremie Talbot describes the breakthrough, saying, “Once you apply a style transfer to naturalistic fire, you can actually start to direct its style and start to bring the artist’s hand into something that is otherwise not touchable. It harmonized the look of Ember in a really unique way.”
However, harnessing the power of machine learning came at a cost—significant computational resources. Rendering all 1,600 shots from Elemental with NST required substantial graphical processing power. Initially, the team contemplated applying the technology only to close-up shots of Ember due to resource limitations. Yet, they soon realized the need for consistency and extended the use of NST to other fire characters in the film. Kanyuk explains, “The requirements went up and up, so we ended up getting a 20X speed-up… by tapping into the GPUs everyone at Pixar has on their computers.” This resourceful approach allowed them to overcome the technical hurdles and drastically reduce rendering time from five minutes to just one second per frame.
Ultimately, the collaboration between Pixar and Disney Research Studios enabled the successful realization of Elemental’s fiery visuals. Kanyuk believes they have only scratched the surface of NST’s potential, but he is thrilled with the elevated imagery they were able to achieve for this film. For Sohn, this endeavor epitomizes the harmonious fusion of art and technology at Pixar, where technology serves as a tool to express emotions and create captivating storytelling. He remarks, “It’s this coming together of left brain and right brain, and using technology as a tool to help express emotions… and in turn, we can connect to tech, versus it feeling like some cold new thing.”
Conclusion:
Pixar’s utilization of AI and machine learning to overcome the challenge of animating fire in Elemental demonstrates their commitment to pushing creative boundaries. By employing neural style transfer, they were able to infuse their distinctive style and control into the dynamic movement of fire, resulting in visually captivating and expressive characters. This breakthrough not only enhances the cinematic experience but also highlights the transformative potential of technology in the animation industry. It showcases the evolving market landscape where advancements in AI can empower storytellers and unleash new realms of artistic expression.
