WILLIAM REN
VFX Artist / 3D Generalist / UIUX / Graphic DesignVFX & Animation
Design
UI/UX & Game Design
Photography
About
Animation Production : Collaborated in a team of six to create a panoramic stereo 3D short animation film, utilizing the CAVERN system at CMU ETC I 2024 I 14 Weeks
Details
Inspired by a given prompt, we crafted a stylized and vibrant world of imagination. By blending music and choreography, in collaboration with dancers, we created a compelling and emotionally resonant experience that goes beyond conventional animation formats.
Throughout the process, I gained invaluable experience in immersive animation production. My contributions, in chronological order, included selecting music, creating the early animatic draft, developing the mocap character animation pipeline, designing lighting, animating environments, and some work related to layout and VFX. My teammates and I are truly proud of what we accomplished in just one short semester.
Visit our project website for more comprehensive development blogs.
https://projects.etc.cmu.edu/pulp-free-studio/
Credits :
Instructor :
Mocap Implimentation UE
Mocap Facial Livelink
Character Animation Layering
Environment Animation
Sound Design
Layout Design
Lighting
VFX
Kailun Deng
Sarah Wille
Zixuan He
Bonny Wang
Xun Zhang
Brenna (Dancer)
Bhuvan (Dancer)
John Dessler
Ruth Comley
▪ Full Animation
▪
Animation
& Cavern Recording
▪ Interactive Format
▪ This is an ongoing test to display our animation in a YouTube 3D surround format with red-blue 3D functionality. A big thanks to Steve Audia for making this possible!
▪ You can drag around using the mouse and switch to 3D mode in the settings at the bottom right.
▪Currently, there is no sound in this test.
▪ CAVERN
Before diving into the finer details of my involvement and what I learned this semester, it’s essential to emphasize the significance of our medium: CAVERN.
The CAVERN. is a projection system, featuring a 270-degree circular screen and a 20-foot-wide play area. It is designed to facilitate local multiplayer interactions within virtual spaces.
▪ Early Development
With a relatively short semester (14 weeks), our instructors—who also acted as our clients—recommended a thorough preproduction package to build upon. Since the prompt and initial vision were already established, we could dive into production immediately without spending additional time on narrative ideation.
This package gave us the freedom to explore world-building and story presentation while providing a structurally strong narrative foundation. Without hesitation, my team and I hit the ground running.
Music Selection and Story Boarding
Music is one of the most important aspects of animation. We prioritized it early in the process to establish a solid foundation for how the entire piece flows together and to identify key emotional moments to highlight.
Storyboarding for our project was very different from traditional media, as we were working with an immersive, large-scale environment. Traditional camera cuts weren’t feasible in this scenario, so we had to explore alternative storytelling techniques.
▪ Music Selection & Animatic
I was able to finalize our music selection in week 3, both from Artist.io. These two amazing pieces transition smoothly and align well with how we envisioned the narrative building up.
After multiple brainstorming sessions and iterations of the storyboard as a team, I created a rough animatic paired with the selected music and the block out of our scene done by our artist. This gave us a preview of the bigger picture and a foundation to build upon.
As simple as the animatic goes, it contains some of the most important moments we want to highlights though out the piece, emotion curves, transitions and how we bring everything to a end with pacing of the music.
The wide, up-to-scale aspect ratio also reminded us to consider the space and how it influences the composition.
▪ Character Animation Pipeline
We were allowed to purchase character assets for our project since we didn’t have a dedicated character artist on the team. This decision also helped speed up our production process. The assets we chose were created by '3DCartoon' on CGTrader.
We named them “Oliver” and “Magaret”
Visit the asset purchasing page here.
By the end of the semester, I successfully completed an animation pipeline tailored to meet the specific needs of our project.
Body mechanics were captured using motion capture at the CMU Mocap Lab, while facial expressions and hand animations were handled separately to ensure flexibility and precision. In the end, we layered everything together in Unreal Engine’s Sequencer to gain maximum control over the animation.
I handed off these animation assets to Sarah so she could set them up in the scene and create additional animations to align with our vision.
Again, big thanks to Justin Macey who is running the motion capture lab and classmate Riche Zhu in our project room introducing me to Liveline face mocap.
▪ Body Motion Capture
At the beginning of this semester, we were fortunate to secure dancers Brenna and Bhuvan from the CMU Ballroom Dance Club for our project. Their incredible improvisational dancing skills and enthusiasm were essential to our work. Thanks again for their time and invaluable contributions—this project wouldn’t have been possible without them. Additionally, Sarah and Kailun on our team contributed significantly as mocap actors.
The data from the Mocap Lab was pre-processed and cleaned up by Justin, making it ready for retargeting. However, this is where I encountered some early roadblocks. It took me about a week to figure out how to accurately translate the mocap data to our character rig.
Maya wasn't working completely in this case, so I dig in to motion builder which specifically shine under retargeting process.
We decided to move forward, as there were still many other aspects of the project to work on that would improve it further, making this issue less of a priority.
▪ Livelink Face Mocap
Using LiveLink with morph targets integrated into my model, I can perform facial expressions effortlessly. Built on the same skeleton and enhanced with an additive animation layer, this setup allows us to seamlessly combine and fine-tune animations with ease.
When acting with this method, it’s both fun and challenging, as I need to keep in mind that some features of my expressions may become exaggerated when applied to a stylized character like Oliver.
▪ Modular Hand Animation
I also modulated hand animations using four keyed presets, covering a range of gestures appropriate for our animation. By leveraging Unreal Engine's sequencer animation blend functions, we can efficiently layer hand performances in a flexible and dynamic way.
▪ Animation Layering
Here’s a quick example of how animation layering is done in Sequencer. Each track represents one of the animation modules I mentioned earlier.
This modular method provides the best control and timing for our character's performance. While it can be a meticulous process, the results are undoubtedly worth it!
One challenge I faced in animation was that the mocap data didn’t align with the environment due to the limited space in the mocap studio. However, I managed to seamlessly extend the walking animation to fit the environment.
▪ Lighting / VFX / Environment Animation
I contributed to completing the lighting design for the ending, where Oliver experiences an emotional journey with his imaginary late wife. This lighting highlights Oliver's emotions and provides a striking contrast to the beginning of the animation.
Since we couldn't use a traditional camera setup in CAVERN's immersive nature, achieving a convincing depth of field was a challenge. I devised a solution by utilizing volumetric materials to simulate this phenomenon effectively.
Here's a comparison of the scene before and after implementing light scattering through volumetric materials. This technique helps achieve a clean and focused background, enhancing the visual clarity and ensuring the characters remain the focal point.
(Warm Hole Scene)
Since CAVERN's projection system darkens the image and wearing shutter glasses further reduces brightness, we needed to assist the audience in tracking our characters. In this scenario, adding a spotlight that follows our character effectively solved the problem.
(Orange World)
I also contributed by adding environment animations and VFX to enhance the immersion and bring our already amazing environment to life.
For example, I incorporated falling leaves, spinning large oranges, and small oranges bouncing to the beat of the music.
(Toast Galaxy)
With the amazing animation layout Sarah Choreographed and the stunning environment designed by Lucy, I was able to add the finishing touches by animating the entire scene to complement our character’s movements. This included elements like toasts flying and butterflies joining the dance, adding an extra layer of life and energy to the animation.
This not only add another layer of fantasy, takes advantage of the scale and immersiveness of our medium, it also showcased our amazing environment (Lucy), texture (Xun) and layouts (Sarah).