LinkedIn emplea cookies para mejorar la funcionalidad y el rendimiento de nuestro sitio web, así como para ofrecer publicidad relevante. Si continúas navegando por ese sitio web, aceptas el uso de cookies. Consulta nuestras Condiciones de uso y nuestra Política de privacidad para más información.
LinkedIn emplea cookies para mejorar la funcionalidad y el rendimiento de nuestro sitio web, así como para ofrecer publicidad relevante. Si continúas navegando por ese sitio web, aceptas el uso de cookies. Consulta nuestra Política de privacidad y nuestras Condiciones de uso para más información.
FMX2013 presentation on challenges and solutions in transitioning from traditional offline CG production towards interactive approach used in creation of animated short Butterfly Effect. Butterfly Effect is a real-time CG-animated short developed in a collaboration between Unity Technologies, Passion Pictures, and NVIDIA.
Butterﬂy Effect:Workﬂow, Rendering & ShadingRenaldas ZiomaUnity Technologies@__ReJ__Monday, April 29, 13This talk will cover workflows and techniques that we adopted while working on theButterfly Effect.•
Unite template2What is it?Real-time animated shortCollaboration between Unity, Passion Pictures and NVIDIAMonday, April 29, 13If you haven’t seen Butterfly Effect is realtime animated short, developed in collaborationbetween Unity, Passion Pictures and NVIDIA•
Opportunity to Learn• Passion Pictures• New world of RealTime• No prior experience with Unity• Unity Technologies• CG Pipeline• Huge amounts of Data3Monday, April 29, 13Passion Pictures is a film and animation production company with broad experience inCG, stop-motion, documentary and traditional media. Passion Pictures have developedvisual elements for animated rock band Gorillaz, cinematics for Rock Band video gameand infamous (in UK) Meerkat advertisements.Prior to this collaboration Passion Pictures had NO experience with Real Timerendering / Game technologies or any experience working with Unity tools.We approached Passion Pictures in late Fall of 2011 looking for collaboration on a real-time short.Real-time was a completely new area for Passion Pictures thus presented a learningopportunity to adopt RealTime for internal pre-visualization and leverage experience forworking with Interactive projects (studio is working on number of such projects since).Passion Pictures: “We felt we had an opportunity to do something special that we hadn’tdone before, get experience with the Unity real time engine and help drive Unity in adirection that could integrate it with our studio.”
Project Outline• 30GB of source Assets• 5GB of runtime Data• 137 shots• 40 “scenes” in Editor4Monday, April 29, 13
Project Organization• Team of ~10 - ﬁrst month• pre production• research / tools• Team of ~30 - next 2 months• content• tools / engine modiﬁcations• Team of 5 - last 3.5 months• polish• optimizations5Monday, April 29, 13
Unite template6Monday, April 29, 13Passion Pictures presented a number of short film ideas and once Dan Sumich’s“Butterfly Effect” pitch was chosen - work started.Storyboard was developed from original pitch. Next storyboard was put into Animatronic- animated sequence of images in sync with music too capture flow and timing.Example of the storyboards for Scooter section in development.
Unite template7Monday, April 29, 13Color Boards helped to capture mood and lighting.It is worth noting how well Story and Color Boards resemble the final look of the short.
Previz• One Sequence• Goals• Initial timings• Visualize Space• Prime lenses• 10, 14, 18, 21, 24, 27, 35, 40, 50, 75, 100, 200• Zoom lens• 18 to 1508Monday, April 29, 13First pre-visualization was done in traditional to Passion Pictures way - directly inSoftimage XSI.Only standard camera settings based on physically cameras and lenses as used in Liveshoots (Prime/Zoom lenses) were used to capture the filmic feel.
Pipeline• Connect Passion pipeline with Unity Editor• Main GoalMinimal changes to Artist workﬂow!9Monday, April 29, 13Very short production cycle (traditional to CG industry) required minimal changes toartist workflow.Both Passion pipeline (XSI scripts) and Unity Editor scripting capabilities proved to beflexible enough.Passion Pictures: “After the creative challenge of finding the right director and script forthe film, the two immediate challenges were making sure that we had the right tools inUnity for the job so that we could take our pipeline into Unity, and also making sure thatwe could still work in the way we normally do as a studio.If we break down our pipeline into animation, simulation, rendering and compositingthen we wanted to be able to bring these as much as possible into Unity working withour existing content creation tools as normal.”
Passion Pipeline• No Change for Artists• Use Existing Asset Pipeline• Sequences SB, BU• Shots SB_01_00, SB_02_00• Assets [Character/Camera/Layout]• Export Models and Publish Shots as normal• Add Unity support10Monday, April 29, 13Standard Passion Pictures asset pipeline could be mapped to Unity asset representationquite easily. Multiple shots were grouped into a single Unity scene - some scenescontained one shot, some multiple. Sequence of shots (SB - SkateBoard, BU - Bus)became a group of Unity scenes sharing same folder.Standard Passion Picture pipeline (set of Softimage XSI scripts) was augmented with“publishing” shots to Unity project file structure and to incorporate SVN version control.SVN was a new tool for artists.
Unite templateCharacter Rig• XSI Softimage Rig• Bones• Envelope• Face Shapes• Controls• Unity Rig• Bones• Blend Shapes• Materials11Monday, April 29, 13
Unite templateFacial Rig• Bones• Head, neck, jaw, eyes• Blend shapes• Face• Mocap shoot• Face markers• Cubic motion• Retarget facial animation onto XSI controls12Monday, April 29, 13Facial animations in Softimage XSI / Unity: combination of bones and blend shapes.Face markers were used for facial motion capture, retargeted on controls in SoftimageXSI.
Unite template13Monday, April 29, 13Mocap facial shoot on the right. Animation transferred onto Softimage XSI character rigon the left.We worked with Centroid for the motion capture shoot. Passion Pictures provided themwith a FK rig containing just the bones and skin for our Hero character. Centroid sentFBX files for our FK rig send back FBX files of the FK rig from Centroid.Using an intermediate rig Passion Pictures transferred the FK animation back onto ourIK character rig in Softimage XSI. By keeping the shoot rig live in the scene, we wereable to either keep the motion capture, animate on top of it or replace it with newanimation, depending what worked in the shot.
Unite template14Monday, April 29, 13Facial animation controls in Softimage XSI after retargeting.
Non-linear time• Treat Sequences as Live Shoot• Cover the action from different angles• Each sequence needs multiple cameras15Monday, April 29, 13Action packed Butterﬂy Effect storyboard pushed Passion Pictures to treat ﬁlmalmost as a Live shoot. In traditional Live shoots multiple cameras are used tocover action from different angles. During the montage footage might overlapor jump in time according to Directors whim.Film was broken down into sequences which correspond to a continuous action(such as Skateboard sequence).Description about every camera was created in Final Cut and exported tocustom XML format (based on Edit Decision List or EDL ﬁle format of Final Cut).
Unite templateNon-linear time• Shot Description• Final Cut edit• Export EDL as XML ﬁle• Camera in/out points• Timeline in Unity• Sequence Preview in RV• Using EDL from Final Cut• Play whole preview sequence in RV• Instant Feedback for Animators16Monday, April 29, 13Action packed Butterﬂy Effect storyboard pushed Passion Pictures to treat ﬁlmalmost as a Live shoot. In traditional Live shoots multiple cameras are used tocover action from different angles. During the montage footage might overlapor jump in time according to Directors whim.Film was broken down into sequences which correspond to a continuous action(such as Skateboard sequence).Description about every camera was created in Final Cut and exported tocustom XML format (based on Edit Decision List or EDL ﬁle format of Final Cut).
Unite templateTimeline in Unity17Monday, April 29, 13Next this data could be read in Unity and used to control cameras, animatedobjects and physical simulations. Custom timeline viewer was developed inUnity using Editor scripting. Changes to the ﬁlm edit, jumping back and forwardin time and reordering of the shots became possible without need to rebakeanimations in Softimage XSI.
Unite templateTimeline in Unity18Monday, April 29, 13Next this data could be read in Unity and used to control cameras, animatedobjects and physical simulations. Custom timeline viewer was developed inUnity using Editor scripting. Changes to the ﬁlm edit, jumping back and forwardin time and reordering of the shots became possible without need to rebakeanimations in Softimage XSI.
Unite templateNon-linear Challenges• Animation• Particles• Physics• Rigid Body simulation19Monday, April 29, 13
Unite templatePoint Cache• Per-vertex Deformations• Cloth• Hair20Monday, April 29, 13Two types of vertex animation: BlendShapes and Point Cache were introduced in Unityfor Butterfly Effect.Passion Pictures: “In our standard pipeline we create point caches for each animatedmodel in the scene using .pc2 or alembic files. This creates a break in the pipeline sothat when we’re rendering we don’t need all the animation rig in the scene. For softwarerendering the cache sizes and data throughput aren’t an issue but in a real time gameengine we have to keep the enveloped characters and bake the animation on the bonesper frame.”
21Point CacheMonday, April 29, 13Point Cache used for extreme face deformation
Unite templateTessellation• CatmullClark• Now: on CPU• Future: OpenSubdiv library22Monday, April 29, 13(Mostly due to multiple layers of resting cloth) CatmullClark tessellation was used tomatch DCC tools exactly. It is problematic to implement CatmullClark using DirectX11hardware tessellation because of recursive nature of certain patches. Instead ofapproximation we went for exact CPU implementation for 1st level of subdivision.
Unite templateHair• Guide “strands” for modeling• GPU Tesselation and “Ampliﬁcation”•23Monday, April 29, 13Guide hair strands provide excellent control for animators.
Unite template24HairMonday, April 29, 13Final look of the hair.
25MentalRay Architectural(MIA) Shaders• Familiar• Small set of parameters• Physically Based• Micro facet• Energy conservingMonday, April 29, 13We aimed to provide PassionPictures artists with familiar tools, hence decision toimplement realtime counterparts for MentalRay architectural shaders.
Unite template261 parameter for DiffuseMonday, April 29, 13Between Clay and Plastic.Screenshot from the Unity Editor.
Unite template272 parameters for SpecularMonday, April 29, 13Roughness and ReflectivityScreenshot from the Unity Editor.
Unite template282 parameters for ReﬂectivityMonday, April 29, 13Reflectivity for depending on angle of incidence. Defined as pair of parameters - for rayhitting at Grazing angle (90) and for ray perpendicular to the surface.Screenshot from the Unity Editor.
Unite templateMIA Implementation• Oren-Nayar• Diffuse• Cook-Torrance• Specular• Geometric + Roughness + Fresnel term• Ward distribution for Roughness• I think... ;)• Direct + Indirect illumination29Monday, April 29, 13
Unite templateUnity Implementation• Oren-Nayar• very expensive• OK approximations• Cook-Torrance• Normalized Blinn-Phong instead of Ward• Roughness to Phong exponent (n):• n = 2 / roughness2 - 2• Tweaked to match MIA look:• n = 2 / roughness4 - 2• Schlick approximation for Fresnel• Only direct illumination at RunTime30Monday, April 29, 13
Unite template31MentalRay UnityMonday, April 29, 13Realtime implementation matches offline MentalRay implementation quite well. Notesome small differences in size of specular and very glossy surfaces.
Unite template32Monday, April 29, 13Most of materials where setup by Passion Pictures in Softimage XSI and weretransferred to Unity. Tweaks were necessary to achieve final look in Unity, but relativecompatibility of the MIA and Unity shaders saved a bulk of the manual work for us.
Unite template33Indirect Illumination (GI)• Point Clouds for Dynamic Geometry• Spherical Harmonics for Diffuse Bounce• Cube Textures for ReﬂectionsMonday, April 29, 13For indirect lighting we went for very practical approach - storing secondary bounces inthe low-resolution textures / Spherical Harmonics probes.
Unite template34Indirect Illumination (GI)• Textures for Static Geometry• Diffuse Bounce• AOMonday, April 29, 13
Unite template35Indirect Illumination (GI)• Calculated offline with Autodesk Beast• Path Tracer + Final Gather• IBL for Sky• Fake CausticsMonday, April 29, 13Beast setup:. 500 Final Gather rays (1 bounce). Path Tracer (4 bounces). 1024 IBL sky raysFinal quality scene bake would take between 20 minutes and 8 hours depending on thesize of the scene. Bake quality for all scenes in the short was carried on over-night on 4machines.
Unite template36Real Time ShadowsMonday, April 29, 13
Unite template37Baked Shadows• Of course baked only for static environmentsMonday, April 29, 13
38Ambient Occlusion• HBAO - Screen SpaceMonday, April 29, 13Horizon-Based Ambient Occlusion - developed by NVIDIA.
39Ambient Occlusion• Too Dark & DirtyMonday, April 29, 13
40Ambient Occlusion• Applied ONLY to Indirect Illumination• Colored to match Sky betterMonday, April 29, 13Screen Space Ambient Occlusion information is calculated at the start of the frame andpassed into the shaders, where it is applied to (occludes / darkens) indirect lighting only.Ambient Occlusion is tinted towards the sky to achieve more plausible look.
Unite templateSkin• Texture Space Diffusion• 3 Layers41Monday, April 29, 13Subsurface only scattering of the skin on the left side. Lighting in texture (unwrappedUV) space on the right.
Unite templateBack Scattering• Translucency Map• Adds intensity to sub-dermal layer42Monday, April 29, 13
Unite templateR44Monday, April 29, 13Final look of the skin.
Unite template45Depth Of Field• Antialiasing aware• FXAA based• With Splatted Bokeh TextureMonday, April 29, 13
46Monday, April 29, 13High variation of depth causes Depth of Field to produce noticeable aliasing on theedges of geometry. We married Depth Of Field filter with additional FXAA basedantialiasing technique driven by radius of Circle of Confusion to blur neighboring pixels.Example of aliasing caused by discontinuities in depth buffer on the left side. Ourapproach on the right side.
Monday, April 29, 13Depth Of Field using “standard” blur - top image.Detecting bright pixels and “splattering” bokeh sprites on top - bottom image.
48Thank you!Talented people, come join our team working on Showcases for Unity!http://unity3d.com/jobsMonday, April 29, 13
References49 [Bavoil08] Image-Space Horizon-Based Ambient Occlusion. L. Bavoil, M. Sainz, 2008 [Catmull78] Recursively generated B-spline surfaces on arbitrary topological meshes. E. Catmull, J.Clark, 1978 [CookTorrance82] A Reflectance Graphics Model for Computer graphic. R. L. Cook, K. E. Torrance,1982 [Hoffman10] Crafting Physically Motivated Shading Models for Game Development. N. Hoffman,2010 [Lottes09] FXAA. T. Lottes, 2009 [Mental07] mental ray Architectural and Design Visualization Shader Library. mental images GmbH,2007 http://www.mentalimages.com/fileadmin/user_upload/PDF/arch_and_design.pdf [Mikkelsen09] Microfacet Based Bidirectional Reflectance Distribution Function. M. Mikkelsen, 2009 [Nießner12] Feature-Adaptive GPU Rendering of Catmull-Clark Subdivision Surfaces. M. Nießner,C. Loop, M. Meyer, T. Derose, 2012 [Olano10] LEAN Mapping. M. Olano, D. Baker, 2010 [Schüler11] The Blinn-Phong Normalization Zoo. C. Schüler, 2011 http://www.thetenthplanet.de/archives/255 [Zhukov98] An ambient light illumination model. S. Zhukov, A. Inoes, G. Kronin, 1998Monday, April 29, 13