Developing and optimizing a procedural game: The Elder Scrolls Blades- Unite 2019
•
3 recomendaciones•5,729 vistas
The Elder Scrolls Blades strove to produce high-quality visuals on modern mobile devices. This talk will describe the challenges of achieving that level of quality in procedurally generated 3D environments. Speakers: Simon-Pierre Thibault - Bethesda Game Studios Sergei Savchenko - Bethesda Game Studios Watch the session here: https://youtu.be/KbxiGH6igBk
Recomendados
Más contenido relacionado
La actualidad más candente
La actualidad más candente (20)
Similar a Developing and optimizing a procedural game: The Elder Scrolls Blades- Unite 2019
Similar a Developing and optimizing a procedural game: The Elder Scrolls Blades- Unite 2019 (20)
Más de Unity Technologies
Más de Unity Technologies (20)
Último
Último (20)
Developing and optimizing a procedural game: The Elder Scrolls Blades- Unite 2019
- 2. Developing and Optimizing a Procedural Game: The Elder Scrolls: Blades Simon-Pierre Thibault Sergei Savchenko Bethesda Game Studios
- 3. The Elder Scrolls: Blades • Advanced visuals on mobile platforms • Procedural dungeons • To achieve this: • Built custom lighting solutions • Significantly optimized for performance and memory
- 5. Level Example • Made of standalone building blocks: Rooms • Assembled by our dungeon generator at runtime • Rooms have different shapes and sizes • Rooms are universal for each theme • Rooms as streamed during gameplay to support larger levels.
- 6. Room Prefab that contains all the data required to use that piece in a level • Art Content • Level design data • Gameplay data • Lighting
- 7. Level Lighting High visual quality is an important pillar for this project • Based on the built-in render pipeline • Uses a modified version of the Unity Standard shader • Mix of real-time and baked lighting
- 8. Lightmaps in a Procedural Context
- 9. Lightmap Seams • Lighting at connections differ from room to room • Our global illumination is not really global… • Rooms are attached at runtime, so we can’t predict that at bake time • Solutions?
- 10. Lightmap Blending Concept: Sample the lightmap on the other side of a connection to blend with it • Assign the secondary lightmap to the material • For each vertex • Assign a blend factor based on the distance to the connection (vertex color alpha) • Assign a extra uv to sample the lightmap texture on the other side (3rd uv set) • Edit the shader to sample the secondary lightmap
- 12. Connection Extension Mesh • Generate lightmap information beyond the connection • Never rendered in-game • Used at runtime to find a proper color for blending • Will avoid stretching the color at the connection
- 14. In Game Final Results
- 15. Limitations and Drawbacks • Extra texture fetch per pixel rendered • Large runtime cost to calculate secondary lightmap UVs for each vertex near the connection • Incompatible with Unity’s static batching
- 16. Light Probes in a Procedural Context
- 17. Light Probes • Contain directional lighting information for a point in space • Is a built-in feature of Unity • Allows dynamic objects to sample baked lighting • Doesn’t work with our procedural pipeline • Light probes data is saved in scenes • Light probes cannot me moved
- 18. Light Probes Pipeline Custom runtime solution Light probes generation (Editor) Unity Renderer Shader
- 19. Custom Light Probes Runtime System • Write probes data to an asset we attach to each room • Load light probes data for each room • Attach probes for each room into a global probes network • Sample the network for each dynamic renderer based on world position • Send the interpolated light probe information to the shader
- 20. Light Probes data • The editor generates a UnityEngine.Rendering.SphericalHarmonicsL2 instance for each light probe. • The shader expects data in a different format. • This is explained in the Unity documentation: • “The Unity shader code for reconstruction is found in UnityCG.cginc and is using the method from Appendix A10 Shader/CPU code for Irradiance Environment Maps from Peter-Pikes paper.” • https://docs.unity3d.com/Manual/LightProbes- TechnicalInformation.html
- 21. Writing the shader properties Use MaterialPropertyBlocks to fill in shader properties without going through a material. https://docs.unity3d.com/ScriptReference/ MaterialPropertyBlock.html Use the [PerRendererData] attribute on your shader properties https://docs.unity3d.com/Manual/SL- Properties.html LightProbeUsage.CustomProvided was added in 2018.1
- 22. Considerations • Sampling the network for a large amount of renderers can be expensive • Only sample once for static objects • Only update dynamic objects if they have moved • Because we place probe on a grid, we sample the network in constant time. • Unity handles ambient color through the same shader properties • Ambient color needs to be added to the probe's values
- 23. Performance…
- 24. Blade’s Frame: CPU side Rendering Main Performance Drivers: • # of draw calls • # of rendering passes • Efficiency of batching • Efficiency of the Graphics API
- 25. Reducing the number of draw calls • Dynamic visibility culling • Portal based strategy for dungeons • Distance based strategy for forests • Dynamic occlusion culling for the town • Buildings as occluders Occlusion Culling
- 27. OpenGL vs Vulkan: Chipset CPU GPU 32/ogl 64/vkn Loading Sim Time Warm up Issue Loading Sim Time Warmu p Issue Exynos 8895 Octa Octa-core (4x2.3 GHz Mongoose M2 & 4x1.7 GHz Cortex-A53) Mali-G71 MP20 25.9 0.041 no 28.1 0.048 no Qualcomm MSM8998 Snapdragon 835 Octa-core (4x2.35 GHz Kryo & 4x1.9 GHz Kryo) Adreno 540 29.3 0.0352 yes 21.3 0.0357 no Qualcomm SDM845 Snapdragon 845 Octa-core (4x2.8 GHz Kryo 385 Gold & 4x1.7 GHz Kryo 385 Silver) Adreno 630 21.8 0.0347 yes 17.6 0.0349 no Qualcomm MSM8998 Snapdragon 835 Octa-core (4x2.35 GHz Kryo & 4x1.9 GHz Kryo) Adreno 540 29.5 0.0354 yes 20.4 0.0348 no Hisilicon Kirin 970 Octa-core (4x2.4 GHz Cortex-A73 & 4x1.8 GHz Cortex-A53) Mali-G72 MP12 20.8 0.092 no 23.6 0.14 no
- 28. Dynamic Graphics API Selection CustomPlayerActivity onCreate this.getIntent().putExtra("unity", "-force-gles"); E.g.: If some Mali based device yes
- 29. To Summarize: • Vulkan is definitely viable and fast • Your game’s performance needs to be tested on Adrenos and Malis • Use dynamic graphics API selection to try to get gains on both sides
- 30. Threads and Cores: Core 0 Core 1 Core 2 Core 3 Core 4 Core 5 Core 6 Core 7 WorkersWorkersWorkersWorkers UnityMain UnityGFXDeviceW ?
- 31. Thread Affinity WorkersWorkersWorkersWorkers UnityMain UnityGFXDeviceW ?
- 32. Default Affinity Model for 8 cores: Core 0 Core 1 Core 2 Core 3 Core 4 Core 5 Core 6 Core 7 WorkersWorkersWorkersWorkers UnityMain UnityGFXDeviceW
- 33. Customized Affinity Model for 8 cores: Core 0 Core 1 Core 2 Core 3 Core 4 Core 5 Core 6 Core 7 WorkersWorkers UnityMain UnityGFXDeviceW WorkersWorkers
- 35. To Summarize: • On Android Devices Unity prefers big cores, this may or may not be beneficial in all scenarios • Affinities can be adjusted on initialization to assign UnityMain and Rendering threads to their own cores and to redistribute workers • One should not expect a major performance gain from this (but the cost of this change is also low)
- 36. Memory…
- 37. • Xcode reports 1.18 G is used… • Unity simple mem profiler says: 451.4M used… • Unity Details says ~569M used… • Instruments say 786.86M resident with ~650M dirty…
- 38. Memory Types Device Virtual Memory External Device Resident Memory Compressed Internal Clean, Unloaded Native memory Managed Heap The Game Reusable
- 39. iOS Memory Tracking • Use task_info call to get accurate current memory summary:
- 40. iOS Memory Tracking: Device Memory Managed Heap Memory Compression
- 41. To Summarize • Different tools account for subset of types when reporting (Xcode widget is the most relevant) • iOS devices have complex memory management system with many memory types • Current memory use can be fetched from the OS
- 43. Heap Growth Control: • Various statics from Boehm-Demers GC implementation can be externed and accessed • Specifically GC_free_space_divisor from alloc.c • Divisor value controls managed heap’s growth increment • Default value: 3 • Custom value e.g: 16
- 44. Low Level Memory API • Memory Profiler API permits capturing snapshots for both native and managed memory
- 46. To Summarize • Boehm GC can be tweaked in the run time so that heap expands in smaller increments • C# objects may cost significantly in terms of memory • Low level memory API enables building custom tools for memory tracking and leak detection
- 47. Questions? Simon-Pierre Thibault Sergei Savchenko Bethesda Game Studios
Notas del editor
- Intros
- How difficult it is to build a procedural game with advanced visuals for mobile platforms? We will try to offer some insights Our experience is from working on The Elder Scrolls Blades – it reimagined TES for modern mobile platforms Specifically we will talk about: precomputed lighting in procedural context some aspects of performance/optimization that we dealt with
- This is an example of what a level looks like from an editor point of view Made of standalone blocks called rooms The red lines represent connections between rooms Different shapes and size but they all fit together Any room can be attached to any other rooms for the same theme. All fit on a grid that acts as a framework for the level generator To support longer levels without using too much memory rooms are streamed in while the player progresses through the level
- Example of a single rooms Rooms are prefab that contain all the data we need to use that piece in a level
- Unity render pipeline Scriptable render pipeline was still experimental when we started the project Realtime lighting Looks great with the PBR shader, especially with high definition normal maps Allows dynamic shadow in key areas like this window Baked global illumination Both lightmaps and light probes Add depth to scenes with indirect lighting Can add lot of baked lights without affecting performance
- On this screenshot what you see is a harsh cut produced by lightmaps along the floor and wall were two different rooms connect. The problem here is that since lightmaps are generated separately for each room, we can get end up with dramatically different colors and brightness around room connection. Because of the way we bake global illumination per room, our global illumination isn’t really global. We also can’t predict which rooms will get connected together because they’re assembled randomly at runtime. Bake time solutions… We concluded that we needed a runtime solution to this
- That runtime solution is lightmap blending. The concept here is rather simple, what if we could blend the lightmap on each side of a connection to make the seam disappear? Then we could start with a 50-50 blend at the connection and the fade it out based on distance. Well you need a few things to achieve this. One, we need to add a secondary lightmap for blending to the materials on each side. Each side needs the other side’s lightmap And since a blend like this needs to be applied per pixel, we need to assign a blend factor to each vertex. And we need a uv set to sample the secondary lightmap texture. Once we have that we simply edit the shader to sample both lightmaps and blend them based on the blending factor. All of this is setup can be done at runtime when attaching two rooms together
- With this technique we only end up using the lightmap color at the seam and stretching it, so any local detail can get blow out of proportion which doesn’t look good. The problem is that we don’t have enough to blend that shadow correctly, we don’t know what that shadow in the blue room would have looked like if it had continued naturally. Well what if we could create that information?
- This is exactly what we did! We’ve created a mesh that is precisely the shape of the connection but stretched based on the blending distance. We make it so that it gets lightmapped by Unity at bake time, but we never actually render it in game. We only use it’s lightmap data. With this we have more accurate data for blending so we don’t need to stretch anything anymore.
- The harsh cut is gone and we don’t have any noticeable artifact. The light on the floor from that torch does fade pretty quickly at the connection. It’s not 100% accurate, but still an acceptable compromise.
- Because of the blending we do have an additional texture fetch per pixel, but we only do that for static renderers that are near a connection. So it’s not very significant. The worst performance problem comes from computing the uv set for the secondary lightmap. It involves finding the closest point on the extension mesh for each vertex of the blended renderers. This can be quite heavy calculations but what we did is run all of that asynchronously in a background thread. When streaming a level, the new rooms are spawned outside of the view of the player so we don’t need to apply lightmap blending immediately. And because we modify meshes at runtime by adding an extra uv set this doesn’t work with Unity’s static batching. You’ll have to either remove the static batching option on meshes that are blended or disable the feature globally in your project settings.
- So this takes care of our biggest lightmap problem, but I mentioned earlier that we also use light probes as part of our lighting pipeline.
- Light probes are object that contain directional lighting information for a specific point in space. They’re a built-in feature in Unity. If you setup a LightProbeGroup component in your scene, Unity will generate light probe information when you bake the lighting for that scene. At runtime they allow the engine to sample baked lighting information for dynamic objects. They’re a complementary feature to lightmaps. Lightmap are for applying baked lightins to static object and light probes are for applying baked lighting to dynamic objects. The problem with light probes is that Unity’s implementation doesn’t support our procedural pipeline. The first reason is that light probe data is stored in scenes only, but we use prefabs for our rooms. The second and main reason is that probe positions are set at bake time and cannot be moved. A specific room’s position in world space is not fixed in our case. We have to be able to spawn that room wherever our level generator tell us to.
- At a high level this is how the light probes pipeline is separated in Unity. We have the generation part that runs in the editor when baking lighting. There’s the renderer that samples light probes at runtime for each dynamic object. It finds the probes that surround the object’s position and interpolate their values and then there’s the shader that uses that information to render the object. The only part of this that’s not compatible with our procedural pipeline is the renderer. So what if we just replaced with our own custom implementation?
- So how would do that? We write the light probes data generated by Unity to a binary file that we attach to each room prefab. When a new room is attached at runtime we deserialize the probes data and attach the probe to a global probes network for the level. Then every frame we sample the probes network to find probe information for each dynamic object based on their position. Then we send over that data to the shader.
- How do we manage that data? Well once lighting an been baked the LightPorbes class contains a SphericalHarmonicsL2 object for each probe in your scene. Under the hood this is just an array of floats so it’s easy to serialize. The main problem we faced here is that the shader expects a different format and we didn’t know how to translate a SphericalHarmonicsL2 object into the format the shader expects. Well it turns out there’s a very important line about this in the Unity documentation. It says: “The Unity shader code for reconstruction is found in UnityCG.cginc and is using the method from Appendix A10 Shader/CPU code for Irradiance Environment Maps from Peter-Pikes paper.” That links to a paper called Stupid Spherical Harmonics tricks by Peter-Pike Sloan. Appendix 10 of that paper contains the precise formulas that you need to feed a SphericalHarmonicsL2 object to Unty’s Standard shader.
- But how exactly do you feed that data to the shader. The standard way of filling in shader properties is through materials. But probe values could be different for every single object even if they share the same material. We don’t want to create a unique material for every object, that would prevent batching. Unity has a feature for cases like this called MaterialPropertyBlocks. This allows you to fill in shader properties per object without having to go through a material. When using this you also need to set the [PerRendererData] tag on you shader property so that the engine knows your property will be filled in through MaterialPropertyBlocks. For probes data specifically you should set the lightProbeUsage value of all your renderers to “Custom Provided” This tells Unity that you’re responsible for filling in spherical harmonics. That way Unity won’t override your values.
- Overall replacing that system worked well for us, but here are a few considerations. Sampling and interpolating probes for each dynamic object each frame can be pretty expensive so you need a way to know which renderers will or won’t move. Objects that don’t move should sample the probes once when spawned and then never again. Objects that do move should not sample probes if they have not moved since the last sampling. In our case we fill rooms with probe based on a 3D grid which means we a regular layout for our probes. This is very helpful because it allows us to find the closest probes to any point in space in constant time. Without a regular layout it becomes a much more complex problem to find the surrounding probes for any position. Unity’s default system supports that with by tetrahedral tessellation, but we didn’t need to reimplement that on our side. Another thing to keep in mind is that the shader properties used for spherical harmonics also contain the scene’s ambient color, so you’ll need to handle that yourself by adding the ambient color with the probe’s values before sending them to the shader.
- Not running at high enough frame rate Android might be more affected
- CPU side rendering is costly and could often be a bottleneck While GPU side could be as well but Tile based GPU deal well with overdraw Generally shader performance is easier to deal with The number of draw calls is a big performance driver for both CPU and often GPU Control the number of passes Shadows Depth pass Pixel lights etc. Efficiency of batching: static and dynamic Efficiency of graphics API used…
- Culling is significant in reducing the number of draw calls A procedural game cannot use precomputed visibility culling Dynamic visibility culling interior strategy exterior strategy Dynamic occlusion culling for town environment Buildings are natural occluders for other buildings, vegetations, decorations, NPCs etc.
- Graphics API is a huge performance driver What is faster OpenGL or Vulkan? OpenGL is a traditional API maintaining internal state Vulkan is much more modern with many facilities to reference states and rendering data
- Performance depends on graphics drivers… Loading times and FPS We discovered shader cache on some OpenGL drivers Vulkan is generally faster… But could be slower compared to OpenGL on other devices
- Dynamic graphics API selection to attempt the best of both world Unity cannot change graphics API in the run time But there is a provision for auto selection and fall back Can also be initialized with a different API Requires custom Unity build To implement -- needs custom player Activity (java) Checking device or SOC name to pass a flag to unity
- Vulkan is viable Not many graphics artefacts Faster overall Beware validation layer Helps shader compilation caching Test in your specific case Both frame performance And loading times Dynamic API selection when beneficial Works in debug out of the box May need a custom Unity build May make it into main line Thanks to Unity for their help working with us to implement
- Continuing with Android: How Unity distributes threads to different cores Most modern Android devices are 8-cores There are big cores that are clocked higher And small cores that are slower but consume less power Unity game can easily have 30-40 threads running Most important ones are: UnityMain UnityGFXDeviceW – multithreaded rendering is presumed to be used Worker threads These will be used more and more with DOTs and Jobs system Some systems use these already E.g.: skinning, physics, particles Use Systrace to find out how threads are distributed
- Core jumping Contention between workers and main/rendering threads Little cores not used…
- Putting everything on big cores might actually be ok These are faster Workers and not uber busy We experimented with alternative affinities
- Reserve cores for UnityMain and UnityGFXDeviceW Allocate workers (and choreographer and audio etc.) to other cores big and small
- This is what the result looks like Generally found this to be measurably faster (5-10%) Only 4 workers We asked Unity for a feature to allocate more workers
- Nav mesh generation benefits from workers on little cores Need to code a custom plugin to discover thread ids and adjust affinities Plugins are very easy with IL2CPP Setaffinity syscall to set core mask Not a major performance improvement Yet measurable We asked Unity for more workers Systrace is very useful iOS has much better single core performance (fewer cores though) Thanks to Google for their help
- Running out of memory iOS is more affected Generally less memory on iOS devices, many with only 2G
- Tools give different reading Xcode widget is the most important reading – OS jettisons apps Other tools give smaller estimates Modern mobile devices memory system is complex
- Native memory for asset data: textures, meshes etc – Reference counted Managed memory for Unity Objects and C# objects – GC collected Doesn’t account for everything in a Unity app though Virtual memory space – large, exceeds physical space Physical wired memory is only a small part Consists of wired dirty memory, Reusable (GPU uses this for storage during rendering) External, encrypted code Some read-only assets can be unloaded Some dirty memory can be compressed
- Use mach functionality to get detailed view
- A practical view of memory types evolving over time Overall use of internal memory is important Managed heap state is important Compressed memory utilization is important
- Xcode widget reflects what OS is using for decision making There are many memory types and several levels of allocators Tools compute memory differently and often conservatively Use mach functionality to get details
- Heap growth is sizable steps Sometimes we are short of memory and have MM of space in unused managed heap slack
- Unity uses Boehm-Demers GC Easy enough to write a plugin that externs some GC statics One to note is GC_free_space_divisor that controls how heap will expand Can increase the divisor to have heap expanding less
- Unity is great of exposing GameObject hierarchy Tools can be used to automate finding GameObject leaks It is harder for native and managed memory There is a Unity memory profiler which is improving Allows to compare captures and drill down Custom tools can be build using low level memory API The API provides a snapshot of all memory blocks in native and managed memory
- We had to build a custom tool to help finding C# level leaks Can take a snapshot that compare with previous snapshot based on type names Also built a feature to see which types refer to which types Often times it is not drilling down into individual object but high level view that helps figuring out leaks Here is an example? Perhaps a leak?
- We have access to Boehm statics Exercise caution as with anything low level Thanks to the team