Talk given in the programming track at Develop in Liverpool 2010 on 25th November.

2. Around the World in 80 Shaders
Stephen McAuley
Bizarre Creations
stephen.mcauley@bizarrecreations.com

3. Overview
• Introduction:
– The FX system
• Around the world tour:
– Default
– Skin
– MPEG corruption
– Refraction mapping
– Shallow water
– Aquarium

4. FX System: Problem
• After finishing PGR4 and The Club we needed
a new solution to handling shaders.
– Two different systems.
– Extremely complicated to add new shaders.
• Have to add new vertex declarations, new exporter
code and new rendering code as well.
– Not data driven.
• Shaders treated as code not data.
• Could never run on an SPU.
– No Maya previewing for artists.

5. FX System: Solution
• Our solution was the FX system:
– Based on .fx files.
– Entirely data driven.
– Automatic Maya previews for artists.
– All rendering code now runs on the SPUs.

6. FX System: Maya and XSI
• Compile Maya and XSI shaders alongside PC
shaders.
• Annotations control how the parameters
appear to the artists:
• Artists can “expose” parameters they want to
be set within game.
texture2D g_tAlbedo < string SasUiLabel = “Colour”; >;
float g_fScale < string SasUiLabel = “Scale”; > = 0.0;

7. FX System: Exporting
• Scene converter:
– Reads vertex input structure
• With the help of annotations.
– Exports relevant vertex data in the correct format.
struct C_VertexInput
{
float3 m_vPosition : POSITION;
float3 m_vNormal : NORMAL;
float4 m_vTexCoord : TEXCOORD0
annotation(xy(uv:0) | zw(uv:luxlightmap) | format(F16_4));
};

8. FX System: Exporting
• Lua script within each shader helps build shader
permutations:
• If a bumpmap is present, game will search for
ShaderB.fx instead of Shader.fx.
• Make sure you have compiled that permutation!
texture2D g_tBumpmap < string SasUiLabel = “Bumpmap”; >;
#if defined(FXFILE_PARSER)
if (g_tBumpmap) then append(“B”, 10) end
#endif

9. FX System: In Game
• Game sets exposed and shared parameters.
• Techniques are changed dynamically:
• e.g. shadow mapping technique, prepass technique…
• Shader parameters can be overridden via our
debugging tool:
– Currently only on all instances of a shader.

11. Default
Athens

12. Default Shader
• This is our base material for everything in
game.
• Applied when artists use the Lambert shader
in Maya.
– Easy to set up.
– Artists just set an albedo texture.
– Other textures are picked up in the exporter using
a naming convention.

13. Default Shader: Material
• The following material options are supported:
– Albedo (_DIFF)
– Normal (_NORM)
– Specular (_IPR0)
– Emissive (_NEON)
• These are all controlled by textures.

14. Default Shader: Lighting
• We support these lighting methods:
– Light maps
– Vertex lighting
– Vertex PRT
– SH lighting
• In Blood Stone, they contained the following
information:
– Colour, sun occlusion and ambient occlusion

15. Default Shader: Lighting
Our BRDF for sun lighting:
𝜌 =
𝑅 𝑑
𝜋
1 − 𝐼𝐹 𝐹0 + 𝐼𝐹(𝐹0)
𝑛 + 2
2𝜋
(𝑅. 𝑉) 𝑛
(𝑁. 𝐿)
where 𝑅 𝑑 is the surface albedo and 𝐹(𝐹0) is
Schlick’s fresnel approximation:
𝐹(𝐹0) = 𝐹0 + (1 − 𝐹0)(𝑁. 𝑉)5

16. Default Shader: Specular
We have three parameters controlling specular:
𝜌 =
𝑅 𝑑
𝜋
1 − 𝐼𝐹 𝐹0 + 𝐼𝐹(𝐹0)
𝑛 + 2
2𝜋
(𝑅. 𝑉) 𝑛
(𝑁. 𝐿)
• 𝐼 is the specular intensity
• 𝑛 is the specular power
• 𝐹0 is the normal specular reflectance

17. Default Shader: Specular
• These three specular properties come from
our specular texture:
– Red channel: Intensity (𝐼)
– Green channel: Power (𝑛)
• 𝑛 = 4 + 8188𝑔2
• 𝑚𝑖𝑝𝑀𝑎𝑝𝐿𝑜𝑑 = 7(1 − 𝑔)
– Blue channel: Reflectance (𝐹0)

19. Skin
Istanbul

20. Skin
• Standard diffuse lighting gives unrealistic
results when applied to skin.
• It looks hard and dry compared to skin’s soft
appearance.
• To achieve this look, we need to model
subsurface scattering.

21. Skin
• The best approach is described by Eugene
d’Eon and David Luebke in [1]:
– Observed that red light scatters in skin more than
green and blue.
– Simulate subsurface scattering by lighting in
texture space, then performing separate gaussian
blurs for red, green and blue channels.
• This is expensive and intrusive. Are there any
cheaper ways?

22. Skin: Approximations
• Standard diffuse lighting:
D = N.L
• Wrapped diffuse lighting:
D = N.L * w + (1 – w)
• i.e. D = N.L * 0.5 + 0.5
• What if we wrap different colours by different
amounts?

23. Skin: Our solution
• Coloured wrapped diffuse lighting:
D = N.L * W + (1 – W)
• i.e. D = N.L * { 0.5, 0.8, 0.9 } + { 0.5, 0.2, 0.1 }
• Simulates subsurface scattering by giving skin
a softer look.
• Simulates red light scattering further in skin.
• Cheap and easy to implement.

24. Standard Diffuse

25. Wrapped Diffuse

26. Coloured Wrapped
Diffuse

27. MPEG
Corruption
Monaco

28. MPEG Corruption
• For the AR phone, we wanted an effect that
simulated MPEG corruption.
• “Blocks” in the image don’t update and
remain from the previous frame.

29. MPEG Corruption
• Algorithm:
– For each pixel, point sample a noise texture.
• Scaled so each texel is the size of a “block”.
– If sample is less than a threshold, take pixel from
previous frame buffer.
• Instead of the previous frame buffer, use a scaled and
offset version of the current frame.
– Else, take pixel from current frame buffer.

30. MPEG Corruption
half2 vRUV = vUV * g_vCorruptionBlockSize;
vRUV += half2(g_fCorruptionAmount * 7.123h, 0.0h);
half fRand = h4tex2D(g_tNoise, vRUV).r;
if(fRand < g_fCorruptionAmount)
{
real2 vDUV = input.m_TexCoord * g_vCorruptionUVScale;
vDUV += g_vCorruptionUVOffset;
vColour = h4tex2D(g_tBackbuffer, vDUV).rgb;
vColour *= (1.0h + fRand - 0.5h * g_fCorruptionAmount);
}
else
{
vColour = h4tex2D(g_tBackbuffer, vUV).rgb;
}

34. Refraction
Mapping
Siberia

35. Refraction Mapping
• We wanted to simulate surfaces that are
coated in a layer of ice.
• Requirements:
– These surfaces have two layers:
• Ice layer
• Base layer
– The base layer is refracted by the ice layer.

36. Refraction Mapping
• Our in-game materials use three textures:
– albedo
– normal
– specular
• We split these textures across the two layers.
Ice layer:
• normal
• specular
Base layer:
• albedo
uv
refracted uv

37. Refraction Mapping
• Calculate the refracted texture coordinate:
• Albedo texture uses vRefractUV.
• Normal and specular maps use vUV.
half3 vRefract = refract(vTangentView, vTangentNormal,
1.0f / g_fRefractiveIndex);
float2 vUVOffset = g_fDisplacement * vRefract.xy / vRefract.z;
float2 vRefractUV = vUV + vUVOffset;

38. Default

39. Displacement

40. Refraction

41. Refraction Mapping: Extensions
• Vary the displacement across the surface:
– Height map.
– Vertex colours.
• Add a frost layer in between the top layer and
the base layer.

42. Default

43. Refraction

44. Frost

45. Refraction Mapping: Problems
• Make sure your tangent space is correct!
– Any errors will very quickly become apparent.
• Displacement breaks as you look at the
surface edge on.

46. Shallow
Water
Burma

47. Shallow water
Deep water

48. Shallow Water
• Why does shallow water appear more
transparent than deep water?
– Water molecules absorb and scatter light.
– The further light travels, the higher probability it is
absorbed or scattered.
– Absorption and scattering are dependent on
wavelength:
• e.g. red light is absorbed more than green or blue,
giving water its characteristic colour.

49. Shallow Water
• Premoze and Ashikhmin suggested the
following model for the absorption and
scattering of light in water [2]:
𝐿 0, 𝜃, 𝜑 = 𝐿 𝑍, 𝜃, 𝜑 𝑒−𝑐𝑅 + 𝐿 𝑑𝑓 0 1 − 𝑒 −𝑐+𝐾 𝑑 𝑐𝑜𝑠𝜃 𝑅

50. Shallow Water
𝐿 = 𝐿 𝑧 𝑒−𝑐𝑅
+ 𝐿0(1 − 𝑒−𝑐𝑅
𝑒−𝐾 𝑑 𝐻
)
R
H
LZ
L0
extinction inscattering
c and Kd are wavelength dependent absorption
and scattering coefficients

51. Shallow Water
𝐿 = 𝐿 𝑧 𝑒−𝑐𝑅
+ 𝐿0 1 − 𝑒−𝑐𝑅
𝑒−𝐾 𝑑 𝐻
• Optimisation:
– Remove the 𝑒−𝐾 𝑑 𝐻term.
– Only changes the falloff of the inscattering, not
the intensity.

52. Shallow Water
𝐿 = 𝐿 𝑧 𝑒−𝑐𝑅
+ 𝐿0 1 − 𝑒−𝑐𝑅
• Optimisation:
– Remove the 𝑒−𝐾 𝑑 𝐻term.
– Only changes the falloff of the inscattering, not
the intensity.

53. Shallow Water
𝐿 = 𝐿 𝑧 𝑒−𝑐𝑅
+ 𝐿0 1 − 𝑒−𝑐𝑅
• Optimisation:
– Remove the 𝑒−𝐾 𝑑 𝐻term.
– Only changes the falloff of the inscattering, not
the intensity.
• Visual artefact:
– Very shallow water is slightly darker.

54. Shallow Water
• The final equation is a lerp between the water
colour and the underwater colour, based on
the extinction:
𝐿 = lerp(𝐿0, 𝐿 𝑧, 𝑒−𝑐𝑅
)
• Final shader code:
half3 vExtinction = exp(-g_vExtinctionCoeffs * fDistUnderwater);
half3 vDiffuse = lerp(vSceneColour, vWaterColour, vExtinction);

55. No extinction

56. Red extinction

57. Green/blue extinction

58. Aquarium
Bangkok

59. Aquarium
• Is an aquarium tank equivalent to shallow
water?
R
LZ
L0

60. With Extinction

61. Image courtesy Jon Rawlinson
jonrawlinson.com

62. Aquarium
• Simulate a light positioned directly above the
tank.
• This is our new inscattering term.
R LZ
L0
M0
H
MZ

63. Aquarium
• Inscattering equation:
𝐼 = 𝑀0 𝑒−𝑐1 𝐻
(1 − 𝑒−𝑐0 𝑅
)
• Light colour, 𝑀0, should be the colour of the
light as it hits the surface of the water.
– i.e. light colour modulated by water colour
• Use a different attenuation coefficient to the
extinction.
– Allows for more interesting hue shifts.

64. Aquarium
• Shader code:
half3 vExtinction = exp(-g_vExtinctionCoeffs *
fDistanceThroughWater);
half3 vInscattering = exp(-g_vInscatteringCoeffs *
fDistanceUnderWater);
vInscattering *= g_vLightColour.rgb;
half3 vDiffuse = lerp(vSceneColour, vInscattering, vExtinction);

65. Aquarium
• For the final touch, add light shafts:
– Project two scrolling textures in the world x-z
plane.
• Sample each texture at a fixed distance behind the
glass front of the aquarium tank.
– Combine texture samples into light shaft term, 𝑘.
– Modify inscattering term:
inscattering = 𝑀0 𝑒−𝑐1 𝐻(1−𝑘)
(1 − 𝑒−𝑐0 𝑅
)

66. With Light

67. With Light Shafts

68. References
• [1] Advanced Techniques for Real-Time Skin
Rendering, Eugene d’Eon and David Luebke,
GPU Gems 3, 2008
• [2] Rendering Natural Waters, Simon Premoze
& Michael Ashikhmin, 2001

69. Credits
• For working on the technology described in
this presentation:
– Paul Malin
– Jan van Valburg
– David Hampson