Gears of Washroom – Pt 3

Last post I talked about some of the RBD prep for the grenade explosions, now time to talk about Pyro and Flip prep!

Every Sim in its right place

All of the simulations (RBD, Pyro and FLIP) live in the same DOP network:

AutoDop_fluid
(Click on image to expand)

The network itself isn’t terribly exciting, it was mostly built using by testing out shelf tools in other files, then I’d reconstruct them to try and understand them a bit better. I might dive into it a little later in this blog series.

Now, on to the networks that feed this beast!

Explosion fuel and force

FuelAndForce
(Click on image to expand)

In this network, I’m importing the grenade geometry and using it as a basis for generating fuel for the Pyro simulation, and also forces that affect the FLIP and RBD sims.
Sorry about the sloppy naming and layout of this one, it’s not as readable as it should be.

Splodey time!

Early in the grenades network, I created a primitive attribute called “splodeyTime”, set about 10 frames apart for each grenade.
This attribute drives the timing of all of the explosion related effects.

For example, I use it to set the active time for the RBD pieces in an attribute wrangle:

if (@Frame > (@splodeyTime-2)) i@active = 1;

You can also see it in the first loop in my Explosion Fuel and Force network:

FuelAndForce_foreach1

Here, I’m doing some similar promotion tricks like in the last post:

  • Iterating over pieces (in this case each piece is a grenade base)
  • Promoting splodeyTime up to a detail attribute.
  • Creating a single new point in the centre of the current grenade, using the Add SOP:
    AddPoint
  • Deleting all geometry except for that point.
  • Also delete the point, if:
    • Current frame number < (splodeyTime – 2)
    • Current frame number > (splodeyTime + 2)

So at the end of the foreach, each grenade will be represented by a single point, that will only exist for 4 frames around the splodeyTime of that grenade.

Great metaballs of fire!

The first thing I do with those points, is generate a cigar of metaballs:

MetaballCigar
I’m using this rather dubious looking thing to break constraints on the RBD (in the Remove Broken SOP solver in the DOP network diagram at the top of the post).

There are two more branches that come off the metaball cigar.

MetaballAndPoints

The left branch just inflates each of the metaballs, it is fed into a Magnet Force, and is only used on the RBD objects, to blow the grenade fragments apart.

The right branch is very similar, except after a slightly more aggressive inflation, I’m generating points from the volume of the metaballs, and adding a velocity attribute to them.

The points, that you can see in the image, are used to drive force in the water that explodes out of the grenades, in the FLIP fluid.

One thing I wanted to do here was to have fluid that is near the wall shoot out pretty straight, but then add a bit more randomness when the fluid is near the chain end of the grenade.

Not the best way to visualize it, but these are the force vectors for that effect:

FluidForceVectors

To start out, I set the velocity attribute to {10,0,0} in an attribute wrangle.

Then, the randomness is achieved in the creatively named attribvop1 in the right branch a few images back, and here is what that looks like:

ForceVectorVOP.png

Stepping through this network a bit…
I’m creating a new noisey velocity, and mixing that with the existing straight down X velocity.

I always want the noisey velocity to be positive in X, but to be much stronger in Y and Z, hence the splitting out of the X component and the abs after the bottom noise.

The mix node uses the X position of the point to work out how much of the straight velocity to use versus the noisey velocity.
So between 0 – 5 in X, I’m blending the two. When the position is greater than 5, we have 100% of the noisey vector.

I messed about with lots of different vector / noise patterns, but I liked the movement this gave my fluid, so I stuck with this ūüôā

Fuel

Backing up a bit, there’s one more branch of this network, and it creates the fuel for my Pyro simulation:

FuelNetwork

I’m copying a sphere onto the explosion points, and then using¬†a fluid source SOP to create a volume for fuel, and using the built in noise settings to get some variation.

The sphere itself is getting scaled by splodeyTime, using copy stamping in uniform scale:

fit((stamp(“../copy2/”, “splodeyTime”, 1) – $F), -2, 2, 4, 10)

I also move it forward each frame using an attribute wrangle:

@P.x += ((@Frame+4) – @splodeyTime)*2;

This is what the fuel looks like over the few frames it is generated:

Fuel

Flipping glass

One last thing for simulation prep, I’m using the glass sections of the grenade as the fluid source of the exploding Flip sim.

Again, this uses the splodeyTime to delete the glass when it is outside of the frame range, so I’m creating the surface volume and particles just for a few frames:

FlipParticlesAndSurface

That’s pretty much each for simulation prep, at least for the main explosion!

Still to come, the bubbling water at the start, and all things materials and rendering.

Advertisements

Gears of Washroom – Pt 2

Last post I talked about the static scene setup for the above video, this post focuses on the RBD preparation and setup for the explosion.

It’s pretty much Houdini from this post onwards, here is the top level layout of my scene:

SceneView

Complete with a “TODO” section on the left that I forgot to clear out ūüėČ

The grenades nodes in the middle are where most of the prep work for the RBD is done, so the internals of those will be the focus for this post.

Import once, fracture twice

Although I probably should have just manually rotated and placed the chains, I decided to do an RBD sim to hang the chains in place, and over the wooden shelf.

GrenadesSimmed

I couldn’t get Concave collision to work with the chains.
They would sim ok for a handful of frames, and then generally start intersecting, or explode apart, or do any number of annoying things.

I had a chat to one of our VFX Artists at work, Mandy Morland, who does all sorts of cool things in Houdini.
She suggested I could fracture the chains, and then simulate the chain links as clusters of Convex Hulls, which was a super awesome idea.

As long as each new fracture piece in a single chain link has the same value in the name attribute, it can be assembled and passed to DOPs as a single object.

PresimFracturing_Chain

So, for example, all of the highlighted sections above would have the same name attribute value “piece2”.

However, the name attribute is created before the fracture, and the fracture creates new internal faces which will not have the correct name attribute!

ChainNewFaces

The above image visualises this problem with colour as a primitive attribute.
The pieces have a green colour applied to them before fracture, and after fracture the internal pieces can’t know what colour to have, so they get initialized to default (I’ve then coloured them blue to make it more obvious).

To avoid this issue, I’m looping over chain pieces, promoting the piece attribute to Detail, then fracturing the chain, then promoting the piece attribute back to Primitive:

PresimFracturing
(The piece attribute is just temp storage for Name).

This is a Houdini trick I do a lot of, in general form:

  • Loop over geometry pieces that I want to share an attribute value
  • At the start of the loop, promote an attribute to Detail.
    Only the current piece of geometry will be affected by this, Detail is essentially per piece at this point.
  • Do the things I want to do in the loop that generate new geometry, or invalidate the attribute in some way I don’t want
  • Promote the attribute back from Detail to whatever context it was before (primitive, point, etc)

Side note on fracturing in a loop…

Fracturing inside a loop isn’t the best idea, it’s going to be slower than running fracture once on a single geometry object.

When fracturing geometry that represents combined objects, and that has per-object attributes you want to keep (eg: a string attribute called “chainID”), a better way might be:

  • Fracture the geometry (new faces will have a blank chainID).
  • Use Connectivity on Primitives to get a attribute (let’s call it Bob)
  • Do a foreach loop on Bob:
    • Promote chainID to Detail (using “maximum”)
    • Promote chainID back to Primitive.

Very similar, but at least it gets the fracture out of the loop.

In the end, node network performance wasn’t too big an issue for me: I go crazy with cache nodes (184 GB for this project), so I only had to wait for it to finish once, and it was still pretty quick for such simple geometry.

Hanging out

With that done, the chains were ready for the hanging sim, I just needed to make sure the bullet sim was creating convex hulls for connected prims:

ChainsConvex

ChainSettle

Back in my Grenades_fractured network, I’m importing the base un-fractured geometry, and then importing the hanging simulation with a dopimport node, with an Import Style set to “Transform Input Geometry”, and using timeshift to pick a frame I like.

So now I have the base un-fractured geometry in a good starting pose:

PosedGrenades

With the grenades now in the right pose, I fracture them again, ready to be blown up!
They don’t get made active straight away, but more on that, explosions and fluids in the next blog post ūüôā

Gears of Washroom – Pt 1

I wanted to do a project that focuses a bit more on simulation work in Houdini, and also rendering, and this is the result. Probably one of the sillier things I’ve done in a while, I suppose ūüôā

Originally, I was going to render it in Renderman, but I settled on using Mantra (which I’m really starting to love).
Renderman in Houdini was just a little too much to deal with, on top of all the other things I was learning.

Modelling

All of the models in the scene are edge weighted sub-d, modelled in Modo.
The water grenade is Gears Of War inspired, cobbled together from whatever concept I could find off the interwebz ūüôā

In a previous post, I showed off the water grenade model when I was talking about transferring edge weighted sub-d models from Modo to Houdini.

After posting it on this great Modo / Houdini thread, Pascal Beeckmans (aka PaQ WaK) informed me that Alembic files keep edge weight values.
Using Alembic files is a much better idea than the silly workaround hack I was doing!

Toilet paper dispenser model

Water grenade sub-d model in Modo

Toilet model

The whole base geometry for the scene (room objects + one grenade) comes in under 20000 triangles, such is the joy of sub-d in Modo.

Grenade materials

The material for the grenade was made in Substance Painter:

Grenade - Substance Painter texturing

GrenadeSubstance_Close

I tried out a few different colour schemes, but settled on eye searing orange.

Something I hadn’t used much in Substance Painter is the “Hard Surface” stamping feature.
This is a really cool way of adding little details that I couldn’t be bothered modelling:

Hard surface stamps on Grenade model in Substance Painter

Substance Painter comes with quite a few to choose from:

GrenadeSubstance_HardSurfaceStamps

I can imagine that if you built up a library of them, you could detail up models super quick!

Designer fun

I decided to do the walls, floor and wood shelf materials in Substance Designer.

Tiles, wood and plaster wall materials

I won’t go through all the networks node by node, but I’ll do a bit of an overview of the Wood Substance, since it’s the slightly more interesting of the three.

Wood Substance

Substance Designer network for wood material
Click for larger view

I’m taking an anisotropic noise, warping it with a crystal noise, taking a creased noise, warping the rest with that, making some gaussian spots, converting them to a normal, vector warping everything with that.

That gives me a greyscale image, that I gradient map to make a diffuse texture.
In case the graph, and that last sentence weren’t confusing enough, here it is in gif form!

WoodTexture

I always find it a bit to talk through Substance Designer networks, because so much of it is fiddling around until you have something you like.
I could probably remake this a lot better, and remove more than half of the nodes!

One really fun part of this was the Gradient ramp right at the end.

In the Gradient node, you can click and drag over anything on your screen (google image search images, in my case) to pick a gradient.
Here’s a great video explaining it:

I ran the picker over a photo of a wood plank that I liked, and then manually cleaned up the points on the gradient a bit:

WoodGradient.png

Setting the scene

Having exported the scene from Modo as Alembic, I’m loading all the parts of the scene separately, and creating Groups for them.

AlembicImport

I just noticed that under the Attributes tab in the Alembic node, there is the option to “Add Path Attribute”, so using that for grouping would be the smarter and neater way go!

The UV layout node in the middle was just me messing around with some of the new packing features in 16.5.
I’d UV’d in Modo already, but I wanted to see how the layout node fills in holes:

H16_5_Unwrap

Turns out it’s pretty great!

The last section of this network, I’m setting up the length of the chain by copying and rotating the one chain piece, and offsetting the end handle part:

GrenadeChainLength

To keep the handle at the end of the chain with a transform node, I can just reference the transform and number of copies properties from the copy1 node.

So the x translation is:

ch(“../copy1/ncy”) * ch(“../copy1/tx”)

And to get the handle at the right 90 degree rotation:

(ch(“../copy1/ncy”) % 2) * 90

It’s nothing exciting, but it’s great how easy it is just to dump expressions into parameters just about anywhere in Houdini.

For the room geometry, the import setup is very similar to the grenade setup.
One thing probably worth pointing out: I’m subdividing the assets at render time.
So although they are not subdivided in the viewport, you’ll just have to trust me that all the edge weighting came in fine ūüôā

FullSceneSetup.png

In the next blog post, I’ll start getting into some of the simulation setup.
From here on, the focus of these posts on this project will be 100% on Houdini.

Edge weighted sub-d: Modo –> Houdini

ModoToHoudini

I don’t need to get edge weights from Modo into other applications much.
Last time I tried was Modo sub-d with edge weights -> Mudbox (this short sword).
I had some issues with it, but it mostly worked ok (I think I used fbx 2013).

This time around, I have an object I want to take from Modo to Houdini.

Failbox

I didn’t have the same luck as I did with Mudbox back in the day.
I don’t think Houdini imports crease (edge weighting) info from fbx. I probably could have tried another format, though. Or maybe there is some checkbox I missed somewhere ūüôā

In the end, I found another fairly painless way of doing it!

Edge selection to UVs

When modelling in Modo, I usually create an edge selection group from all edges of the same edge weight:

hardedges

For this exporting trick, having this group makes life easy.
Also, only having 1 set of edge weights helps (i.e everything is either 30% or 0%), since we’d need a new UV set per edge group (more on that later).

So, with all those edges selected, I run the unwrap tool:

Unwrap

Export the mesh with the new unwrap to fbx, and then into Houdini.

Here is what my geometry network looks like (merged the fbx into the scene, and re-arranged a bit):

AddEdgeWeightsNetwork.png

So what I love about this is that it ends up being really simple:

  1. Split the vertices of the mesh by the uv attribute
  2. Group unshared edges (this gets the open edges that were created by the vertex split, which basically gets us back the Modo hard edge selection set).
  3. Set the crease weight values for edges in the group
  4. Copy the crease weights back onto the non split mesh with a wrangle
    (@creaseweight = @opinput1_creaseweight)
  5. Subdivide

On the “crease” node, I just dragged the value slider around until it matched Modo (a value of 3 seemed to do it).

If I had edges weighted at different values, I’d need to create a UV set for each unique value, and do the vertex split + group + crease for each UV set.

HoudiniFinal

I’m still hanging on to the dream of more companies adopting edge weighted sub-d in games.
Creased sub-d is coming to UE4 someday, so that’s exciting ūüėõ

In the meantime, at least I have a pretty easy/slightly hacky workaround!

 

Subsurface Scattering spherical harmonics – pt 3

Welcome to part 3 of this exciting series on how to beat a dead horse.

By the time I got to the end of the work for the last post, I was just about ready to put this project to bed (and by that, I mean P4 obliterate…).

There was just one thing I wanted to fix: The fact that I couldn’t rotate my models!
If I rotate the object, the lighting rotates with it.

Spaaaaaaace

To fix the rotating issue, in the UE4 lighting pass, I need to transform the light vector into the same space that I’m storing the SH data (object space, for example).

RotateSpace

To do that, I need to pass through at least two of those object orientation vectors to the lighting pass (for example, the forward and right vectors of the object).

So, that’s another 6 floats (if I don’t compress them) that I need to pass through, and if you remember from last time, I’d pushed the limits of MRTs with my 16 spherical harmonics coefficients, I don’t have any space left!

This forced me to do one of the other changes I talked about: Use 3 band Spherical Harmonics for my depth values instead of 4 band.
That reduces the coefficients from 16 to 9, and gives me room for my vectors.

<Insert montage of programming and swearing here>

3bandSH

So yay, now I have 3 band SH, and room for sending more things through to lighting.

Quality didn’t really change much, either, and it helped drop down to 5 uv channels, which became very important a little later…

Going off on a tangent

I figured that since I was solving the problem for object orientation, maybe I could also do something for deforming objects too?
For an object where the depth from one side to the other doesn’t change much when it’s deforming, it should be ok to have baked SH data.

The most obvious way to handle that was to calculate and store the SH depth in Tangent space, similar to how Normal maps are usually stored for games.

I wanted to use the same tangent space that UE4 uses, and although Houdini 15 didn’t have anything native for generating that, there is a plugin!

https://github.com/teared/mikktspace-for-houdini

With that compiled and installed, I could plonk down a Compute Tangents node, and now I have Tangents and Binormals stored on each vertex, yay!

At this point, I create a matrix from the Tangent, Binormal and Normal, and store the transpose of that matrix.
Multiplying a vector against it will give me that vector in Tangent space. I got super lazy, and did this in a vertex wrangle:

matrix3 @worldToTangentSpaceMatrix;
vector UE4Tang;
vector UE4Binormal;
vector UE4Normal;

// Tangent U and V are in houdini coords
UE4Tang         = swizzle(v@tangentu, 0,2,1);
UE4Binormal     = swizzle(v@tangentv, 0,2,1);
UE4Normal       = swizzle(@N, 0,2,1);

@worldToTangentSpaceMatrix = transpose(set(UE4Tang, UE4Binormal, UE4Normal));

The swizzle stuff is just swapping Y and Z (coordinate systems are different between UE4 and Houdini).

Viewing the Tangent space data

To make debugging easier, at this point I made a fun little debug node that displays Tangents, Binormals and Normals the same as the model viewer in UE4.

It runs per vertex, and creates new coloured line primitives:

TangentFace

Haven’t bothered cleaning it up much, but hopefully you get the idea:

TangentPrimsVOP.png

And the vectorToPrim subnet:

VectorToPrimsVOP.png

So, add a point, add some length along the input vector and add another point, create a prim, create two verts from the points, set the colour.
I love how easy it is to do this sort of thing in Houdini ūüôā

The next step was to modify the existing depth baking code.

For each vertex in the model, I was sending rays out through the model, and storing the depth when they hit the other side.
That mostly stays the same, except that when storing the rays in the SH coefficients, I need to convert them to tangent space first!

HitsToSH.png

Getting animated

Since most of the point of a Tangent space approach was to show a deforming object not looking horrible, I needed an animated model.

I was going to do a bunch of animation in Modo for this, but I realized that transferring all my Houdini custom data to Modo, and then out to fbx might not be such a great idea.

Time for amazing Houdini animation learningz!!
Here’s a beautiful test that any animator would be proud of, rigged in Houdini and dumped out to UE4:

StupidTube.gif

So, I spent some time re-rigging the Vortigaunt in Houdini, and doing some more fairly horrible animation that you can see at the top of this post.

RiggedVort.png

Although the results aren’t great, I found this weirdly soothing.
Perhaps because it gave me a break from trying to debug shaders.

At some point in the future, I would like to do a bit more animation/rigging/skinning.
Then I can have all the animators at work laugh at my crappy art, in addition to all the other artists…

Data out

Hurrah, per-vertex Tangent space Spherical Harmonic depth data now stored on my animated model!

This was about the part where I realized I couldn’t find a way to get the Tangents and Binormals from the Houdini mesh into Unreal…

When exporting with my custom data, what ends up in the fbx is something like this:

   UserDataArray:  {
    UserDataType: "Float"
    UserDataName: "tangentu_x"
    UserData: *37416 {...

When I import that into UE4, it doesn’t know what that custom data is supposed to be.

If I export a mesh out of Modo, though, UE4 imports the Tangents and Binormals fine.
So I jumped over into Modo, and exported out a model with Tangents and Binormals, and had a look at the fbx.
This showed me I needed something more like this:

LayerElementTangent: 0 {
 Version: 102
 Name: "Texture"  
 MappingInformationType: "ByPolygonVertex"
 ReferenceInformationType: "Direct"
 Tangents: *112248 {...
This is probably around about when I should have set the project on fire, and found something better to do with my time but…

C# to the rescue!!

I wrote an incredibly silly little WPF program that reads in a fbx, changes tangentu and tangentv user data into the correct layer elements.

Why WPF you ask?
Seriously, what’s with all the questions? What is this, the Spanish inquisition?
Real answer: Almost any time I’ve written any bit of code for myself in the past 7 years, it’s always a WPF program.
80% of them end up looking like this:
AmazingUI
The code is horrible, I won’t paste it all, but I build a list of all the vectors then pass them through to a function that re-assembles the text and spits it out:
        public string CreateLayerElementBlock(List<Vector3D> pVectors, string pTypeName)
        {
            string newBlock = "";

            int numVectors  = pVectors.Count;
            int numFloats   = pVectors.Count * 3;

            newBlock += "\t\tLayerElement" + pTypeName + ": 0 {\n";
            newBlock += "\t\t\tVersion: 102\n";
            newBlock += "\t\t\tName: \"Texture\"\n";
            newBlock += "\t\t\tMappingInformationType: \"ByPolygonVertex\"\n";
            newBlock += "\t\t\tReferenceInformationType: \"Direct\"\n";
            newBlock += "\t\t\t" + pTypeName + "s: *" + numFloats + " {\n";
            newBlock += "\t\t\t\ta: ";
	...

Gross. Vomit. That’s an afternoon of my life I’ll never get back.
But hey, it worked, so moving on…

UE4 changes

There weren’t many big changes on the UE4 side, just the switching over to 3 band SH, mostly.

One really fun thing bit me in the arse, though.
I’d been testing everything out on my static mesh version of the model.
When I imported the rigged model, I needed to change the material to support it:
UseWithSkeletal
And then the material failed to compile (and UE4 kept crashing)…
So, apparently, skinned meshes use a bunch of the UV coordinate slots for… Stuff!
I needed to switch back to my old approach of storing 6 coefficients in TexCoord1,2 and 3, and the remaining three SH coeffs in vertex colour RGB:
RiggedMatChanges.png
Cropped this down to exclude all the messy stuff I left in for texture based SH data, but those three Appends on the right feed into the material pins I added for SH data in the previous posts.
And yeah, there’s some redundancy in the math at the bottom too, but if you don’t tell anyone, I won’t.

Shader changes

Now to pass the Tangent and Binormal through to the lighting pass.

I ended up compressing these, using Octahedron normal vector encoding, just so I could save a few floats.
The functions to do this ship with UE4, and they allow me to pass 2 floats per vector, rather than x,y,z, and the artifacts are not too bad.
Here’s some more information on how it works:
OctahedronEncoding.png
So now the Tangent and Binormal data is going through to the lighting pass, and I transform the light to tangent space before looking up the SH data:
 float3x3 TangentToWorld =
 {
  GBuffer.WorldTangent,
  GBuffer.WorldBinormal,
  cross(GBuffer.WorldTangent, GBuffer.WorldBinormal),
 };

 float3 TangentL = mul(L, transpose(TangentToWorld));

 float DepthFromPixelToLight  = saturate(GetSH(SHCoeffs, TangentL));
Probably could do that transposing in BassPassPixelShader I guess, and save paying for it on every pixel for every light, but then there’s a lot of things I probably could do. Treat my fellow human beings nicer, drink less beer, not stress myself out with silly home programming projects like this…

Conclusion

If I were to ever do this for real, on an actual game, I’d probably build the SH generation into the import process, or perhaps when doing stuff like baking lighting or generating distance fields in UE4.

If you happened to have a bunch of gbuffer bandwidth (i.e, you had to add gbuffers for something else), and you have a lot of semi translucent things, and engineering time to burn, and no better ideas, I suppose there could be a use for it.
Maybe.

Subsurface Scattering spherical harmonics ‚Äď pt 2

 

This is my 2nd blog post on using spherical harmonics for depth based lighting effects in Unreal 4.

The first blog post focused on generating the spherical harmonics data in Houdini, this post focuses on the Unreal 4 side of things.

I’m going to avoid posting much code here, but I will try to provide enough information to be useful if you choose to do similar things.

SH data to base pass

The goal was to look up the depth of the object from each light in my scene, and see if I could do something neat with it.

In UE4 deferred rendering, that means that I need to pass my 16 coefficients from the material editor ‚Äď> base pass pixel shader -> the lighting pass.

First up, I read the first two SH coefficients out of the red and green vertex colour channels, and the rest out of my UV sets (remembering that I kept the default UV set 0 for actual UVs):

SHBaseMatUVs

Vertex colour complications

You notice a nice little hardcoded multiplier up there… This was one of the annoyances with using vertex colours: I needed to scale the value of the coefficients¬†in Houdini to 0-1, because vertex colours are 0-1.

This is different to the normalization part I mentioned in the last blog post, which was scaling the depth values before encoding them in SH. Here, I’m scaling the actual computed coefficients. I only need to do this with the vertex colours, not the UV data,¬†since UVs¬†aren’t restricted to 0-1.

The 4.6 was just a value that worked, using my amazing scientific approach of “calculate SH values for half a dozen models of 1 000 – 10 000 vertices, find out how high and low the final sh values go, divide through by¬†that number +0.1”. You’d be smarter to use actual math to find the maximum range for coefficients for normalized data sets, though… It’s probably something awesome like 0 –> 1.5 pi.

Material input pins

Anyway, those values just plug into the SH Depth Coeff pins, and we’re done!!

Unreal 4 SH depth material

Ok.
That was a lie.
Those pins don’t exist usually… And neither does this shading model:

SHDepthShadingModel

So, that brings me to…

C++ / shader side note

To work out how to add a shading model, I searched the source code for a different shading model (hair I think), and copied and pasted just about everything, and then went through a process of elimination until things worked.
I took very much the same approach to the shader side of things.

This is why I’m a Tech Artist, and¬†not a programmer… Well, one of many reasons ūüėČ
Seriously though, being able to do this is one of the really nice things about having access to engine source code!

The programming side of this project was a bunch of very simple changes across a wide range of engine source files, so I’m not going to post much of it:

P4Lose

There is an awful lot of this¬†code that really should be data instead. But Epic gave me an awesome engine and lets me mess around with source code, so I’m not going to complain too much ūüėõ

Material pins (continued…)

So I added material inputs for the coefficients, plus some absorption parameters.

Sh coeffs

The SH Coeffs material pins are new ones, so I had to make a bunch of changes to material engine source files to make that happen.
Be careful when doing this: Consistent ordering of variables matters in many of these files. I found that out the easy way: Epic put comments in the code about it ūüôā

Each of the SH coeffs material inputs is a vector with 4 components, so I need 4 of these to send my 16 coefficients through to the base pass.

Custom data (absorption)

The absorption pins you might have noticed from my material screenshot are passed as “custom data”.
Some of the existing lighting models (subsurface, etc) pass additional data to the base pass (and also through to lighting, but more on that later).

These “custom data” pins can be renamed for different shading models. So you can use these¬†if you’d rather not go crazy adding new pins, and you’re happy with passing through just two extra float values.
Have a look at MaterialGraph.cpp, and GetCustomDataPinName¬†if that sounds like a fun time ūüôā

Base pass to lighting

At this point, I’d modified enough code that I could start reading and using my SH values in the base pass.

A good method for testing if the data was valid was using the camera vector to look up the SH depth values. I knew things were working when I got similar results to what I was seeing in Houdini when using the same approach:

BasePassDebug

That’s looking at “Base Color” in the buffer visualizations.

I don’t actually want to do anything with the SH data in the base pass, though, so the next step is to pass the SH data through to the lighting pass.

Crowded Gbuffer

You can have a giant parameter party, and read all sorts of fun data in the base pass.
However, if you want to do per-light stuff, at some point you need to write all that data into a handful of full screen buffers that the lighting pass uses. By the time you get to lighting, you don’t have per object data, just¬†those full screen buffers and your lights.

These gbuffers are lovingly named GBufferA, GBufferB, GBuffer… You get the picture.

You can visualize them in the editor by using the various buffer visualizers, or explicitly using the¬†“vis” command, e.g: “vis gbuffera”:

visGbuffers

There are some other buffers being used (velocity, etc), but these are the ones I care about for now.

I need to pass an extra 16 float values through to lighting, so surely I could just add 4 new gbuffers?

Apparently not, the limit for simultaneous render targets is 8 ūüôā

I started out by creating 2 new render targets, so that covers half of my SH values, but what to do with the other 8 values?

Attempt 1 – Packing it up

To get this working, there were things that I could sacrifice from the above existing buffers to store my own data.

For example, I rarely use Specular these days, aside from occasionally setting it to a constant, so I could use that for one of my SH values, and just hard code Specular to 1 in my lighting pass.

With this in mind, I overwrote all the things I didn’t think I cared about for stylized translucent meshes:

  • Static lighting
  • Metallic
  • Specular
  • Distance field anything (I think)

Attempt 2 – Go wide!

This wasn’t really ideal. I wasn’t very happy about losing static lighting.

That was about when I realized that although I couldn’t add any more simultaneous render targets, I could change the format of them!

The standard g-buffers are 8 bits per channel, by default. By going 16 bit per channel, I could pack two SH values into each channel, and store all my SH data in my two new g-buffers without the need for overwriting other buffers!

Well, I actually went with PF_A32B32G32R32F, so 32 bits per channel because I’m greedy.

It‚Äôs probably worth passing out in horror at the cost of all this at this point: 2 * 128bit buffers is something like 250mb of data. I‚Äôm going to talk about this a little later ūüôā

Debugging, again

I created a few different procedural test assets in Houdini with low complexity as test cases, including one which I deleted all but one polygon as a final step, so that I could very accurately debug the SH values ūüôā

On top of that, I had a hard coded matrix in the shaders that I could use to check, component by component, that I was getting what I expected when passing data from the base pass to lighting, with packing/unpacking, etc:

const static float4x4 shDebugValues = 
{
	0.1, 0.2, 0.3, 0.4,
	0.5, 0.6, 0.7, 0.8,
	0.9, 1.0, 1.1, 1.2,
	1.3, 1.4, 1.5, 1.6
};

It seems like an obvious and silly thing to point out, but it saved me some time ūüôā

Here are some of my beautiful procedural test assets (one you might recognize from the video at the start of the post):

Houdini procedural test asset (rock thing)testobject3testobject2testobject1

“PB-nah”, the lazy guide to not getting the most out of my data

Ok, SH data is going through to the lighting pass now!

This is where a really clever graphics programmer could use if for some physically accurate lighting work, proper translucency, etc.

To be honest, I was pleasantly surprised that anything was working at this stage, so I threw in¬†a very un-pbr scattering, and called it a day! ūüôā

float3 SubsurfaceSHDepth( FGBufferData GBuffer, float3 L, float3 V, half3 N )
{
	float AbsorptionDistance 	= GBuffer.CustomData.x;
	float AbsorptionPower 		= lerp(4.0f, 16.0f, GBuffer.CustomData.y);

	float DepthFromPixelToLight 	= Get4BandSH(GBuffer.SHCoeffs, L);
	float absorptionClampedDepth 	= saturate(1.0f / AbsorptionDistance * DepthFromPixelToLight);
	float SSSWrap 			= 0.3f;
	float frontFaceFalloff 		= pow(saturate(dot(-N, L) + SSSWrap), 2);

	float Transmittance 		= pow(1 - absorptionClampedDepth, AbsorptionPower);

	Transmittance *= frontFaceFalloff;

	return Transmittance * GBuffer.BaseColor;
}
It’s non view dependent scattering, using the SH depth through the model towards the light, then dampened by the absorption distance.
The effect falls off by face angle away from the light, but I put a wrap factor on that because I like the way it looks.
For all the work I’ve put into this project, probably the least of it went into the actual lighting model, so I’m pretty likely to change that code quite a lot ūüôā
What I like about this is that the scattering stays fairly consistent around the model from different angles:
GlowyBitFrontGlowyBitSide
So as horrible and inaccurate and not PBR as this is, it matches what I see in SSS renders in Modo a little better than what I get from standard UE4 SSS.

The End?

Broken things

  • I can’t rotate my translucent models at the moment ūüėõ
  • Shadows don’t really interact with my model properly

I can hopefully solve both of these things fairly easily (store data in tangent space, look at shadowing in other SSS models in UE4), I just need to find the time.
I could actually rotate the SH data, but apparently that’s hundreds of instructions ūüôā

Cost and performance

  • 8 uv channels
  • 2 * 128 bit buffers

Not really ideal from a memory point of view.

The obvious optimization here is to drop down to 3 band spherical harmonics.
The quality probably wouldn’t suffer, and that’s 9 coefficients rather than 16, so I could pack them into one of my 128 bit gbuffers instead of two (with one spare coefficient left over that I’d have to figure out).

That would help kill some UV channels, too.

Also, using 32 bit per channel (so 16 bits per sh coeff) is probably overkill. I could swap over to using a uint 16 bits per channel buffer, and pack two coefficients per channel at 8 bits each coeff, and that would halve the memory usage again.

As for performance, presumably evaluating 3 band spherical harmonics would be cheaper than 4 band. Well, especially because then I could swap to using the optimized UE4 functions that already exist for 3 band sh ūüôā

Render… Differently?

To get away from needing extra buffers and having a constant overhead, I probably should have tried out the new Forward+ renderer:

https://docs.unrealengine.com/latest/INT/Engine/Performance/ForwardRenderer/

Since you have access to per object data, presumably passing around sh coefficients would also be less painful.
Rendering is not really my strong point, but my buddy Ben Millwood has been nagging me about Forward+ rendering for years (he’s writing his own renderer http://www.lived3d.com/).

There are other alternatives to deferred, or hybrid deferred approaches (like Doom 2016’s clustered forward, or Wolfgang Engels culled visibility buffers) that might have made this easier too.
I very much look forward to the impending not-entirely-deferred future ūüôā

Conclusion

I learnt some things about Houdini and UE4, job done!

Not sure if I’ll keep working on this at all, but it might be fun to at least fix the bugs.

 

Subsurface Scattering spherical harmonics – pt 1

In this post, I’ll be presenting “SSSSH”, which¬†will be¬†the sound made by any real programmer who happens to accidentally read this…

This has been a side project of mine for the last month or so with a few goals:

  • Play around more with Houdini (I keep paying for it, I should use it more because it’s great)
  • Add more gbuffers to UE4, because that sounds like a useful thing to be able to do and understand.
  • Play around with spherical harmonics (as a black box) to understand the range and limitations of the technique a bit better.
  • Maybe accidentally make something that looks cool.

Spherical harmonics

I won’t go too much into the details on spherical harmonics because:
a) There’s lots of good sites out there explaining them and
b) I haven’t taken the time to understand the math, so I really don’t know how it works, and I’m sort of ok with that for now ūüėõ

But at my basic understanding level, spherical harmonics is a way of representing data using a set of functions that take spherical coordinates as an input, and return a value. Instead of directly storing the data (lighting, depth, whatever), you work out a best fit of these functions to your data, and store the coefficients of the functions.

Here is a very accurate diagram:

DataSphere

You’re welcome!
Feel free to reuse that amazing diagram.

SH is good for data that varies rather smoothly, so tends to be used for ambient/bounced lighting in a lot of engines.

The function series is infinite, so you can decide how many terms you want to use, which determines how many coefficients you store.

For this blog post, I decided to go with 4-band spherical harmonics, because I’m greedy and irresponsible.
That’s 16 float values.

Houdini SH

Thanks to the great work of Matt Ebb, a great deal of work was already done for me:

http://mattebb.com/weblog/spherical-harmonics-in-vops/

I had to do a bit of fiddling to get things working in Houdini 15, but that was a good thing to do anyway, every bit of learning helps!

What I used from Matt were two nodes for reading and writing SH data given the Theta and Phi (polar and azimuthal) angles:

SHFunctions

Not only that, but I was able to take the evaluate code and adapt it to shader code in UE4, which saved me a bunch of time there too.

It’s not designed to be used that way, so I’m sure that it isn’t amazingly efficient. If I decide to actually keep any of this work, I’ll drop down to 3 band SH and use the provided UE4 functions ūüôā

Depth tracing in Houdini

I’m not going to go through every part of the Houdini networks, just the meat of it, but here’s what the main network looks like:

NetworkOverview

So all the stuff on the left is for rendering SH coefficients out to textures (more on that later), the middle section is where the work is done, the right hand side a handful of debug modes visualizers, including some from the previously mentioned Matt Ebb post.

Hits and misses

I’m doing this in SOPs (geometry operations), because it’s what I know best in Houdini¬†at the moment, as a Houdini noob¬†ūüôā
I should try moving it to shops (materials/per pixel) at some point, if that is at all possible.

To cheat, if I need more per-pixel like data, I usually just subdivide my meshes like crazy, and then just do geometry processing anyway¬†ūüėõ

The basic functionality is:

  • For each vertex in the source object:
    • Fire a ray in every direction
    • Collect every hit
    • Store the distance to the furthest away primitive that is facing away from the vertex normal (so back face, essentially)

All the hits are stored in an array, along with the Phi and Theta angles I mentioned before, here’s what that intersection network looks like currently:

IntersectAll

I’m also keeping track of the maximum hit length, which I will use later to normalize the depth data. The max length is tracked¬†one level up from the getMaxIntersect network from the previous screenshot:

GenerateHits

This method currently doesn’t work very well with objects with lots of gaps in them, because the gaps in the middle of an object will essentially absorb light when they shouldn’t.
It wouldn’t be hard to fix, I just haven’t taken the time yet.

Normalizing

Before storing to SH values, I wanted to move all the depth values into the 0-1 range, since there are various other places where having 0-1 values makes my life easier later.

One interesting thing that came up here: when tracing rays out from a point, there are always more rays that miss than hit.

That’s because surfaces are more likely to be convex than concave, so at least half of the rays are pointing out into space:

FurryPlane

Realistically, I don’t really care about spherical data, I probably want to store hemispherical data around the inverse normal.
That might cause data problems in severely concave areas of the mesh, but I don’t think it would be too big a problem.
There are hemispherical basis functions that could be used for that, if I were a bit more math savvy:

A Novel Hemispherical Basis for Accurate and EfÔ¨Ācient Rendering

Anyway, having lots of values shooting out to infinite (max hit length) was skewing all of the SH values, and I was losing a lot of accuracy, so I encoded misses as zero length data instead.

Debug fun times!

So now, in theory, I have a representation of object thickness for every vertex in my mesh!

One fun way to debug it (in Houdini) was to read the SH values using the camera forward vector, which basically should give me depth from the camera (like a z buffer):

SHDepth

And, in a different debug mode that Matt Ebb had in his work, each vertex gets a sphere copied onto it, and the sphere is displaced in every direction by the SH value on the corresponding vertex:

vortigauntBalloons

vortigauntBalloons2

This gives a good visual indicator on how deep the object is in every direction, and was super useful once I got used to what I was looking at ūüôā

And, just for fun, here is shot from a point where I was doing something really wrong:

vortigauntClicker

Exporting the data

My plans for this were always to bake out the SH data into textures, partially just because I was curious what sort of variation I’d get out of it (I had planned to use displacement maps on the mesh in Houdini to vary the height).

SHImages
And yes, that’s 4 images worth of SH data, best imported as HDR.
But hey, I like being a bit over the top with my home projects…

One of my very clever workmates, James Sharpe, had the good suggestion of packing the coeffs into UV data as I was whining to him over lunch about the lack of multiple vertex color set support in UE4.
So I decided to run with UVs, and then move back to image based once I was sure everything was working ūüôā

PixelVSVertex

Which worked great, and as you can probably see from the shot above, per-vertex (UVs or otherwise) is perfectly adequate ūüôā

Actually, I ended up putting coefficients 1-14 into uvs, and the last two into the red and green vertex color channels, so that I could keep a proper UV set in the first channel that I could use for textures.

And then, all the work…

Next blog post coming soon!

In it, I will discuss all the UE4 work, the things I should have done, or done better, might do in the future and a few more test shots and scene from in UE4!

To be continued!!