Houdini looping particles

Looping fluid sim

For a while I’d been planning to look into making looping particle systems in Houdini, but hadn’t found a good excuse to jump in. I don’t really do much VFX related work at the best of times, something I need to do more of in the future 🙂

Anyway, I was recently chatting with Martin Kepplinger, who is working on Clans Of Reign, and he was looking to do a similar thing!

So begins the looping particle journey…

Technique overview

I won’t go into the fluid sim setup, it doesn’t really matter too much what it is.

There are a few conditions that make my approach work:

  • Particles must have a fixed lifetime
  • The first chosen frame of the simulation must have a lead up number of frames >= the particle lifetime
  • The last frame of the loop must be >= the first frame number + the particle lifetime

I have some ideas about how to get rid of these requirements, but not sure if I’ll get back to that any time soon.

For the example in this post, I am keeping particle lifetime pretty low (0.8 to 1.0 seconds, using a @life attribute on the source particles, so a 24 frame loop).

The fluid sim I’m using is some lumpy fluid going into a bowl:

Full fluid sim

The simulation is 400 frames long (not all shown here), but that ended up being overkill, I could have got away with a much shorter sim.

Going back to my rules, with particles that live 24 frames I must choose a first frame >= 24 (for this example, I’ll choose 44).
The last frame needs to be after frame 68, so I’m choosing 90.
This makes a loop that is 46 frames long, here it is with no blending:

Looping particle system with no blending

The technique I’m going to use to improve the looping is somewhat like a crossfade.

For this loop from 44 –> 90, I’m modifying the particles in two ways:

  1. Deleting any particles that are spawning after frame 66 (i.e, making sure all particles have died before frame 90)
  2. From frames 66 to 90, copying in all the particles that spawn between frame 20 –> 44.

This guarantees that all the particles that are alive on frame 89 match exactly with frame 44.

To illustrate, this gif shows the unedited loop on the left, and next to it on the right is the loop with no new particles spawned after frame 66 (particles go red on 66):

Particles stopped spawning after frame 66

Next up is the loop unedited on the left, and on the right are the new particles from frames 20 – 44 that I’m merging in from frame 66 onward:

Pre loop particles added to end of the unlooped sim

And now, the unedited loop next to the previous red, green and blue particles combined:

Pre spawn and end spawn particles combined

And finally, just the result of the looping by itself without the point colours:

Final looping particles

One thing that might be slightly confusing about the 2nd gif with the green pre-loop particles is that I’m always spawning particles in the bowl itself to keep the fluid level up, in case you were wondering what that was about 🙂

Setting up the loop

This is the SOPs network that takes the points of a particle sim, and makes it loop:

Full SOPs network for looping particles

The first part of setting up the looping simulation is picking the start and end frames (first and last) of the loop.

I created a subnetwork (FrameFinder, near the top of the above screenshot) that has parameters for two frame numbers, and displays a preview of the two frames you are selecting, so you can find good candidates for looping:

FrameFinder subnetwork preview

The loop setup I chose for the Unity test at the top of the blog was actually a bit different to the range I chose for the breakdowns in the last section.
For Unity, I wanted the shortest looping segment I could, because I didn’t want it to be super expensive (memory wise), so I chose start and end frames 25 frames apart.

You can see that the frames don’t need to match exactly. The main thing I wanted to avoid was having a huge splash in the bowl at the bottom, or over the edge, because that would be hard to make look good in a short loop.

Node parameters

In the screenshot above, you can see that I have First Frame and Last Frame parameters on my FrameFinder network.

I don’t tend to make my blog posts very tutorial-y, but I thought I’d just take the time to mention that you can put parameters on any node in Houdini.

Example:

  • Drop a subnetwork node
  • Right click and select “Parameters and Channels –> Edit Parameter Interface…”:Parameter editing
  • Select a parameter type from the left panel, click the arrow to add the parameter, and set defaults and the parameter name on the right:Edit parameter interface dialog
  • Voila! Happy new parameter:
    floaty

You can then right click on the parameter and copy a reference to it, then use the reference in nodes in the subnetwork, etc.
In the edit parameter interface window, you can also create parameters “From Nodes”, which lets you pick an existing parameter of any node in the sub network to “bubble up” to the top, and it will hook up the references.

If this is new to you, I’d recommend looking up tutorials on Digital Assets (called “HDAs”, used to be called “OTLs”).

I do this all the time on subnetworks like on the FrameFinder, but also to add new fields to a node that contain parts of an expression that would get too big, or that I want to reference from other nodes, etc.

On Wrangles (for example), I find myself adding colour ramp parameters a lot for use within the wrangle code.

FrameFinder

This subnetwork has two outputs: the particles with some Detail Attributes set up, and another output that is a preview of the two frames which I showed before, but here’s what that looks like again:

FrameFinder subnetwork preview

It’s the first time I’ve created multiple outputs from a subnetwork, usually I just dump a “Preview” checkbox as a parameter on the network, but I like this more I think, particularly if I end up turning the whole thing into an HDA.

Here is what the FrameFinder network looks like:

Contents of framefinder subnetwork

In this project, I’m using Detail attributes to pass around a lot of data, and that starts in with the attribcreate_firstAndLastFrame node.

This node creates Detail parameters for each of the frames I chose in the subnet parameters (firstFrame and lastFrame):

Create details attributes for first and last loop frame

Right under the attribCreate node, I’m using two timeshift nodes: one that shifts the simulation to the first chosen frame, and one to the last frame, and then I merge them together (for the preview output). I’ve grouped the lastFrame particles so that I can transform them off to the right to show them side by side, and also giving all the particles in both frames random colours, so it’s a little easier to see their shape.

Time ranges and ages

Back in the main network, straight after the frameFinder subnetwork I have another subnetwork called timeRangesAndAges, which is where I set up all the other Detail attributes I need, here is what is in that subnetwork:

Time Ranges and Ages subnetwork

The nodes in the network box on the right side are used to get the maximum age of any particle in the simulation.
In hindsight, this is rather redundant since I set the max age on the sim myself (you could replace all those nodes with an Attribute Create that makes a maxAge attribute with a value of 24), but I had planned to use it on simulations where particles are killed through interactions, etc 🙂

The first part of that is a solver that works out the max age of any particle in the simulation:

Solver that calculates maximum particle life

For the current frame of particles, it promotes Age from Point to Detail, using Maximum as the promotion method, giving me the maximum age for the current frame.

The solver merges in the previous frame, and then uses an attribute wrangle to get the maximum of the previous frame and current frame value:

@maxAge = max(@maxAge, @opinput1_maxAge);

Right after the solver, I’m timeshifting to the last frame, forcing the solver to run through the entire simulation so that the MaxAge attribute now contains the maximum age of any particle in the simulation (spolier: it’s 24 :P).

I then delete all the points, since all I care about is the detail attribute, and use a Stash node to cache that into the scene. With the points deleted, the node data is < 12 Kb, so the stash is just a convenient way to stop the maxAge recalculating all the time.
If I turn this whole thing into an HDA, I’ll have to rethink that (put “calculate max particle age” behind a button or something).

There are two more wrangle nodes in time ranges and ages.

One of them is a Point wrangle that converts the particle age from seconds into number of frames:

@frameAge = floor(@age/@TimeInc);

And the next is a Detail wrangle that sets up the rest of the detail attributes I use:

// Copy max age from input 1
int maxAge = i@opinput1_maxAge;

// A bunch of helper detail variables referenced later
int loopLength = i@lastFrame - i@firstFrame;
int loopedFrame = (int)(@Frame-1) % (loopLength+1);

i@remappedFrame = i@firstFrame + loopedFrame;

int distanceFromSwap = loopedFrame - loopLength;

i@blendToFrame = max(1, i@remappedFrame - loopLength);
i@numberOfFramesIntoBlend = max(0, maxAge + distanceFromSwap);

When I hit play in the viewport, I want to see just the looping segment of the simulation over and over, so that complicates all this a little.

With that in mind, there are 3 important attributes set up here.

@remappedFrame

If my start frame is 20, for example, and the end frame is 60, I want to see the 20-60 frame segment over and over, hence the wrapping loopedFrame temporary variable.

So if my viewport time slider is on 15, I really want to see frame 34, the value of remappedFrame will be 34. It will always be a number between firstFrame and lastFrame.

@blendToFrame

This takes the remappedFrame, and shifts it back to before the start of the loop.
I only use this value when we hit the last 24 frames of the loop, but I’m clamping it at one just so the timeshift I use later doesn’t freak out.

This attribute will be used in the 2nd part of the technique: combining in pre-loop particles.

@numberOfFramesIntoBlend

When we are getting into the last 24 frames of the loop, this value increases from 0 to 24.
It’s used in the 1st part of the technique to stop spawning particles that have an age less than this value.

Timeshifts and recombining particles

Back to the main network:

Full SOPs network for looping particles

After the timeRangesAndAges node, the network splits: on the left side, I’m timeshifting the simulation using the remappedFrame detail attribute as the frame using this expression:

detail("../timeRangesAndAges/", "remappedFrame", 0)

On the right side I’m time shifting the simulation using the blendToFrame attribute as the frame using this expression:

detail("../timeRangesAndAges/attribwrangle_calcTimeRanges", "blendToFrame", 0)

I’ve colour coded the nodes in the network with the same colours I’ve shown in the gifs in the technique section.

Since I’ve timeshifted the simulation, the detail attributes get time-shifted too.
But I don’t really want that, so I’m attribute transferring the two detail attributes I care about (remappedFrame and numberOfFramesIntoBlend) back onto the remapped sims using Attribute Transfer.

After the attribute transfers, on both sides I’m creating a new point group called inBlendingFrames.

Group expression node for particles in blending range

detail(0, "numberOfFramesIntoBlend", 0) > 0

I probably didn’t need a point group for this, considering every particle is either in or out of this group on a given frame, it just made life easier with the Split node I use on the left.

On the left side, I do a split using inBlendingFrames.
When we’re not in the blending range, we don’t have to do anything to the particles, so that’s the blue colour node.

For both the red and green nodes section, I start with deleting anything not in the inBlendingFrames group.

For the green particles (the pre-loop particles that we’re merging in), we’ve already got the right frame, due to the timeshift up the top.
If we’re on blending frame 2 of the blend (for example), we will still have particles that were spawned 24 frames ago, but we really only want particles that should spawn after the blend starts.
I use an attribute wrangle to clean the older particles up, since I have the frameAge attribute I can use:

if ((@frameAge > i@numberOfFramesIntoBlend))
{
	removepoint(0, @ptnum);
}

Here’s what that looks like for a frame about halfway through the blend.

Pre loop particles with older particles removed

For the red nodes section (where we take the original loop, and delete any particles that start spawning after the blend), I use an attribute wrangle to clean the new particles up:

if (@frameAge < i@numberOfFramesIntoBlend)
{
	removepoint(0, @ptnum);
}

Particle loop end with new particles deleted

So, I merge the red, blue and green particles all together, and we end up with the result I showed in the technique section!

Pre spawn and end spawn particles combined

Here again uncolourised:

Final looping particles

Unity, Alembic and all that jazz

This post is already crazy long, so I’m just going to gloss over the Houdini –> Unity stuff.
If anyone is really interested in those details, I might do another post.

So now that I have a looping particle system, I can use a regular Particle Fluid Surface with default settings, and a polyreduce node to keep the complexity down:

A frame of the looped fluid sim remeshed

I exported the range of frames as an Alembic file, and imported it into Unity with the Alembic plugin.

I threw together a really quick monoBehaviour to play the Alembic stream:

using System.Collections;
using System.Collections.Generic;
using UnityEngine;
using UTJ.Alembic;

[RequireComponent(typeof(AlembicStreamPlayer))]
public class PlayAlembic : MonoBehaviour
{
	public float playSpeed = 0.02f;

	AlembicStreamPlayer sPlayer;

	// Use this for initialization
	void Start ()
	{
		sPlayer = GetComponent();
	}

	// Update is called once per frame
	void Update () 
	{
		sPlayer.currentTime += playSpeed;
		sPlayer.currentTime = sPlayer.currentTime % 1.0f;
	}
}

As a last little thing, I packaged the network up a lot neater, and dumped it in a loop that ran the process on 16 different 46 frame range segments of the original simulation.
The idea being, why try to find good first and last frames when you can just go for a coffee, and come back and have 16 to choose from!

The loops with big splashes definitely don’t work very well (they look like they aren’t looping because lots of particles dying on the same frame), but there are some fun examples in here:

16 different looping segments

Chopped Squabs – Pt 4

 

Last post was about smoke, particles and weird looking ball things.
This post is about the audio for the video.

I wanted to make a sort of windy / underwater muffled sound, and managed to get somewhat close to what I wanted, just using Houdini CHOPs!

Pulse Noise

I decided not to create the audio in the same hip file, since it was already getting a bit busy, and because the data I want to use is already cached out to disk (geometry, pyro sim, etc).

The new hip file just has a small geometry network and a chops network.
Here’s the geometry network:

Pulse amount network

I wanted the audio to peak at times where the pulse was peaking on the tuber bulbs, so the first step was to import the bulbs geometry:

Tuber ball geometry

Next I’m promoting the pulse amount from points to detail, using “sum” as the promotion method (this adds up all the pulse amounts for all points in the bulbs every frame).
I don’t care about the geometry any more, because the sum is a detail attribute, so I delete everything except for a single point.

I had a bit of a hard time working out how to bring the values of a detail attribute into CHOPs as a channel. I think it should be simple to do with a Channel CHOP, but I didn’t succeed at the magic guessing game of syntax for making that work.

Anyway, since importing point positions is easy, I just used an attribute wrangle to copy the pulse sum into the position of my single point:

@P = detail(@opinput1, "pulse");

Audio synthesis fun

I had some idea of what I wanted, and how to make it work, from experiments in other software (Supercollider, PureData, etc).

I found that creating an interesting wind sound could be achieved through feeding noise into a lowpass filter.

I also tried this out in Unity, grabbing the first C# code I could find for filters:
https://stackoverflow.com/questions/8079526/lowpass-and-high-pass-filter-in-c-sharp

Here is a Unity MonoBehaviour I built from that code above:

using System;
using System.Collections;
using System.Collections.Generic;
using UnityEngine;

[RequireComponent(typeof(AudioSource))]
public class PlayerAudio : MonoBehaviour
{
    AudioSource _requiredAudioSource;
    public int _samplerate = 44100;

    private const float _resonance      = 1.0f;
    private float _phase                = 0.0f;
    private float _rampInAmount         = 0.0f;
    private float _frequency            = 800.0f;
    private System.Random _rand1        = new System.Random();
    private float[] _inputHistory       = new float[2];
    private float[] _outputHistory      = new float[3];

    void Start ()
    {
        _requiredAudioSource = GetComponent<AudioSource>();
        AudioClip testClip = AudioClip.Create("Wind", _samplerate * 2, 1, _samplerate, true, OnAudioRead);
        _requiredAudioSource.clip = testClip;
        _requiredAudioSource.loop = true;
        _requiredAudioSource.Play();
    }

    void Update ()
    {
        _rampInAmount = Mathf.Min(_rampInAmount + (Time.deltaTime/2.0f), 1.0f);
    }

    void OnAudioRead(float[] data)
    {
        float c, a1, a2, a3, b1, b2;

        for (int i = 0; i < data.Length; i++)
        {
            // Create a random amplitude value
            double currentRand  = _rand1.NextDouble();
            float amplitudeRand = (float)(currentRand * 2.0f - 1.0f) * _rampInAmount;
            amplitudeRand /= 2.0f;

            // Phase over a few seconds, phase goes from 0 - 2PI, so wrap the value
            float randRange = Mathf.Lerp(-1.0f, 1.0f, (float)currentRand);
            _phase += 1.0f / _samplerate;
            _phase += randRange * 200.0f / _samplerate;
            _phase %= (Mathf.PI * 2.0f);

            float interpolator = (Mathf.Sin(_phase) + 1.0f) / 2.0f;
            _frequency = Mathf.Lerp(100, 200, interpolator);

            // Low pass filter
            c = 1.0f / (float)Math.Tan(Math.PI * _frequency / _samplerate);
            a1 = 1.0f / (1.0f + _resonance * c + c * c);
            a2 = 2f * a1;
            a3 = a1;
            b1 = 2.0f * (1.0f - c * c) * a1;
            b2 = (1.0f - _resonance * c + c * c) * a1;

            float newOutput = a1 * amplitudeRand + a2 * this._inputHistory[0] + a3 * this._inputHistory[1] - b1 * this._outputHistory[0] - b2 * this._outputHistory[1];

            this._inputHistory[1] = this._inputHistory[0];
            this._inputHistory[0] = amplitudeRand;

            this._outputHistory[2] = this._outputHistory[1];
            this._outputHistory[1] = this._outputHistory[0];
            this._outputHistory[0] = newOutput;

            data[i] = newOutput;
        }
    }
}

It’s pretty gross, doesn’t work in webgl, and is probably very expensive.
But if you’re a Unity user it might be fun to throw in a project to check it out 🙂
I take no responsibility for exploded speakers / headphones…

With that running, you can mess with the stereo pan and pitch on the Audio Source, for countless hours of entertainment.

Back to Chops

When testing out Audio in a hip file, I could only get it to play if I use the “Scrub” tab in the audio panel.
You need to point it to a chopnet, and make sure that the chopnet is exporting at least one channel:

Audio panel settings

You should also make sure that the values you output are clamped between -1 and 1, otherwise you’ll get audio popping nastiness.

The CHOP network I’m using to generate the audio looks like this:

Audio generation chopnet

What I’d set up in Unity was a channel of noise that is run through a High Pass filter with a varying Cutoff amount.
I tried to do exactly the same thing in Houdini: Created a noise chop, a Pass Filter chop, and in the Cutoff field sample the value of the noise chop using an expression:

chop("chan1")

I was hoping that the whole waveform would be modified so I could visualize the results in Motion FX View, but what I found instead is that the waveform was modified by the current frames noise value in Motion FX.
With my limited knowledge of CHOPs, it’s hard to describe what I mean by that, so here’s gif of scrubbing frames:

Scrubbing through frames with high pass

It’s likely that it would still have given me the result I wanted, but having the wave form jump around like that, and not being able to properly visualize it was pretty annoying.

So, instead of creating noise to vary the high pass cutoff, I instead created a bunch of noise channels, and gave each of them their own high pass cutoff, then I blend those together (more on that part later).

In my Pass Filter, I created two new parameters (using Edit Parameter Interface) that I refence from a few different nodes:

Custom params on high pass filter

Through trial and error, I found that low cutoff values from 0.3 to 0.6 gave me what I want, so I use an expression to filter each of the channels with a cutoff in that range, based on the channel ID:

ch("baseCutoff") + (fit($C/(ch("passBands")-1), 0, 1, 0.2, 0.5))

The baseCutoff is sort of redundant, it could just be built into the “fit” range, but I was previously using it in other nodes too, and never got around to removing it 🙂

I’m using the “passBands” parameter in that expression, but I also use it in the Channel creation in the noise CHOP:

Noise channel creation from pulse bands param

It probably would have been more sensible just to hard code the number of channels here, and then just count how many channels I have further downstream, but this works fine 🙂

In the Transform tab of the noise, I’m using $C in the X component of the Translate, to offset the noise patterns, so each channel is a bit different. In hindsight, using $C in the “seed” gives a better result.

So now I have some channels of filtered noise!
I’ll keep passBands to 5 for the rest of this post, to keep it easy to visualize:

Filter noise, 5 bands

Combining noise and pulse

The top right part of the CHOP network is importing the pulse value that I set up earlier.

Geometry import to chops

I’m importing a single point, and the pulse value was copied into Position, as mentioned earlier.
I’m deleting the Y and Z channels, since they are the same as X, normalizing the data using Limit, moving the data into the 0-1 range, and then lagging and filtering to soften out the data a bit.

Here is what the pulse looks like before and after the smoothing:

Pulse attribute channel

So this pulse channel is tx0, and I merge that in with all the noise channels (chan1-x).

I want to use the value of this pulse channel to blend in the noise channels I previously created.
So, for example, if I have 4 bands: between 0 and 0.25 pulse value I want to use band 1 noise, between 0.25 and 0.5 I’ll use band 2, 0.5 and 0.75 use band 3, etc.

I didn’t want a hard step between bands, so I’m overlapping them, and in the end I found that having quite a wide blend worked well (I’m always getting a little of all noise channels, with one channel dominant).

This all happens in the vopchop_combineNoiseAndPulse:

vop: Combine Noise And Pulse

The left side of the network is working out the band midpoint ((1 / number of channels) * 0.5 * current channel number).

The middle section gets the current pulse value, and finds how far away that value is from the band midpoint for each channel.

If I just output the value of that fit node, and reduce the overlap, hopefully you can see what this is doing a little more obviously:

Motion view of channel blending

As the red line increases, it goes through the midpoints of each noise band, and the corresponding noise channel increases in intensity.

Since I’ve lowered the overlap for this image, there are actually gaps between bands, so there are certain values of the pulse that will result in no noise at all, but that’s just to make the image clearer.

The rest of the network is multiplying the pulse value with the noise channel value, and I end up with this:

Multiplied noise and pulse value

With that done, I delete the pulse channel, and I add all the noise channels together.

The last thing I’m doing is outputting the noise in stereo, and for that I’m just arbitrarily panning the sound between two channels using noise.

I create two new stereo noise channels together, and then use a vop to flip the second channel:

Noise channel flip

So I end up with this:

Stereo balance waveform

I multiply this back on the original noise:

StereoNoise

There’s a few other little adjustments in there, like making sure the amplitude is between -1 and 1, etc.

Also, I ramp in and out the audio over the first and last 20 frames:

RampInAndOut

And that’s it!

Hopefully you’ve enjoyed this series breaking down the squab video and my adventures in CHOPs. I can’t imagine I’ll use it for audio again, but it was definitely a fun experiment.

Today is Houdini 17 release day, so I can’t wait to dig in, and play around with all the new interesting features, perhaps there will be some vellum sims for future blog posts 🙂

Chopped Squabs – Pt 2

 

Last post, I talked about the motion of the squab, and the points created for the tubers sprouting from its back.

This post will be about the creating the tuber geometry and motion.

A series of tubes

Here’s the tuber network:

Houdini network for tuber creation
(Click to expand the image)

The Object Merge at the top is bringing in the tuber start points, which I talked a bit about last post.

I’m pushing those points along their normals by some random amount, to find the end points.
I could have done this with a Peak SOP, but wrangles are far more fun!

float length = fit(rand(@ptnum), 0, 1, 1.3, 2.2);

@P += @N*length;

Sometimes I just find it a lot quicker to do things in VEX, especially when I want to do value randomization within a range like this.

The points chopnet, near the top, is straight out of the “Introduction To CHOPs” by Ari Danesh. I highly recommend watching that series for more detailed information on CHOPs.

The chopnet takes the end point positions, adds noise, uses spring and lag CHOPs to delay the motion of the end points so they feel like they are dragging behind a bit:

The CHOPnet for tuber end points

After that, I create a polygon between each end and start point using an Add sop:

Add sop to create tuber lines

I’m then using Refine to add a bunch of in-between points.

In the middle right of the network, there are two network boxes that are used to add more lag to the centre of each wire, and also to create a pulse that animates down the wires.

When the pulse gets to the bulb at the end of the tuber, it emits a burst of particles and smoke, but more on that in a later post.

Here is the chopnet that handles the pulse, the lag on the centre of the wires, and a slightly modified version of the pulse value that is transferred onto the bulbs:

Chopnet that creates wire centre movement and pulse attributes

The wire lag is pretty simple, I’m just adding some more lag to the movement (which I latter mask out towards both ends of the wire so it just affects the centre).

The pulse source is a little more interesting.
Before this network, I’ve already created the pulse attribute, initialized to 0. I’ve also created a “class” attribute using connectivity, giving each wire primitive it’s own id.

When I import the geometry, I’m using the class attribute in the field “organize by Attribute”:

Class used to create a channel per wire

This creates a channel for each wire.

I also have a Wave CHOP with the type set to “ramp”, and in this CHOP I’m referencing the number of channels created by the geometry import, and then using that channel id to vary the period and phase.

Here are the wave settings:

Pulse source chop Wave waveform settings

And here is where I set up the Channels to reference the Geometry CHOP:

Channel settings for Wave Chop

To visualize how this is creating periodic pulses, it’s easier if I add a 0-1 clamp on the channels and remove a bunch of channels:

Wave CHOP clamped 0-1, most channels removed

So hopefully this shows that each wire gets a pulse value that is usually 0, but at some point in the animation might slowly ramp up to 1, and then immediately drop to 0.

To demonstrate what we have so far, I’ve put a polywire on the wires to thicken them out, and coloured the pulse red so it’s easier to see:

NoodleFight

It’s also sped up because I’m dropping most of the frames out of the gif, but you get the idea 🙂

The “Pulse Source Bulbs” section of the chopnet is a copy of the pulse, but with some lag applied (so the pulse lasts longer), and multiplied up a bit.

Tuber geometry

The remaining part of the network is for creating the tuber geometry, here is that section of the network zoomed in from the screenshot early in this post:

Tuber geometry creation network section

I’m creating the geometry around a time shifted version of the wires (back to the first frame), and then using a lattice to deform the geometry each frame to the animated wires.

Tuber cross-section

By triangulating a bunch of scattered points, dividing them with “Compute Dual” enabled, convert them to nurbs and closing them, I get a bunch of circle-ish shapes packed together.
There are probably better ways to do this, but it created a cross section I liked once I’d deleted the outside shapes that were distorted:

Splines created for gemetry of tuber cross section

To kill those exterior shapes, I used a Delete SOP with an expression that removes any primitive with a centre that was a certain distance from the centre of the original circle:

length($CEX, $CEY, $CEZ) > (ch(“../circle4/radx”) * 0.6)

This cross-section is then run up the wire with a Sweep:

Geometry for a single tuber

The Sweep SOP is in a foreach, and there are a few different attributes that I’m varying for each of the wires, such as the number of twists, the scale of the cross-section.

The twisting took a little bit of working out, but the Sweep sop will orient the cross section towards the Up attribute of each point.
I already have an Up attribute, created using “Add Edge Force” with the old Point SOP, it points along the length of the wire.

The normals are pointing out from the wire already, so I rotate the normals around the up vector along the length of the wire:

Rotated wire normals

The Sweep SOP expects the Up vector to be pointing out from the wire, so I swap the Normal and Up in a wrangle before the Sweep.

So, now I have the resting tuber geometry:

Tubers geometry undeformed

To skin these straight tubers to the bent wires, I use a Lattice SOP:

Tuber lattice geo

I Lattice each wire separately, because sometimes they get quite close to each other.
Last thing I do with the tubers is push the normals out a bit as the pulse runs through.

I already have an attribute that increases from 0 – 1 along each tuber, called “rootToTip”.
On the wrangle, I added a ramp parameter that describes the width of the tuber along the length (so the tuber starts and ends flared out a bit).

The ramp value for the tuber shape is fit into a range I like through trial and error, and I add the pulse amount to it, then use that to push the points out along their normals

This is the wrangle with the ramp parameter for the shape:

Tuber width wrangle

@tuberProfile = chramp("profile", @rootToTip, 0);

float tuberWidth = fit(@tuberProfile, 0, 1, 0.0, .04);
float pulseBulge = ((@pulse)*0.1*(1-@rootToTip));

@P = @P + (@N * (tuberWidth + pulseBulge));

This gives me the pulse bulge for the tubers:

Tuber pulse bulge

That does it for the tubers!

In future posts, I’ll cover the stalk bulbs, particles, pyro, rendering and audio.

 

Gears of Washroom – Pt 5

Last post I went through all the setup for the bubble sim, now for lighting, rendering, materials, fun stuff!

Scene materials

I talked about the texture creation in the first post, but there are also quite a lot of materials in the scene that are just procedural Houdini PBR materials.

Materials.png

Most of these are not very exciting, they are either straight out of the material palette, or they are only modified a little from those samples.

The top four are a little more interesting, though (purplePaint, whiteWalls, wood and floorTiles), because they have some material effects that are driven from the simulation data in the scene.

If you squint, you might notice that the walls and wood shelf get wet after the grenades explode, and there are scorch marks left on the walls as well.

Here is a shot with the smoke turned off, to make these effects obvious:

WetAndScorched.png

Scorch setup

To create the scorch marks in a material, I first needed some volume data to feed it.
I could read the current temperature of the simulation, but that dissipates over a few frames, so the scorch marks would also disappear.

The solution I came up with was to generate a new low resolution volume that keeps track of the maximum value of temperature per voxel, over the life of the simulation.

PyroMaxTemp

To start out with, I import the temperature field from the full Pyro sim, here is a visualization of that from about 2/3rds the way through the sim:

FullSimSmoke

I only need the back half of that, and I’m happy for it to be low resolution, so I resample and blur it:

SimplifiedSmoke

Great! That’s one frame of temperature data, but I want the maximum temperature that we’ve had in each voxel so far.

The easiest way I could think of doing this was using a solver, and merging the current frame volume with the volume from the previous frame, using a volume merge set to “Maximum”:

VolumeMaxSolver

And the result I get from this:

SimplifiedSmokeMax

So that’s the accumulated max temperature of the volume from the current frames, and all the frames before it!

Scorch in material

Back in the whiteWalls material, I need to read in this volume data, and use it to create the scorch mark.

Here is an overview of the white walls material:

whiteWallsMaterial.png

Both the wetness and scorch effects are only modifying two parameters: Roughness and Base Colour. Both effects darken the base colour of the material, but the scorch makes the material more rough and the wetness less rough.

For example, the material has a roughness of 0.55 when not modified, 0.92 when scorched and 0.043 when fully wet.

The burnScorch subnet over on the left exposes a few different outputs, these are all just different types of noises that get blended together. I probably could have just output one value, and kept the Scorch network box in the above screenshot a lot simpler.

Anyway, diving in to the burnScorch subnet:

BurnScorchSubnet.png
(Click for larger image)

One thing I should mention straight up: You’ll notice that the filename for the volume sample is exposed as a subnet input. I was getting errors if I didn’t do that, not entirely sure why!

The position attribute in the Material context is not in world space, so you’ll notice I’m doing a Transform on it, which transforms from “Current” to “World”.
If you don’t do that, and just use the volume sample straight up, you’ll have noise that crawls across the scene as the camera moves.
I found that out the hard way, 10 hours of rendering later.

Anyway, I’m sampling the maximum temperature volume that I saved out previous, and fitting it into a few different value ranges, then feeding those values into the Position and in one case Frequency of some turbulence noise nodes.

The frequency one is interesting, because it was totally a mistake, but it gave me a cool swirly pattern:

SwirlyNoise.png

When combined with all the other noise, I really liked the swirls, so it was a happy accident 🙂

That’s really it for the scorch marks! Just messing about with different noise combinations until I liked the look.

I made it work for the white walls first, then copied it in to the purple walls and wood materials.

Wetness setup

Similar concept to what I did for the temperature, I wanted to work out which surfaces had come in contact with water, and save out that information for use in the material.

WetnessSetup

On the left side, I import the scene geometry, and scatter points on it (density didn’t matter to me too much, because I’m breaking up the data with noise in the material anyway):

WetnessPoints

The points are coloured black.

On the right side, I import the fluid, and colour the points white:

WetnessPointsSim

Then I transfer the colour from the fluid points onto the scatter points, and that gives me the points in the current frame that are wet!

As before, I’m using a solver to get the wetness from the previous frame, and max it with the current frame.

WrangleWetness

In this case, I’m doing it just on the red channel, because it means wetness from the current frame is white, and from the previous accumulated frames is red.
It just makes it nice to visualize:

WetnessSolver

I delete all the points that are black, and then cache out the remaining points, ready to use in the material!

Wetness in material

I showed the high level material with the wetness before, here is the internals of the subnet_wetness:

subnet_wetness.png
(Click for larger image)

So I’m opening the wetness point file, finding all points around the current shading point (which has been transformed into world space, like before).
For all wetness points that are within a radius of 7 centimetres, I get the distance between the wetness point and the current shading point, and use that to weight the red channel of the colour of that point.
I average this for all the points that were in the search radius.

In the loop, you’ll notice I’m adding up a count variable, but I worked out later that I could have used Point Cloud Num Found instead of doing my own count. Oh well 🙂

I take the sampled wetness, and feed it into a noise node, and then I’m basically done!

If you want an idea of what the point sampled wetness looks like before feeding it through noise, here is what it looks like if I bypass the noise and feed it straight into baseColour for the white walls (white is wet, black is dry):

WetnessPointSample.png

Next up, Mantra rendering setup and lighting, should be a rather short post to wrap up with 🙂

Subsurface Scattering spherical harmonics – pt 3

Welcome to part 3 of this exciting series on how to beat a dead horse.

By the time I got to the end of the work for the last post, I was just about ready to put this project to bed (and by that, I mean P4 obliterate…).

There was just one thing I wanted to fix: The fact that I couldn’t rotate my models!
If I rotate the object, the lighting rotates with it.

Spaaaaaaace

To fix the rotating issue, in the UE4 lighting pass, I need to transform the light vector into the same space that I’m storing the SH data (object space, for example).

RotateSpace

To do that, I need to pass through at least two of those object orientation vectors to the lighting pass (for example, the forward and right vectors of the object).

So, that’s another 6 floats (if I don’t compress them) that I need to pass through, and if you remember from last time, I’d pushed the limits of MRTs with my 16 spherical harmonics coefficients, I don’t have any space left!

This forced me to do one of the other changes I talked about: Use 3 band Spherical Harmonics for my depth values instead of 4 band.
That reduces the coefficients from 16 to 9, and gives me room for my vectors.

<Insert montage of programming and swearing here>

3bandSH

So yay, now I have 3 band SH, and room for sending more things through to lighting.

Quality didn’t really change much, either, and it helped drop down to 5 uv channels, which became very important a little later…

Going off on a tangent

I figured that since I was solving the problem for object orientation, maybe I could also do something for deforming objects too?
For an object where the depth from one side to the other doesn’t change much when it’s deforming, it should be ok to have baked SH data.

The most obvious way to handle that was to calculate and store the SH depth in Tangent space, similar to how Normal maps are usually stored for games.

I wanted to use the same tangent space that UE4 uses, and although Houdini 15 didn’t have anything native for generating that, there is a plugin!

https://github.com/teared/mikktspace-for-houdini

With that compiled and installed, I could plonk down a Compute Tangents node, and now I have Tangents and Binormals stored on each vertex, yay!

At this point, I create a matrix from the Tangent, Binormal and Normal, and store the transpose of that matrix.
Multiplying a vector against it will give me that vector in Tangent space. I got super lazy, and did this in a vertex wrangle:

matrix3 @worldToTangentSpaceMatrix;
vector UE4Tang;
vector UE4Binormal;
vector UE4Normal;

// Tangent U and V are in houdini coords
UE4Tang         = swizzle(v@tangentu, 0,2,1);
UE4Binormal     = swizzle(v@tangentv, 0,2,1);
UE4Normal       = swizzle(@N, 0,2,1);

@worldToTangentSpaceMatrix = transpose(set(UE4Tang, UE4Binormal, UE4Normal));

The swizzle stuff is just swapping Y and Z (coordinate systems are different between UE4 and Houdini).

Viewing the Tangent space data

To make debugging easier, at this point I made a fun little debug node that displays Tangents, Binormals and Normals the same as the model viewer in UE4.

It runs per vertex, and creates new coloured line primitives:

TangentFace

Haven’t bothered cleaning it up much, but hopefully you get the idea:

TangentPrimsVOP.png

And the vectorToPrim subnet:

VectorToPrimsVOP.png

So, add a point, add some length along the input vector and add another point, create a prim, create two verts from the points, set the colour.
I love how easy it is to do this sort of thing in Houdini 🙂

The next step was to modify the existing depth baking code.

For each vertex in the model, I was sending rays out through the model, and storing the depth when they hit the other side.
That mostly stays the same, except that when storing the rays in the SH coefficients, I need to convert them to tangent space first!

HitsToSH.png

Getting animated

Since most of the point of a Tangent space approach was to show a deforming object not looking horrible, I needed an animated model.

I was going to do a bunch of animation in Modo for this, but I realized that transferring all my Houdini custom data to Modo, and then out to fbx might not be such a great idea.

Time for amazing Houdini animation learningz!!
Here’s a beautiful test that any animator would be proud of, rigged in Houdini and dumped out to UE4:

StupidTube.gif

So, I spent some time re-rigging the Vortigaunt in Houdini, and doing some more fairly horrible animation that you can see at the top of this post.

RiggedVort.png

Although the results aren’t great, I found this weirdly soothing.
Perhaps because it gave me a break from trying to debug shaders.

At some point in the future, I would like to do a bit more animation/rigging/skinning.
Then I can have all the animators at work laugh at my crappy art, in addition to all the other artists…

Data out

Hurrah, per-vertex Tangent space Spherical Harmonic depth data now stored on my animated model!

This was about the part where I realized I couldn’t find a way to get the Tangents and Binormals from the Houdini mesh into Unreal…

When exporting with my custom data, what ends up in the fbx is something like this:

   UserDataArray:  {
    UserDataType: "Float"
    UserDataName: "tangentu_x"
    UserData: *37416 {...

When I import that into UE4, it doesn’t know what that custom data is supposed to be.

If I export a mesh out of Modo, though, UE4 imports the Tangents and Binormals fine.
So I jumped over into Modo, and exported out a model with Tangents and Binormals, and had a look at the fbx.
This showed me I needed something more like this:

LayerElementTangent: 0 {
 Version: 102
 Name: "Texture"  
 MappingInformationType: "ByPolygonVertex"
 ReferenceInformationType: "Direct"
 Tangents: *112248 {...
This is probably around about when I should have set the project on fire, and found something better to do with my time but…

C# to the rescue!!

I wrote an incredibly silly little WPF program that reads in a fbx, changes tangentu and tangentv user data into the correct layer elements.

Why WPF you ask?
Seriously, what’s with all the questions? What is this, the Spanish inquisition?
Real answer: Almost any time I’ve written any bit of code for myself in the past 7 years, it’s always a WPF program.
80% of them end up looking like this:
AmazingUI
The code is horrible, I won’t paste it all, but I build a list of all the vectors then pass them through to a function that re-assembles the text and spits it out:
        public string CreateLayerElementBlock(List<Vector3D> pVectors, string pTypeName)
        {
            string newBlock = "";

            int numVectors  = pVectors.Count;
            int numFloats   = pVectors.Count * 3;

            newBlock += "\t\tLayerElement" + pTypeName + ": 0 {\n";
            newBlock += "\t\t\tVersion: 102\n";
            newBlock += "\t\t\tName: \"Texture\"\n";
            newBlock += "\t\t\tMappingInformationType: \"ByPolygonVertex\"\n";
            newBlock += "\t\t\tReferenceInformationType: \"Direct\"\n";
            newBlock += "\t\t\t" + pTypeName + "s: *" + numFloats + " {\n";
            newBlock += "\t\t\t\ta: ";
	...

Gross. Vomit. That’s an afternoon of my life I’ll never get back.
But hey, it worked, so moving on…

UE4 changes

There weren’t many big changes on the UE4 side, just the switching over to 3 band SH, mostly.

One really fun thing bit me in the arse, though.
I’d been testing everything out on my static mesh version of the model.
When I imported the rigged model, I needed to change the material to support it:
UseWithSkeletal
And then the material failed to compile (and UE4 kept crashing)…
So, apparently, skinned meshes use a bunch of the UV coordinate slots for… Stuff!
I needed to switch back to my old approach of storing 6 coefficients in TexCoord1,2 and 3, and the remaining three SH coeffs in vertex colour RGB:
RiggedMatChanges.png
Cropped this down to exclude all the messy stuff I left in for texture based SH data, but those three Appends on the right feed into the material pins I added for SH data in the previous posts.
And yeah, there’s some redundancy in the math at the bottom too, but if you don’t tell anyone, I won’t.

Shader changes

Now to pass the Tangent and Binormal through to the lighting pass.

I ended up compressing these, using Octahedron normal vector encoding, just so I could save a few floats.
The functions to do this ship with UE4, and they allow me to pass 2 floats per vector, rather than x,y,z, and the artifacts are not too bad.
Here’s some more information on how it works:
OctahedronEncoding.png
So now the Tangent and Binormal data is going through to the lighting pass, and I transform the light to tangent space before looking up the SH data:
 float3x3 TangentToWorld =
 {
  GBuffer.WorldTangent,
  GBuffer.WorldBinormal,
  cross(GBuffer.WorldTangent, GBuffer.WorldBinormal),
 };

 float3 TangentL = mul(L, transpose(TangentToWorld));

 float DepthFromPixelToLight  = saturate(GetSH(SHCoeffs, TangentL));
Probably could do that transposing in BassPassPixelShader I guess, and save paying for it on every pixel for every light, but then there’s a lot of things I probably could do. Treat my fellow human beings nicer, drink less beer, not stress myself out with silly home programming projects like this…

Conclusion

If I were to ever do this for real, on an actual game, I’d probably build the SH generation into the import process, or perhaps when doing stuff like baking lighting or generating distance fields in UE4.

If you happened to have a bunch of gbuffer bandwidth (i.e, you had to add gbuffers for something else), and you have a lot of semi translucent things, and engineering time to burn, and no better ideas, I suppose there could be a use for it.
Maybe.

Subsurface Scattering spherical harmonics – pt 2

 

This is my 2nd blog post on using spherical harmonics for depth based lighting effects in Unreal 4.

The first blog post focused on generating the spherical harmonics data in Houdini, this post focuses on the Unreal 4 side of things.

I’m going to avoid posting much code here, but I will try to provide enough information to be useful if you choose to do similar things.

SH data to base pass

The goal was to look up the depth of the object from each light in my scene, and see if I could do something neat with it.

In UE4 deferred rendering, that means that I need to pass my 16 coefficients from the material editor –> base pass pixel shader -> the lighting pass.

First up, I read the first two SH coefficients out of the red and green vertex colour channels, and the rest out of my UV sets (remembering that I kept the default UV set 0 for actual UVs):

SHBaseMatUVs

Vertex colour complications

You notice a nice little hardcoded multiplier up there… This was one of the annoyances with using vertex colours: I needed to scale the value of the coefficients in Houdini to 0-1, because vertex colours are 0-1.

This is different to the normalization part I mentioned in the last blog post, which was scaling the depth values before encoding them in SH. Here, I’m scaling the actual computed coefficients. I only need to do this with the vertex colours, not the UV data, since UVs aren’t restricted to 0-1.

The 4.6 was just a value that worked, using my amazing scientific approach of “calculate SH values for half a dozen models of 1 000 – 10 000 vertices, find out how high and low the final sh values go, divide through by that number +0.1”. You’d be smarter to use actual math to find the maximum range for coefficients for normalized data sets, though… It’s probably something awesome like 0 –> 1.5 pi.

Material input pins

Anyway, those values just plug into the SH Depth Coeff pins, and we’re done!!

Unreal 4 SH depth material

Ok.
That was a lie.
Those pins don’t exist usually… And neither does this shading model:

SHDepthShadingModel

So, that brings me to…

C++ / shader side note

To work out how to add a shading model, I searched the source code for a different shading model (hair I think), and copied and pasted just about everything, and then went through a process of elimination until things worked.
I took very much the same approach to the shader side of things.

This is why I’m a Tech Artist, and not a programmer… Well, one of many reasons 😉
Seriously though, being able to do this is one of the really nice things about having access to engine source code!

The programming side of this project was a bunch of very simple changes across a wide range of engine source files, so I’m not going to post much of it:

P4Lose

There is an awful lot of this code that really should be data instead. But Epic gave me an awesome engine and lets me mess around with source code, so I’m not going to complain too much 😛

Material pins (continued…)

So I added material inputs for the coefficients, plus some absorption parameters.

Sh coeffs

The SH Coeffs material pins are new ones, so I had to make a bunch of changes to material engine source files to make that happen.
Be careful when doing this: Consistent ordering of variables matters in many of these files. I found that out the easy way: Epic put comments in the code about it 🙂

Each of the SH coeffs material inputs is a vector with 4 components, so I need 4 of these to send my 16 coefficients through to the base pass.

Custom data (absorption)

The absorption pins you might have noticed from my material screenshot are passed as “custom data”.
Some of the existing lighting models (subsurface, etc) pass additional data to the base pass (and also through to lighting, but more on that later).

These “custom data” pins can be renamed for different shading models. So you can use these if you’d rather not go crazy adding new pins, and you’re happy with passing through just two extra float values.
Have a look at MaterialGraph.cpp, and GetCustomDataPinName if that sounds like a fun time 🙂

Base pass to lighting

At this point, I’d modified enough code that I could start reading and using my SH values in the base pass.

A good method for testing if the data was valid was using the camera vector to look up the SH depth values. I knew things were working when I got similar results to what I was seeing in Houdini when using the same approach:

BasePassDebug

That’s looking at “Base Color” in the buffer visualizations.

I don’t actually want to do anything with the SH data in the base pass, though, so the next step is to pass the SH data through to the lighting pass.

Crowded Gbuffer

You can have a giant parameter party, and read all sorts of fun data in the base pass.
However, if you want to do per-light stuff, at some point you need to write all that data into a handful of full screen buffers that the lighting pass uses. By the time you get to lighting, you don’t have per object data, just those full screen buffers and your lights.

These gbuffers are lovingly named GBufferA, GBufferB, GBuffer… You get the picture.

You can visualize them in the editor by using the various buffer visualizers, or explicitly using the “vis” command, e.g: “vis gbuffera”:

visGbuffers

There are some other buffers being used (velocity, etc), but these are the ones I care about for now.

I need to pass an extra 16 float values through to lighting, so surely I could just add 4 new gbuffers?

Apparently not, the limit for simultaneous render targets is 8 🙂

I started out by creating 2 new render targets, so that covers half of my SH values, but what to do with the other 8 values?

Attempt 1 – Packing it up

To get this working, there were things that I could sacrifice from the above existing buffers to store my own data.

For example, I rarely use Specular these days, aside from occasionally setting it to a constant, so I could use that for one of my SH values, and just hard code Specular to 1 in my lighting pass.

With this in mind, I overwrote all the things I didn’t think I cared about for stylized translucent meshes:

  • Static lighting
  • Metallic
  • Specular
  • Distance field anything (I think)

Attempt 2 – Go wide!

This wasn’t really ideal. I wasn’t very happy about losing static lighting.

That was about when I realized that although I couldn’t add any more simultaneous render targets, I could change the format of them!

The standard g-buffers are 8 bits per channel, by default. By going 16 bit per channel, I could pack two SH values into each channel, and store all my SH data in my two new g-buffers without the need for overwriting other buffers!

Well, I actually went with PF_A32B32G32R32F, so 32 bits per channel because I’m greedy.

It’s probably worth passing out in horror at the cost of all this at this point: 2 * 128bit buffers is something like 250mb of data. I’m going to talk about this a little later 🙂

Debugging, again

I created a few different procedural test assets in Houdini with low complexity as test cases, including one which I deleted all but one polygon as a final step, so that I could very accurately debug the SH values 🙂

On top of that, I had a hard coded matrix in the shaders that I could use to check, component by component, that I was getting what I expected when passing data from the base pass to lighting, with packing/unpacking, etc:

const static float4x4 shDebugValues = 
{
	0.1, 0.2, 0.3, 0.4,
	0.5, 0.6, 0.7, 0.8,
	0.9, 1.0, 1.1, 1.2,
	1.3, 1.4, 1.5, 1.6
};

It seems like an obvious and silly thing to point out, but it saved me some time 🙂

Here are some of my beautiful procedural test assets (one you might recognize from the video at the start of the post):

Houdini procedural test asset (rock thing)testobject3testobject2testobject1

“PB-nah”, the lazy guide to not getting the most out of my data

Ok, SH data is going through to the lighting pass now!

This is where a really clever graphics programmer could use if for some physically accurate lighting work, proper translucency, etc.

To be honest, I was pleasantly surprised that anything was working at this stage, so I threw in a very un-pbr scattering, and called it a day! 🙂

float3 SubsurfaceSHDepth( FGBufferData GBuffer, float3 L, float3 V, half3 N )
{
	float AbsorptionDistance 	= GBuffer.CustomData.x;
	float AbsorptionPower 		= lerp(4.0f, 16.0f, GBuffer.CustomData.y);

	float DepthFromPixelToLight 	= Get4BandSH(GBuffer.SHCoeffs, L);
	float absorptionClampedDepth 	= saturate(1.0f / AbsorptionDistance * DepthFromPixelToLight);
	float SSSWrap 			= 0.3f;
	float frontFaceFalloff 		= pow(saturate(dot(-N, L) + SSSWrap), 2);

	float Transmittance 		= pow(1 - absorptionClampedDepth, AbsorptionPower);

	Transmittance *= frontFaceFalloff;

	return Transmittance * GBuffer.BaseColor;
}
It’s non view dependent scattering, using the SH depth through the model towards the light, then dampened by the absorption distance.
The effect falls off by face angle away from the light, but I put a wrap factor on that because I like the way it looks.
For all the work I’ve put into this project, probably the least of it went into the actual lighting model, so I’m pretty likely to change that code quite a lot 🙂
What I like about this is that the scattering stays fairly consistent around the model from different angles:
GlowyBitFrontGlowyBitSide
So as horrible and inaccurate and not PBR as this is, it matches what I see in SSS renders in Modo a little better than what I get from standard UE4 SSS.

The End?

Broken things

  • I can’t rotate my translucent models at the moment 😛
  • Shadows don’t really interact with my model properly

I can hopefully solve both of these things fairly easily (store data in tangent space, look at shadowing in other SSS models in UE4), I just need to find the time.
I could actually rotate the SH data, but apparently that’s hundreds of instructions 🙂

Cost and performance

  • 8 uv channels
  • 2 * 128 bit buffers

Not really ideal from a memory point of view.

The obvious optimization here is to drop down to 3 band spherical harmonics.
The quality probably wouldn’t suffer, and that’s 9 coefficients rather than 16, so I could pack them into one of my 128 bit gbuffers instead of two (with one spare coefficient left over that I’d have to figure out).

That would help kill some UV channels, too.

Also, using 32 bit per channel (so 16 bits per sh coeff) is probably overkill. I could swap over to using a uint 16 bits per channel buffer, and pack two coefficients per channel at 8 bits each coeff, and that would halve the memory usage again.

As for performance, presumably evaluating 3 band spherical harmonics would be cheaper than 4 band. Well, especially because then I could swap to using the optimized UE4 functions that already exist for 3 band sh 🙂

Render… Differently?

To get away from needing extra buffers and having a constant overhead, I probably should have tried out the new Forward+ renderer:

https://docs.unrealengine.com/latest/INT/Engine/Performance/ForwardRenderer/

Since you have access to per object data, presumably passing around sh coefficients would also be less painful.
Rendering is not really my strong point, but my buddy Ben Millwood has been nagging me about Forward+ rendering for years (he’s writing his own renderer http://www.lived3d.com/).

There are other alternatives to deferred, or hybrid deferred approaches (like Doom 2016’s clustered forward, or Wolfgang Engels culled visibility buffers) that might have made this easier too.
I very much look forward to the impending not-entirely-deferred future 🙂

Conclusion

I learnt some things about Houdini and UE4, job done!

Not sure if I’ll keep working on this at all, but it might be fun to at least fix the bugs.

 

Subsurface Scattering spherical harmonics – pt 1

In this post, I’ll be presenting “SSSSH”, which will be the sound made by any real programmer who happens to accidentally read this…

This has been a side project of mine for the last month or so with a few goals:

  • Play around more with Houdini (I keep paying for it, I should use it more because it’s great)
  • Add more gbuffers to UE4, because that sounds like a useful thing to be able to do and understand.
  • Play around with spherical harmonics (as a black box) to understand the range and limitations of the technique a bit better.
  • Maybe accidentally make something that looks cool.

Spherical harmonics

I won’t go too much into the details on spherical harmonics because:
a) There’s lots of good sites out there explaining them and
b) I haven’t taken the time to understand the math, so I really don’t know how it works, and I’m sort of ok with that for now 😛

But at my basic understanding level, spherical harmonics is a way of representing data using a set of functions that take spherical coordinates as an input, and return a value. Instead of directly storing the data (lighting, depth, whatever), you work out a best fit of these functions to your data, and store the coefficients of the functions.

Here is a very accurate diagram:

DataSphere

You’re welcome!
Feel free to reuse that amazing diagram.

SH is good for data that varies rather smoothly, so tends to be used for ambient/bounced lighting in a lot of engines.

The function series is infinite, so you can decide how many terms you want to use, which determines how many coefficients you store.

For this blog post, I decided to go with 4-band spherical harmonics, because I’m greedy and irresponsible.
That’s 16 float values.

Houdini SH

Thanks to the great work of Matt Ebb, a great deal of work was already done for me:

http://mattebb.com/weblog/spherical-harmonics-in-vops/

I had to do a bit of fiddling to get things working in Houdini 15, but that was a good thing to do anyway, every bit of learning helps!

What I used from Matt were two nodes for reading and writing SH data given the Theta and Phi (polar and azimuthal) angles:

SHFunctions

Not only that, but I was able to take the evaluate code and adapt it to shader code in UE4, which saved me a bunch of time there too.

It’s not designed to be used that way, so I’m sure that it isn’t amazingly efficient. If I decide to actually keep any of this work, I’ll drop down to 3 band SH and use the provided UE4 functions 🙂

Depth tracing in Houdini

I’m not going to go through every part of the Houdini networks, just the meat of it, but here’s what the main network looks like:

NetworkOverview

So all the stuff on the left is for rendering SH coefficients out to textures (more on that later), the middle section is where the work is done, the right hand side a handful of debug modes visualizers, including some from the previously mentioned Matt Ebb post.

Hits and misses

I’m doing this in SOPs (geometry operations), because it’s what I know best in Houdini at the moment, as a Houdini noob 🙂
I should try moving it to shops (materials/per pixel) at some point, if that is at all possible.

To cheat, if I need more per-pixel like data, I usually just subdivide my meshes like crazy, and then just do geometry processing anyway 😛

The basic functionality is:

  • For each vertex in the source object:
    • Fire a ray in every direction
    • Collect every hit
    • Store the distance to the furthest away primitive that is facing away from the vertex normal (so back face, essentially)

All the hits are stored in an array, along with the Phi and Theta angles I mentioned before, here’s what that intersection network looks like currently:

IntersectAll

I’m also keeping track of the maximum hit length, which I will use later to normalize the depth data. The max length is tracked one level up from the getMaxIntersect network from the previous screenshot:

GenerateHits

This method currently doesn’t work very well with objects with lots of gaps in them, because the gaps in the middle of an object will essentially absorb light when they shouldn’t.
It wouldn’t be hard to fix, I just haven’t taken the time yet.

Normalizing

Before storing to SH values, I wanted to move all the depth values into the 0-1 range, since there are various other places where having 0-1 values makes my life easier later.

One interesting thing that came up here: when tracing rays out from a point, there are always more rays that miss than hit.

That’s because surfaces are more likely to be convex than concave, so at least half of the rays are pointing out into space:

FurryPlane

Realistically, I don’t really care about spherical data, I probably want to store hemispherical data around the inverse normal.
That might cause data problems in severely concave areas of the mesh, but I don’t think it would be too big a problem.
There are hemispherical basis functions that could be used for that, if I were a bit more math savvy:

A Novel Hemispherical Basis for Accurate and Efficient Rendering

Anyway, having lots of values shooting out to infinite (max hit length) was skewing all of the SH values, and I was losing a lot of accuracy, so I encoded misses as zero length data instead.

Debug fun times!

So now, in theory, I have a representation of object thickness for every vertex in my mesh!

One fun way to debug it (in Houdini) was to read the SH values using the camera forward vector, which basically should give me depth from the camera (like a z buffer):

SHDepth

And, in a different debug mode that Matt Ebb had in his work, each vertex gets a sphere copied onto it, and the sphere is displaced in every direction by the SH value on the corresponding vertex:

vortigauntBalloons

vortigauntBalloons2

This gives a good visual indicator on how deep the object is in every direction, and was super useful once I got used to what I was looking at 🙂

And, just for fun, here is shot from a point where I was doing something really wrong:

vortigauntClicker

Exporting the data

My plans for this were always to bake out the SH data into textures, partially just because I was curious what sort of variation I’d get out of it (I had planned to use displacement maps on the mesh in Houdini to vary the height).

SHImages
And yes, that’s 4 images worth of SH data, best imported as HDR.
But hey, I like being a bit over the top with my home projects…

One of my very clever workmates, James Sharpe, had the good suggestion of packing the coeffs into UV data as I was whining to him over lunch about the lack of multiple vertex color set support in UE4.
So I decided to run with UVs, and then move back to image based once I was sure everything was working 🙂

PixelVSVertex

Which worked great, and as you can probably see from the shot above, per-vertex (UVs or otherwise) is perfectly adequate 🙂

Actually, I ended up putting coefficients 1-14 into uvs, and the last two into the red and green vertex color channels, so that I could keep a proper UV set in the first channel that I could use for textures.

And then, all the work…

Next blog post coming soon!

In it, I will discuss all the UE4 work, the things I should have done, or done better, might do in the future and a few more test shots and scene from in UE4!

To be continued!!

The devil is in the decals

autodecal

Frequently when talking about mesh decals in UE4, I get comments about them being annoying to maintain, because every time you change your meshes you have to rebuild / adjust layers of decals.

Now, personally, I don’t really care that much, because my projects are all pretty small, and fixing up decals in Modo is generally a very quick job.

But it’s come up enough that I figured I’d make a “2 metres short of Minimum Viable Product” example of how you could address this.

Houe4dengine

That’s what I’m calling Houdini Engine + UE4 now, just to continue the tradition of me being annoying.

Right. Houdini stuff.
I made a digital asset:

Network.png

There are two inputs, which will get fed in from UE4 (later).
In the Houdini scene, #1 input is the object I want to generate a decal on, object #2 is a projection plane.

The stuff on the left is actually all redundant, but what I was planning to do was construct layout patterns in Houdini for different decals on one sheet, and let Houdini just automatically do the UV layout. But procedural UV’ing got super annoying, so I decided not to do that.

Anyway…

Extrude plane, cookie with box:

ExtrudeAndCookie.png

Delete faces that are on the opposite side of the projection (dot product driven delete sop, basically).

Since I couldn’t really get the UVs working the way I wanted, I created a centre point on the projection plane, get the normal and constructed U and V vectors, which I then project onto the verts in the decal mesh.

I did that all in VEX, because it seemed like a good idea at the time.

I was fairly annoyed with working on it by this point, so I just exposed the rotation and scale of the decal so you can play with it in Unreal 🙂

AutoDecalParams.png

Back in UE4

With that done, and the thing saved as a Houdini Digital Asset, time to load up a shamefully unfinished UE4 project (there are lots of choices here…).

The workflow is:

  • Load the digital asset into the content browser.
  • Drag a copy into the scene.
  • Using “World Outliner Input”, Select a plane for the projection, and an object to put decals on:

AutoDecal_outlinerSelect.png

Bam! New decal mesh, floating over the top of the original object, you can save it out using the Houdini engine bake stuff, or whatever you want to do.

Conclusion

I didn’t bother taking this too far, because I don’t really intend to use it myself, but if I thought it was going to be useful there are a bunch of things I’d probably do.

I mean, aside from completely re-build it from scratch, because it’s a whole bunch of broken hack right now…

  • Expose a few different projection types
  • Create separate Houdini asset that lets you lay out planes on a decal sheet to define regions for different decals (which I started on)
  • Make it work with multiple planes passed into the one asset

With any luck, Epic will just come along with a similar workflow where you can press a button on a projected decal in editor, and it will do this sort of thing for you 🙂

(In the meantime, I’ll just stick with manually doing it in Modo, thanks very much…)

 

City scanner scene – Breakdown pt2

Webs.gif

This is part 2 of the breakdown for my recent scene Half-Life 2 scanner scene (part 1 here).

This time, I’m going to focus on the Houdini web setup.

Although it took me a while to get a very subtle result in the end, it was a fun continuing learning experience, and I’m sure I’ll re-use a bunch of this stuff!

Go go Gadget webs!

I saw a bunch of really great photos of spider webs in tunnels (which you can find yourself by googling “tunnel cobwebs concrete” :)).

I figured it would be a fun time to take my tunnel into Houdini, and generate a bunch of animated hanging webby things, and bring them back into UE4.

This fun time ended up looking like a seahorse:

itsaseahorselol.png

I will break this mess a bit 🙂

Web starting points

PointsAndRaysGraph.png

I import the geometry for the tunnel and rails, and scatter a bunch of points over it, setting their colour to red.

On the right hand side of the seahorse is a set of nodes for creating hanging webs, which is just some straight down line primitives, with a few attributes like noise and thickness added to them.
I’ll come back to these later:

HangingWebs.png

In the top middle of the seahorse, I have a point vop apply two layers of noise to the colour attribute, and also blend the colour out aggressively below the rails, because I only wanted webs in the top half of the tunnel.

The web source points look like this:

WebPoints.png

From these points, I ray cast out back to the original geometry.

Ray casting straight out of these points would be a little boring, though, so I made another point vop that randomizes the normals a little first:

WebNormals.gif

After this, I have a few nodes that delete most of the points generated from the pipe connections: they have a high vertex density, compared to every other bit of mesh, so when I first ran the thing, I had a thousand webs on the pipe connections.
I also delete really small webs, because they look lame.

We are now at seahorse upper left.

Arcy Strangs.

ArcyStrangs.png

Not sure what I was thinking when naming this network box, but I’m rolling with it.

So anyway, the ray cast created a “dist” attribute for distance from the point to the ray hit, in the direction of the normal.

So my “copy1” node takes a line primitive, copies it onto the ray points, sets the length of the line to the “dist” attribute (my word, stamping is such a useful tool in Houdini).

CopyLines.png

Before the copy, I set the vertex red channel from black to red along the length of the line, just for convenience.

Previous up the chain, I found the longest of all the ray casts, and saved it off in a detail attribute. This is very easy to do by just using Attribute Promote, using Maximum as the Promotion Method.

So, I now define a maximum amount of “droop” I want for the webs, a bit of random droop, and then I use those values to move each point of each web down in Y a bit.

WebDroop.png

I use sample that ramp parameter up there using the web length, and then multiply that over the droop, so that each end of the web remains fastened in place.
And I don’t really care if webs intersect with the rails, because that’s just how I roll…

Fasten your seatbelts, we are entering seahorse spine.

Cross web connecty things

ConnectingWebStrands.png

For each of the webs in the previous section, I create some webs bridging between them.
Here’s the network for that.

ConnectingStrands.png

I use Connect Adjacent Pieces, using Adjacent Pieces from Points, letting the node connect just about everything up.

I use a carve node to cut the spline up, then randomly sort the primitives.

At this point, I decided that I only want two connecting pieces per named web, and I got lazy so I wrote vex for this:

string CurrentGroupName = "";

string PickedPieces[];
int PieceCount[];

int MaxPerPiece = 2;
int success = 0;

addprimattrib(geoself(), "toDelete", 0, "int");

for (int i = 0; i < nprimitives(geoself()); i ++)
{
    string CurrentName = primattrib(geoself(), "name", i, success);

    int FindIndex = find(PickedPieces, CurrentName);
    
    if (FindIndex < 0)
    {
        push(PickedPieces, CurrentName);        
        push(PieceCount, 1);
    }
    else
    {  
        int CurrentPieceCount = PieceCount[FindIndex];
        
        if (CurrentPieceCount >= MaxPerPiece)
        {
            setprimattrib(geoself(), "toDelete", i, 1, "set");
        }
        else
        {
            PieceCount[FindIndex] = CurrentPieceCount + 1;
        }
    }
    
    setprimattrib(geoself(), "name", i, CurrentName);
}

So that just creates an attribute on a connecting piece called “toDelete”, and you can probably guess what I do with that…

The rest of the network is the same sort of droop calculations I mentioned before.

One thing I haven’t mentioned up to this point, though, is that each web has a “Primitive ID” attribute. This is used to offset the animation on the webs in UE4, and the ID had to get transferred down the chain of webs to make sure they don’t split apart when one web meets another.

At this point, I add a bunch of extra hanging webs off these arcy webs, and here we are:

AllWebWires.png

Then I dump a polywire in, and we’re pretty much good to go!

Well… Ok. There’s the entire seahorse tail section.

For some reason, Polywire didn’t want to generate UVs laid out along the web length.

I ended up using a foreach node on each web, stacking the web sections up vertically in UV space, using a vertex vop, then welding with a threshold:

LayoutUVs.png

Since I have the position, 0-1, along the current web, I could use that to shift the UV sections up before welding.

With that done on every web, my UVs look like this:

UVsHoriz.png

Which is fine.
When I import the meshes into UE4, I just let the engine pack them.

Seriously, though… These are the sorts of meshes that I really wish I could just bake lighting to vertex colours in UE4 instead of a lightmap.
It would look better, and have saved me lots and lots of pain…

And here we are, swing amount in red vertex channel, primitive offset (id) in green:

FinalWebs.png

Web contact meshes

I wanted to stamp some sort of mesh / decal on the wall underneath the hanging meshes.
If you have a look back at the top of the seahorse, you might notice an OUT_WebHits node which contains all the original ray hits.

I’m not going to break this down completely, but I take the scatter points, bring in the tunnel geometry, and use the scatter points to fracture the tunnel.

I take that, copy point colour onto the mesh, and subdivide it:

WallWebsSubd.png

Delete all the non red bits, push the mesh out along normals with some noise, polyreduce, done 🙂

WallWebsFinal.png

I could have done much more interesting things with this, but then life is full of regrets isn’t it?

Back to UE4

So, export all that stuff out, bring it into UE4.

Fun story, first export I did was accidentally over 1 million vertices, and the mesh still rendered in less than half a millisecond on a GeForce 970.
We are living in the future, people.

CobwebsMaterial.png

Most of this material is setting up the swinging animation for the webs, using World Position Offset.

There’s two sets of parameters for everything: One for when the web is “idle”, one for when it is being affected by the Scanner being near it.

To pass the position of the scanner into the material, I have to set up a Dynamic Material Instance, so this is all handled in the web blueprint (which doesn’t do much else).

It also passes in a neutral wind direction for when the webs are idle, which I set from the forward vector of an arrow component, just to make things easy:

WindDirection.png

So now I have the scanner position, for each vertex in each web I get the distance between it, and the scanner, and use that to lerp between the idle and the “windy” settings.

All of these values are offset by the position id that I put in the green channel, so that not all of the webs are moving at exactly the same time.

Still to come…

Animation approach from Modo to blueprints, lighting rig for the scanner, all the fun stuff! 🙂

Factory – pt 4 – (Trimming the flowers)

Part 4 of https://geofflester.wordpress.com/2016/02/07/factory-pt-1/

FlowerPower

Alpha card objects

In most games, you have some objects that have on/off alpha transparency, generally for objects that you wouldn’t model all the detail for (leaves, flowers, etc).

AlphaCard

^ Exactly like that, beautiful isn’t it?
Years of art training not wasted at all…

Also referred to as punch-through / 1-bit / masked materials, btw.
So, you can see that the see-through portion of that polygon is pretty large.

When rendering these types of assets, you are still paying some of the cost for rendering all of those invisible pixels. If you are rendering a lot of these on screen, and they are all overlapping, that can lead to a lot of overdraw, so it’s fairly common to cut around the shape to reduce this, something like this:

CutAround

What does this have to do with the factory?
Aren’t you supposed to be building a factory?

I get distracted easily…

I’m not really planning to have a lot of unique vegetation in my factory scene, but I am planning on generating a bunch of stuff out of Houdini.

When I create LODs, that will be in Houdini too, and the LODs will probably be alpha cards, or a combination of meshes and alpha cards.

When I get around to doing that, I probably don’t want to cut around the alpha manually, because… Well, because that sounds like a lot of work, and automating it sounds like a fun task 🙂

Houdini mesh cutting tool

The basic idea is to get my image plane, use voronoi fracture to split up the plane, delete any polygons that are completely see-through, export to UE4, dance a happy dance, etc.

For the sake of experiment, I want to try a bunch of different levels of accuracy with the cutting, so I can find a good balance between vertex count, and overdraw cost.

Here’s the results of running the tool with various levels of cutting:

FlowerCutouts

Here’s what the network looks like, conveniently just low resolution enough so as to be totally no use… (Don’t worry, I’ll break it down :))

FullNetwork

The first part is the voronoi fracture part:

FracturePart_nodes

I’m subdividing the input mesh (so that I end up with roughly a polygon per pixel), then use an Attribute VOP to copy the alpha values from the texture onto the mesh, then blur it a bunch:

AlphaBlur_plane

I scatter points on that, using the alpha for density, then I join it with another scatter that is just even across the plane. This makes sure that there are enough cuts outside the shape, and I don’t get weird pointy polygons on the outside of the shape.

Here is an example where I’ve deliberately set the even spread points quite low, so you can see the difference in polygon density around the edges of the shape vs inside the shape:

FracturePart_plane2.png

Counting up the alpha

So, earlier, I mentioned that I subdivided up the input mesh and copied the alpha onto it?
I’ll call this the pixelated alpha mesh, and here’s what that looks like:

AlphaPrims

 

Next, I created a sub network that takes the pixelated alpha mesh, pushes it out along its normals (which in this case, is just up), ray casts it back to the voronoi mesh, and then counts up how many “hits” there are on each voronoi polygon.

Then we can just delete any polygon that has any “hits”.

Here is that network:

ProjectAndGetHits

After the ray sop, each point in the pixelated alpha mesh has a “hitprim”, which will be set to the primitive id that it hit in the voronoi mesh.

I’m using an Attribute SOP to write into a integer array detail attribute on the voronoi mesh for each “hitprim” on the pixelated alpha mesh points, and here’s that code:


int success = 0;
int primId = pointattrib(0, &quot;hitprim&quot;, @ptnum, success);

int primhits[] = detail(0, &quot;primhits&quot;);

if (primId &gt;= 0)
{
setcomp(primhits, 1, primId);
setdetailattrib(0, &quot;primhits&quot;, primhits, &quot;add&quot;);
}

After all that stuff, I dump a “remesh” node, which cheapens up the mesh a lot.

And back to UE4…

So, with all the above networks packaged into a Digital Asset, I could play with the parameters (the two scatter values), and try out a few different levels of cutting detail, as I showed before:

FlowerCutouts

I’m creating a rather exaggerated setup in UE4 with an excessive amount of overdraw, just for the purposes of this blog post.

For now, I’ve made the alpha cards huuuuuuuuge, and placed them where I used to have the flowers in my scene:

UE4Scene

Then, all I need to do is swap in each different version of my alpha card, and then GPU profile!

Profiling

The camera shot, without any alpha plane cutting optimization, took about 12 ms.

Test1, which is 27 vertices, seemed to be the best optimization. This came in at about 10.2 ms, so a saving of more than 1.5 ms, which is pretty great!

I was actually expecting Test2 to be the cheapest, since it chops quite a bit more off the shape, and at 84 vertices I didn’t think the extra vertex cost would even register on a GTX 970. Turns out I was wrong, Test2 was marginally more expensive!

This just goes to show, never trust someone about optimization unless they’ve profiled something 😛

Test3, at 291 vertices, costs about another 0.3 ms.

Conclusion

Winner.png

Of course, the savings are all quite exaggerated, but in a “real world” scenario I would probably expect to have a lot more instances, all a lot smaller. In which case, going with the lower vertex count mesh seems like it would still make sense (although I will, of course, re-profile when I have proper meshes).

Lots of more fun things to do with this: get it working on arbitrary meshes (mostly working), see if I can use Houdini engine and integrate it into UE4, etc.
Still not sure how much vegetation I’ll have in my factory scene, but I think this will still be useful 🙂