Rigging and Blending Normal Maps

Hi all,

It’s been a while since I’ve posted. I thought I would post a project I’ve been working on in the last year or so. I should preface this by saying that I am in no way a professional. And I will be making up some terminology here because I don’t actually know it… If anybody does, please let me know.

The concept of the project is to figure out a way to blend and rig multiple normal maps to deform whatever is being rigged. In this case, I am rigging an Orc bust. This method isn’t limited to character work though. This project was inspired by one of the god of war 3 unlockable featurettes. Other huge inspirations include Azog from the hobbit, Caesar from the newer Planet of the apes films… and actually pretty much any digital character created in recent years that has very complex and realistic facial distortion. I know I’m not the first one to do this within blender. but I found that when it came to complex face rigging with blended normal maps, documentation and video was seriously lacking. So this is my meager attempt to mimic the kind of stuff I see in movies and games.

In order to create the different normal maps, I sculpted an initial bust of the character with as much detail as I could put into the sculpt. A disclaimer on the Sculpt, It was a rather quick sculpt I did with as much facial detail as I could put into it. I also forgot to sculpt the eyes closed… which actually leads to some problems with the eyebrows as you might notice later on… but, whatever this is just a test.


I didn’t model the teeth. Instead I used a file from blendswap made by the user lucasfalcao. Here is the link:

Once I had a main object I could start making some shape-key objects. Starting with the obvious choices (to me at least) Brow up and Brow down. The thought was that I would move the whole brow up (and down) and then separate them into left, right, inner and outer later on.

Brow up is in on the left and brow down is on the right. (I had to link this image because it was too big, also sorry for the nasty aliasing…)

The next step was combining them into a mesh that I would deform. I settled on a multires level of 1 (14, 662 polygons) for the deform mesh. Once I had the Shape-Keys being driven by a bone, I needed to blend the normal maps I created from the three different sculpt objects. So, Much like a shape-key driver, I needed a positive, a base and a negative image. Here is the shader group I came up with.

It’s pretty simple. The eyebrow up normal map goes in Image Pos, the Main or Base normal map goes into image base and the eyebrow down normal map goes into image neg. The value slider on the bottom goes from -1 to 1 and blends between the 3 images. Here is a closer look at the inner workings of the group:


I don’t know if this needs to be explained but I will anyways because I love getting technical… It’s a condition. Image pos goes into Color 2 (value 1), image basis goes into Color 1 (value 0). The color output of the first mix node is then plugged into Color 1 of the second mix node and image neg goes into color 2. Basically what this means is that image pos and image neg will never blend between anything other than the basis. The value slider is then plugged into the factor of the mix nodes, and is multiplied by -1 before being plugged into the second mix node so that when the slider reaches -1 the mix node will be at 1. Again sorry if this seemed like it didn’t need explaining, don’t judge me.

Pretty much the only other step is to apply the same driver on the value slider of the group node that was driving the shape-keys. By the way, in the driver editor any driver that was added to a node will only show up if that node is selected.

Now, the fruit of my labor:

This isn’t the end of this project by the way. There’s plenty more fun to be had with blending normal maps and I’ve been losing tons of much needed sleep trying to figure out the simplest way of setting up these nodes… Here is a preview of it’s blending glory.

(only 3 images per post so here is another link…)

Anyways, hopefully my need to get super technical and over explain didn’t rustle too many jimmies. I’m really only going in depth so that others can take what I’ve done here and make it better. If anybody notices something that I could do better in the future please feel free to let me know.

-Soupertrooper

I should mention that I was shown how to add drivers to node values by a guy from cg cookie named Richard Wilks. He and I exchanged e-mails back and forth a few times earlier this year and his help is really what made this project possible for me to understand. That said, I’m really excited to show what I’ve been able to figure out. The video in my previous post, although exciting for me, is really one of the easier challenges of this whole concept. I’m actually on my second iteration of the node set up for the character’s material. This is the first version of the material: http://i.imgur.com/G4OPUiD.png

As you can tell, I went a little crazy with reroute nodes. Reroute nodes kick so much ass. I chose to start with Jaw movements and eye movements since it seems to me that they are a good base for the other normal maps to blend over top (opinions may vary). The eye shape-keys I created are only open and close and not upper eyelid lower eyelid, I will get there eventually though. For Jaw movements, I combined the up and down (open close) movements the same way I did with the eyebrows. Jaw left and right could also use that method, but the problem is that using a mix node set to 0.5 to blend with the up/down node halves the different details. It’s very faint but the more you mix the more you diminish previous details. On the other hand overlay doubles the similar details.


In order to use overlay, I had to process the normal maps in photoshop and cancel out the similar details before combining them. It’s a pretty simple method, Just take the basis normal map and a different normal map (Jaw Left or right) and bring them into photoshop. Then bake out a normal map with no detail and bring that into photoshop as the bottom layer. Invert the basis and then set it to overlay and you’re left with only the differences between the two normal maps… sort of. The similar details are faint. This just helps Identify what you want to keep since it’s more pronounced. Merge the two top layers and mask out the areas that aren’t essential.


Then it’s just a matter of combining the new Left and Right normal maps with the no detail normal map and combining the left/right node and the up/down node with a mix node set to overlay. That way the detail is only additive, and you can use this method as many times as you want without losing detail.


That’s all for now, I have more to post later today about the eyelids that should complete this lengthy explanation. I’m just waiting for the example video to finish rendering. I’m really excited about this project. It’s turning out to be a lot more complicated than I had anticipated, but the results I’m getting are better than I expected. Hope people are enjoying this.

-Soupertrooper

Combining the eyes is actually quite simple, and it’s going to introduce another node group. It’s almost exactly the same as the NormalMapBlend node group. Here it is:


It’s actually used for differentiating left and right, so it needs masks that include only the areas you want to be effected in order to work right. Another thing this requires is to duplicate the NormalMapBlend node that the eye normal maps are plugged into, so that we’re left with an individual slider for both left and right. Then just plug the left and right normalmapblend node into Image L and Image R, plug the left and right masks into their corresponding inputs and finally I plugged the Jaw up/down normalmapblend node into the image base and that’s it.


Again I’m almost certain people fully understand what’s going on inside the node, but I love explaining things. For the first mix node Image L goes into color 2(value 1) and Image Base goes into color 1(value 0) Mask L goes into the factor.For the second mix node Image R goes into color 2, the output from the first mix node goes into color 1 and Mask R goes into the factor. It’s basically just an organized way of setting this up so that I don’t have to create two mix nodes every time I want to do something like this… Because I’m lazy. Here is what I added to the Node set up from my previous post:


Here a link to an image of the whole node set up so far: http://i.imgur.com/ZeVryUr.jpg

And this is the video I was rendering earlier today: http://youtu.be/IzspySBH-VI

I feel like I could have sculpted a better Jaw Open Shape-Key. The wrinkles on his neck aren’t quite where I want them. At this stage, I don’t have a lot more to explain. I’m just going to be using everything I described to create a node set-up for facial deformations. I’ll be posting the faces I’ve sculpted and as many videos like the one above as possible. If anybody has any suggestions for shape-keys I could add to this, please let me know. Also if there are any questions feel free to ask. Thanks for reading my ramblings. I hope people learned some things and that there are people out there striving to take what I’ve done and make it better.

-Soupertrooper

I found some inspiration on the youtubes. This rig was created by Snappers Systems. https://www.youtube.com/watch?v=C4z_RYApUuQ&list=FLQNgsbFV54eZHY3jYxQQaqg&index=1. This is the general concept I’m going for. I think this first try might not be as complex as that though.

Here is an update on the shapes I’ve created: http://i.imgur.com/mwwYBnk.png. I have plans to separate the eye shapes into upper and lower. I also think that Smile and Jaw down could be better; I might change them. I have more shapes on the way. Feel free to make suggestions. Here is the node group with all 15 of the faces shown in the image linked above: http://i.imgur.com/dFVBexF.png

-Soupertrooper