Ehhh… I guess I’ll leave this stuff to people who are a lot smarter than me for now and just continue baking my textures in MODO or Xnormal as I usually do. I just can’t get any predictable results in Blender and it’s texture system has never made any sense to me to begin with 
The improved bevel modifier in Blender 2.70 can add support loops to sharp edges automatically. Just take a copy of the lowpoly target, remove the edge split, add the bevel. There’s your cage. 
@NinthJake
I have to agree.
Cycles baking is my only hope.
First testing results look already superior to BI.
It isn’t just a matter of normal maps, right tangent space etc etc
it is the horrible lack of AA, the unpredictable artifacts when transferring textures on another UV set (diffuse textures)
Keep in mind, baking on 8k or 16 k and turn it to a 2k in Pshop doesn’t fix the horrible artifacts. It isn’t just a lack of AA.
What about organic stuff, like human heads and faces?
eye: A cage should always just be the exact same mesh, topology-wise. This is so the vertex index remains the same and each vertex knows where to project.
This must be a limitation of xNormal or some other tool because it certainly doesn’t apply to the method presented here.
It’s not a limitation, it’s just how (for instance) max handles its cage. It’s a linked copy of the mesh, in a modifier. Having to unwrap the cage and editing it by hand seems like a drag to me, unless I misunderstood you. I was just replying to
‘There may be other ways to do this (shrink wrap, nearest surface projection etc.) but they would only complicate things and make them less predictable.’
The lowpoly mesh is unwrapped only once. The cage is a copy of the unwrapped lowpoly, with optional modifications. Of course, if you don’t intend to modify the cage, you might as well bake straight to the original lowpoly. However, you have to modify the cage if your lowpoly has split edges, otherwise you’ll have discontinuity in the bake. Whether you add support loops or just make the cage all smooth is up to you. If you make it all smooth without support loops, you’ll get gradients and/or distorted projections.
a small note: the tangent baking is an ART. you cant make perfect results. never.
((I still have some questions:))
nevermind :). now i understand that the cage mentioned in the first post is different than what xnormal calls cage
Yes they were. I actually worked closely with Bastien and Campbell to get the current implementation working correctly as it is now. Unfortunately any opinions on that thread were misinformed.
You can still get fine grain control without modifying the cage mesh in the way you describe though. At most people would want to move a few verts around to help baking with ears etc.
Ok, fair enough.
Your method of baking is not a cage. You are baking to modified/optimised geometry to an OS normal map and then re-projecting the bake onto a TS normal map. This is not using a cage. It’s an old technique that has been used by myself and others for a long time. It’s a valid technique ofc, but it isn’t a cage.
I even spoke about it in my talk about the Blender conf last year.
Also Pea is right, btw.
Interesting post. So, do I have the work method right for this is:
Model High Poly > Model Low Poly > UV Unwrap Low Poly > Duplcate Low Poly (Cage) > Add Control Loops to cage > Bake High Poly to Cage > Apply Normal Map to Low Poly.
No, bake hi-poly to cage with object-space normal map. Then bake cage to low-poly with tangent-space normal map.
The object space/tangent space parts are really important.
EDIT: my apologies:
Do not bake from the proxy to the lowpoly. Instead, assign a material to the lowpoly that uses the object space normal map that you previously baked on the proxy. (This is why proxy and lowpoly require compatible UV layouts.) Then bake the normals of the lowpoly to a new tangent space normal map (selected to active = OFF!).
So bake hi-poly to proxy in object space. Set lowpoly to proxy-object-space map. Then bake low-poly with proxy-normals onto a tangent space normal map. Use last map.
I will now call it “proxy” to avoid confusion.
However, it’s better than the (original) cage method, and that’s the point of this thread.
It’s an old technique that has been used by myself and others for a long time. It’s a valid technique ofc, but it isn’t a cage.
I even spoke about it in my talk about the Blender conf last year.
I think someone just volunteered to write the requested tutorial.
I tried to test this method, see picture:
As you can see there is a problem when i put in extra edge loops at the end and in the middle: in the uv map those triangles became too small and can’t represent the detail too well even at 2048 resolution.
Big question: I don’t understand how should i “bake the material from this proxy to the lowpoly” Can someone explain it in details?
what I did is baking the proxy(with the object space normal map applied to it) to the lowpoly as tangent map, but the result still looks like an objectspace map. Nevertheless the result is not that bad except the resolution problem. Also there is a distortion on one of the smileys which is strange because the object space map was ok. (second picture: left is the low poly, right the “tangent” space nm baked from cage. sthe left smiley’s smile got distorted)
There are multiple issues here that affect the result:
- The lowpoly model is triangulated. This will prevent the use of the loop cut tool (which has an option to preserve the UV layout while cutting). Temporary triangulation while baking (using a Triangulate modifier) is good, but you should not apply the modifier until the lowpoly model is finished and ready for export to the game engine.
- The shading of the highpoly model is not smooth.
- Triangulation also prevents the use of the “Follow Active Quads” unwrap function which could otherwise give you a perfectly rectangular and seamless UV layout.
- The UV layout must be split where the lowpoly model has sharp edges.
Big question: I don’t understand how should i “bake the material from this proxy to the lowpoly” Can someone explain it in details?
what I did is baking the proxy(with the object space normal map applied to it) to the lowpoly as tangent map, but the result still looks like an objectspace map. Nevertheless the result is not that bad except the resolution problem. Also there is a distortion on one of the smileys which is strange because the object space map was ok. (second picture: left is the low poly, right the “tangent” space nm baked from cage. sthe left smiley’s smile got distorted)
[ATTACH=CONFIG]288987[/ATTACH]
Do not bake from the proxy to the lowpoly. Instead, assign a material to the lowpoly that uses the object space normal map that you previously baked on the proxy. (This is why proxy and lowpoly require compatible UV layouts.) Then bake the normals of the lowpoly to a new tangent space normal map (selected to active = OFF!).
Hey, thanks for the comments!
-I added the loopcuts before triangulation(which i did only at the very end). I don’t know about this “preserving the uv layout” option. I will look into it
-yes i should had smoothed the high poly though I usually bake from much higher res models where it doesn’t matter much.
I will try the baking the way you described when i hacve time (though sounds a bit strange because when i applied that object space map on the low poly the details were way off… thinking about it, it must had been because the uv layout was not “preserved”)
The option is called “Correct UVs”. If you enable it, the position of the loop cut in the UV layout will be proportional to the position in the mesh.
I’d recommend to assign a color grid texture to the mesh after unwrapping it. This will tell you early on whether there is enough texture space mapped to areas that need to capture lots of highpoly detail.
I have made an other test and this time it worked as it should. I’ve found that for sharp edges this “proxy” cage method gives the best result (better than using a cage in xnormal) not much difference, but still. For organic things I’d use a normal cage since it’s much easier to make (alt+s on a duplicate).
((for those who are as slow learner as I am here is the detailed method for the last part: bake high poly to the proxc cage, selected to active, object space map. Then give a material to the low poly and use the baked object space map-don’t forget to turn on normal map in image sampling AND change it from tangent to objectspace (this is what i forgot). Then bake from the low poly a tangent space map (create a new image for this)-this time selected to active should be off as only the low poly needed))
Thanks again for explaining it!
