GLSL Python. Realtime Reflections and UV Maps.

Hello,

I am trying to figure out how to make a GLSL Shader use the objects UV map.

I am trying to modify this reflection shader, I want the shader to use the UV map of the diffuse and bump texture, instead of specifying a custom UV tiling. I am looking at the floor reflection shader. Defined by the text objects “reflect_ground” and “ground_reflection.py” in the “stained glass” scene.

This is the shader below. As you can see the UV for the diffuse and bump textures is defined by


gl_TexCoord[0] = gl_MultiTexCoord0*0.5+0.5;

and then later for example


...
float texscale = 40.0;
...
vec4 diffuse = texture2D(diffuse, gl_TexCoord[0].st*texscale).rgba;

How would I tell the shader I want to use the UV map defined in Blender? If I could understand this, then I could use this shader for my Vray lightmaps + UV tiled texture method, and with reflections of course :wink:

Complete shader “reflect_ground”:


from bge import logic as g
from bge import render as r
import bgl

cont = g.getCurrentController()
own = cont.owner

VertexShader = """

attribute vec4 Tangent;
varying vec3 T, B, N;


void main() 
{
    vec3 pos = vec3(gl_Vertex);  
    
    T   = Tangent.xyz;
    B   = cross(gl_Normal, Tangent.xyz);
    N   = gl_Normal; 

    gl_TexCoord[1] = ftransform();
    gl_TexCoord[2].xyz = pos - gl_ModelViewMatrixInverse[3].xyz;
    gl_TexCoord[0] = gl_MultiTexCoord0*0.5+0.5;
    gl_Position = ftransform();
}

"""

FragmentShader = """

varying vec3 T, B, N;
uniform sampler2D reflection,refraction, normals, diffuse;

float aberration = 0.005;
float bump = 0.2;
float texscale = 40.0;

vec3 TangentSpace(vec3 v)
{
    vec3 vec;
    vec.xy=v.xy;
    vec.z=sqrt(1.0-dot(vec.xy,vec.xy));
    vec.xyz= normalize(vec.x*T+vec.y*B+vec.z*N);
    return vec;
}

void main() {
   
    vec2 coord = gl_TexCoord[1].xy/gl_TexCoord[1].w*0.5+0.5;
    coord = clamp(coord,0.005,0.995);

    vec3 normal = texture2D(normals, gl_TexCoord[0].st*texscale).rgb*2.0-1.0;
    normal.x = -normal.x;
    normal.y = -normal.y;

    vec3 nVec = TangentSpace(vec3(normal*bump));     
    vec3 vVec = normalize(gl_TexCoord[2].xyz);
    
    float cosine = abs(dot(-vVec,nVec)); 
    float fresnel = pow(0.8-cosine+0.2,1.0); 
   
    vec3 reflect = texture2D(reflection, coord+nVec.st).rgb;
    
    vec3 luminosity = vec3(0.30, 0.59, 0.11);
    float reflectivity = pow(dot(luminosity, reflect.rgb),5.0);

    vec3 refract = vec3(0.0);
    refract.r = texture2D(refraction, (coord+nVec.st)*1.0).r;
    refract.g = texture2D(refraction, (coord+nVec.st)*1.0-(vVec.st*aberration)).g;
    refract.b = texture2D(refraction, (coord+nVec.st)*1.0-(vVec.st*aberration*2.0)).b;

    vec4 diffuse = texture2D(diffuse, gl_TexCoord[0].st*texscale).rgba;
    vec3 color = mix(diffuse.rgb*0.8+0.2,reflect,clamp(fresnel+reflectivity,0.0,1.0));
    
    gl_FragColor = vec4(color,1.0);
}
"""

mesh = own.meshes[0]
for mat in mesh.materials:
    shader = mat.getShader()
    if shader != None:
        if not shader.isValid():
            shader.setSource(VertexShader, FragmentShader, 1)
        shader.setAttrib(g.SHD_TANGENT)
        shader.setSampler('reflection',0)
        #shader.setSampler('refraction',1)
        shader.setSampler('normals',2)
        shader.setSampler('diffuse',3)


This is just a guess, but I suspect that the UV coordinates that you want are stored in 'gl_MultiTexCoord<X>" where <X> is the 0-indexed texture of your texture setup.

i.e.
In Blender select your object and look at the “Texture” tab. The first texture UV coordinates are probably in gl_MultiTexCoord0, the 2nd texture’s UVs are probably in gl_MultiTexCoord1, the 3rd in gl_MultiTexCoord2, etc, etc.

As I said, that is just an educated guess so it might be wrong.

No, I think that is wrong. You find the definition of the texture slots in the bottom of the code:


shader.setSampler('reflection',0)         
#shader.setSampler('refraction',1)         
shader.setSampler('normals',2)         
shader.setSampler('diffuse',3)

However, my problem is not knowing which texture to use, but how to tell the GLSL shader what kind of UV mapping to use. At the moment the shader code is defining the UV tiling itself, and the texture is set to “Generated”. I want the shader to use the UV Map that I define with the UV/Image Editor.

More info on gl_MultiTexCoord: http://www.opengl.org/sdk/docs/man2/xhtml/glMultiTexCoord.xml

bge.texture.ImageMirror(scene, cam, obj, matID) seems to do the same thing as I was looking for.

http://www.tutorialsforblender3d.com/BGE_Python/Video_Texture/VideoTexture_ImageMirror.html

Blend: http://www.mediafire.com/?3cjirr46c187t6c



# import bge module
import bge

# get current scene
scene = bge.logic.getCurrentScene()

# get list of objects in scene
objList = scene.objects

# get a list of the cameras
camList = scene.cameras

# get the camera named PlayerOne
cam = camList["Camera"]

# get object named Mirror
obj = objList["Mirror"]

# get the mirror material ID
# Name of my Mirror material is Reflect
matID = bge.texture.materialID(obj, "MAReflect")
          
# texture I'm using for the mirror is in first texture channel
texChannel = 2

# get the texture to be used as a mirror
mirror = bge.texture.Texture(obj, matID, texChannel)

# get the mirror source
mirror.source = bge.texture.ImageMirror(scene, cam, obj, matID)

mirror.source.clip=5000.0

# save the mirror as an object variable
obj["Mirror"] = mirror

# update the mirror image
mirror.refresh(True)

Your example looks better than Battlefield 4. We just need to add another shader to fake the volumetric lighting and shadows and we could make BF4 Blender Edition. lol

To better demonstrate my problem.

1 - bge.texture.ImageMirror

  • Good for small surfaces.
  • Good control in the material settings.
  • Reflections limited to the extent of the plane, only makes sense using one single plane.
  • No fresnel.


2 - GLSL Shader

  • Unlimited planar reflections, possible to use over multiple surfaces in the same plane.
  • Fresnel and other effects
  • Less control, all in python code
    - Do not know how to use UV Maps, therefore no lightmap (this is my main problem)
  • Only 4 texture slots allowed (I think, but ok for me)


I haven’t figured out why the offset of the reflection, but it should be possible to solve it.

The GLSL Shader mas many possibilities, here some kind of metal:


I managed to get the GLSL Shader to work as I wanted it. I simply played by the rules and mapped in the lightmap as a simple tiled square, I still did not match it up perfectly, but if I study how the UV mapping works with gl_MultiTexCoord I think I could understand it better.


You can take a look at it by yourself, compare the results in the two scenes: http://www.mediafire.com/?tntts8zy19216zx

I am at the moment rendering the reflection twice, once for the water, another time for the floor, I have to figure out how to use the same videotexture for both surfaces, since they are pratically coplanar. You would only render two videotextures if the planes were far apart. I can imagine that I will be having scenes with an indoor reflective floor, and a swimming pool, where it would be useful to only render one videotexture and use it for both the floor and water.

Still my question remains:

How do I tell the shader to use the UVMaps of the objects?

I really hope you find your answer as the GLSL shader version looks stunning. Good luck Martin

Keep up the amazing work,

Pm kupoman or mogari?

is there such a thing as a Cycles controlled GSL shader?

like you use cycles to generate the UV and then code to populate it?

Let me see if I understood, here’s a simple example. exmp.blend (738 KB)

As you can see in the object data panel I have two UV maps, no tile and tile, if you select the tiled and open the UV editor you’ll see that I made a scale to the UV map to get the tiling effect. The object also has two texture slots, notice that they are not referenced. Now in the shader code try to change line 29 from “gl_TexCoord[0]” to “gl_TexCoord[1]”.

I hope that helps.

That does help a bit, I think I am on the right track, however I am having problems, I will describe them when I get my head around it!

EDIT: It does seem to matter that the texture and UV slots are the same, I am getting different results if I move around the second texture in your example and point the code to it. The tiling is different.

EDIT: Thank you Aryok, you really helped me solve the problem!

Ok I solved the problem. I am correct with the following statement?


gl_MultiTexCoord0 is the mapping method defined in the first texture, slot 0.
gl_MultiTexCoord1 the method defined in slot 1.
gl_MultiTexCoord2 in slot 2. etc.

So it does not matter which UVMap the texture is pointing to, but it must be set to UV for the shader to look for uv maps, if it is set to generated the mapping is defined by the shader itself?

Blend demo: http://www.mediafire.com/download/25k2u12qazv2bo2/Columns_Shader.blend

I found the offset problem of the reflection, it was simply the center of the plane that slightly below the mesh. I fixed it with Object > Transform > Origin to Geometry.

Finally everything is working as I want. Except that if I use a lightmap with alpha, it crashes the standalone blenderplayer. Should I add something to the code for it to handle alpha? It seems to work fine in the embedded player, but not in the standalone player. Is this a bug, or something I have done wrong? I sidestepped the problem by cropping the lightmap and saving it as a new file.

The finished blend (with cropped lightmap, no crash): http://www.mediafire.com/?96du4h2r2h15jvu

Is it correct to use “gl_TexCoord[3] = ftransform();” instead of “gl_TexCoord[1] = ftransform();”?

Shader

from bge import logic as g
from bge import render as r
import bgl

cont = g.getCurrentController()
own = cont.owner

VertexShader = """

attribute vec4 Tangent;
varying vec3 T, B, N;


void main() 
{
    vec3 pos = vec3(gl_Vertex);  
    
    T   = Tangent.xyz;
    B   = cross(gl_Normal, Tangent.xyz);
    N   = gl_Normal; 

    gl_TexCoord[3] = ftransform();
    gl_TexCoord[2].xyz = pos - gl_ModelViewMatrixInverse[3].xyz;
    gl_TexCoord[0] = gl_MultiTexCoord0+7.115;
    gl_TexCoord[1] = gl_MultiTexCoord1;
    gl_Position = ftransform();
}

"""

FragmentShader = """

varying vec3 T, B, N;
uniform sampler2D reflection, normals, diffuse, lightMap;

float aberration = 0.01;
float bump = 0.1;
float texscale = 0.1;
float lmscale = 1;

vec3 TangentSpace(vec3 v)
{
    vec3 vec;
    vec.xy=v.xy;
    vec.z=sqrt(1.0-dot(vec.xy,vec.xy));
    vec.xyz= normalize(vec.x*T+vec.y*B+vec.z*N);
    return vec;
}

void main() {
   
    vec2 coord = gl_TexCoord[3].xy/gl_TexCoord[3].w*0.5+0.5;
    coord = clamp(coord,0.005,0.995);

    vec3 normal = texture2D(normals, gl_TexCoord[0].st).rgb*2.0-1.0;
    normal.x = -normal.x;
    normal.y = -normal.y;

    vec3 nVec = TangentSpace(vec3(normal*bump));     
    vec3 vVec = normalize(gl_TexCoord[2].xyz);
    
    float cosine = abs(dot(-vVec,nVec)); 
    float fresnel = pow(0.1+cosine-0.75,1.0); 
   
    vec3 reflect = texture2D(reflection, coord+nVec.st).rgb;
    
    vec3 luminosity = vec3(0.30, 0.59, 0.11);
    float reflectivity = pow(dot(luminosity, reflect.rgb),1.0);

    vec4 diffuse = texture2D(diffuse, gl_TexCoord[0].st).rgba;
    vec4 lightmap = texture2D(lightMap, gl_TexCoord[1].st).rgba;
    vec3 color = mix(lightmap.rgb*0.5+diffuse*0.8-0.5,reflect,clamp(-fresnel*0.25+reflectivity*0.15,0.0,1.0));
    
    gl_FragColor = vec4(color,1.0);
}
"""

mesh = own.meshes[0]
for mat in mesh.materials:
    shader = mat.getShader()
    if shader != None:
        if not shader.isValid():
            shader.setSource(VertexShader, FragmentShader, 1)
        shader.setAttrib(g.SHD_TANGENT)
        shader.setSampler('reflection',0)
        shader.setSampler('lightMap',3)
        shader.setSampler('diffuse',1)
        shader.setSampler('normals',2)


(please please post some more of those mind blowing screenshots now that you have it working? :slight_smile: )

Here is how you change the lightmap to see the crash in Standalone Blenderplayer. Change the lightmap to Bottomslab, it will not be uvmapped correctly but it will crash Standalone.


It looks the same, it is just that you do not need to define the mapping in the code, just map everything with UV Maps. The lightmap is lined up perfectly, it is not in the version above.

With some 2dfilters at least:


The video files played fine, without a hitch here (from youtube)

Thank you for the info! Maybe it is something with my “old” Nvidia GTX 465 and rendering flash video… :confused: