BGE Development: Rasterizer and Scenegraph optimization (proposals, discussion)

@Elfaz_Blend: Yes, this is better UI, should be no problem implement the dropdown list
@SolarLune:
1.) It’s only replaced if its necessary
2.) Thanks for the hint, will try this to get better replacement performance

I will upload a test build soon. Thanks for the replies!

greetings, moerdn

I miss a option to set the distance the meshes are replaced at. It would also be good to be able to hide object when they too far away or too near. I think hiding can be the big save - small objects on the ground we can probably hide at a pretty close distance.

Hide too near objects is god because we can make big low poly object representing a bit of a forest or a piece of a city that is showed at big distance, when we get closer we show individual object and hide the big pieces. That should help us keep the number of objects down.

A bit of a gap between the ‘step up’ and ‘step down’ distance would reduce mesh switching when going back and forth. We get a little more objects in total but that’s usually behind us so not in the camera view.

And I want to see the patch rather then a build - even if a linux build would be nice. But patches is nicer because they works like a guide - showing the files and function handling this stuff and thus help people learn to understand the code base.

And most important - Thanks - this is great work!

LaH, you mentioned some good points. Distance and hiding should be in UI options. Replacing one big objects through man small could also be helpful.

I primarily work on linux, but making a dependency free build is a bit more difficult, than for windows. I will also release the source code as well! You can find all of my blender coding stuff (group references, group constraints …) on gitoriuos:

https://gitorious.org/~moerdn/blenderprojects/moerdn-bge-sandbox

LOD stuff is not uploaded, will try to clean up the code a little bit.

thanks for your hints,
moerdn

Thanks for that link. Is the blender repo that master is a copy of kept up to date with blender svn? In that case this is great for keeping all patches around and easy to update. NEWER MIND - apparently it is :slight_smile:

Gitorius looks nice to…

I’m a bzr (bazaar) user my self but it looks like I need to learn that git thing…

I try to keep everything up-to-date but its not as actual as latest svn revision. I’m a hg (mercurial) user, but git and bzr are awesome as well. On gitorious you can find a wiki to my other bge specific proposals as well. A good place to write down some code specific documentations.

Feedback to code is always appreciated :slight_smile:
moerdn

Hi,

I would like to show you the first performance tests for LOD mechanism. First, it seems that scenegraph drop in performance is not as obvious as I had expected. With a more game-like test scene the objects are distributed more randomly.

To get a scene for testing purpose I, build a little benchmark where you can automate the placement of objects.

This picture shows the testscene. On the left side you see all plain highpoly objects, on the right side you can see LOD in action.

http://img696.imageshack.us/img696/1734/scenediff.jpg

Polycount for red objects: 2000 Faces
Polycount for yellow objects: 1000 Faces
Polycount for green objects: 100 Faces
Objectcount in total: 500

The next picture shows the framrate in more detail. LOD is a little bit varying, but in this case three times better than with plain highpoly objects. Be aware that this depends also on object count and difference in mesh density. The Camera is rotating around the scene, that causes a lot of LOD switches.

http://img814.imageshack.us/img814/2987/benchmark.png

The benchmark is inspired from sintel game. You can download the blendfile here:

http://dl.dropbox.com/u/2779060/Poly-Bench.zip

It also includes two billboarding trees. If you like so see how the placement operator is used and what impact the LOD has on the performance, please look at this video:

The benchmark could also be useful for other rasterizer analysis. I will try to add another profiling entry for LOD calculations.

greetings,
moerdn

Amazing, this is going very well, and a LOD system ll be great!
Thanks for your effeorts in order to increase the power of our engine

I have created a branch now for lod implementation ‘ras-lod’ on gitorious:
https://gitorious.org/~moerdn/blenderprojects/moerdn-bge-sandbox#help

I will merge my local changes to this repo soon, so you can checkout the code. My first step is the blender rna cleanup. For now I simply added a bunch of properties to the object type.


struct Object { ...
char lod1[32];
char lod2[32];
short use_lod;
...

These props are stored per object at toplevel but a more elegant solution would be better. Maybe I can create a new lod property group that contains the data.

bpy.context.object.lod.level_1 = 
bpy.context.object.lod.level_2 = 
bpy.context.object.lod.use_lod = True

Or maybe a dynamic lod level list for objects would be nice.

bpy.context.object.lod.levels.append("")
bpy.context.object.lod.use_lod = True

There is also another question that come up: Distance setting should also be stored. Where to put them? Maybe scene or world properties could be a good place. We also could show them in the per-object lod settings.

Just some thoughts on implementation. Ideas and suggestions are welcome.

greetings,
moerdn

That is really amazing and good work. What else to expect from Moerdn:-) Thanks

I think that this would be a more elegant and blender way:

struct LodData {
            struct LodData *next, *prev;
            char lod_mesh_name[32];
            int distance;
}

and then add

 struct object {
           LodData *first, *last;
           char use_lod;
....

the first loddata should have null in previous, the last in next. This way you can have how much levels as you want, end each mesh can have a different number of levels. I think per object lod is better.

You can iterate thought all the structs with a simple for statement

I think 4 lod levels are enough. Who makes more than 4 lod levels of his model.
I also think per object lod is better.

More than 4 lods can be useful for enormous meshes, like mountains, skyscraper, etc…

Nice Progress so far.
Looking forward to the final build.

There is some previous work from Benoit in implementing static LoD meshes. It’s back from 2009, but it may be worth taking a look at it. It’s in the bb_dev (a branch against 2.49). I don’t think it even got ported to 2.5 (bb_dev_25).

If I remember correctly he implemented the internals and UI for this. But I’m not sure how he was handling the mesh switching based in the distance. If you go over the logs of the branch you will likely have a good clue. I even think the code may work in trunk since not much was changed in Rasterized since then.

@dfelinto: Thank you for the hint. I remember this branch, but I thought only UI was working… I will take a look on how he implemented this stuff.

greetings,
moerdn

Now I’ve implemented LOD data as ListBase type, as Makers_F proposed. (Thanks for the reply with all the good tips and hints!)

You can now have a variable number of lod levels per object, each level has its own distance and mesh settings. (atm they are shared, need to copy them on shift+D and share them on alt+D?)

Here you can see the UI:
http://dl.dropbox.com/u/2779060/LoD-UI_new.ogv

Not everything is working right now. Especially saving and loading the data from blendfile have to be implemented. I will provide a new thread for this in the coding forum and wiki entry that gives a little bit more information about the progress and the implementation detail.

thanks everyone,
moerdn

That is a beautiful UI, one that ought to do the system justice.
Out of interest, will you be doing anything on “out of frustrum occlusion?”

I’m happy you found my reply useful :slight_smile:
Saving may be a little more tricky since your are using a dynamic list. I didn’t saved jet this kind of data with the blender dna, so i don’t exactly know how it works. Maybe you can look to how the other ListBase structs are saved and loaded [1], and exactly copy the behaviour :wink:

About the out of frustum, i think it shouldn’t have sense to replace it, since the rasterizer already ignore the meshes outside of frustum. The only problem can happen with bullet, but i don’t think he’s changing the physic mesh, so there is no problem

[1] Looking how the controllers are saved(they are a list base).
In writefile.c, line 1324 add

write_loddatas(wd, &ob->loddatas);

and then at 1026

static void write_loddatas(WriteData *wd, ListBase *lb)
{
    LodData *lod;

    lod= lb->first;
    while(lod) {
        writestruct(wd, DATA, "LodData", 1, lod);
        lod= lod->next;
    }
} 

i assume

typedef struct LodData{
    struct LodData *next, *prev;
    char mesh[32];
    int distance
} LodData 

Look at readfile.c to find how to load them, shouldn’t be hard( and it may work out of the box… )

Hello and thanks!

@agoose77

Yes, I also think the frustum culling via bullet is quite good for now.

@Makers_F
[1] This is exactly what I’m doing right now :slight_smile: I was a little bit confused because there are writestruct AND writedata functions that are use for logicbricks for example.

Need to look closer at this. Thanks for the hints!
moerdn

Cool job moerdn!