Multithreaded Level of Detail & Procedural Loading [High Performance]

Game Engine Game Engine Resources
1

Hello Everyone,

My name is Zalán (from Hungary).
I would like to introduce a little script to you.
As we know, the main BGE loop runs only on a single thread. This thread manages everything, so it’s not surprising if it’s over exhausted sometimes. The bigger games requires Level of Detail, to keep the scene low poly. It’s easy to make a LOD algorithm, but it could be CPU expensive… especially if every object needs to be replaced in the scene.

Solution?

  • Make a separate thread just for the logic.
  • Only one, object independent LOD script, which examines and handles all objects in the scene, and make decisions if they needs to be replaced or not.
  • Make this LOD script run on the separate thread.

Result:

ZLOD - Multithreaded Level of Detail calculation script

Usage with Object’s game properties:Every Objects in the scene can have:
“2nddist” - distance to switch to the second detailed mesh (not necessary)
“3rddist” - distance to switch to the second detailed mesh (not necessary, won’t work without 2nddist)
“LODdist” - distance to turn the object invisible/visible OR load/unload it when Procedural Loading is turned on.Just for the Owner of this script:
“Procedural Loading” (Boolean) - If it’s true, Every object with the “LODdist” property will be dinamically loaded from an inactive layer. (depending on it’s distance and LODdist)
“Use LOD percentage” (Boolean) - Enables the execution of the Graph_INIT.py which will align all of your object’s LOD distances to the percent you gave (in the config.txt in the same directory, or if it’s not exist, the “userVar” in “Graph_INIT.py”) (Level of Graphic)

RULES of use:

1: Just the base objects needs LOD properties.
2: If you use 2nddist and/or 3rddist, your object’s mesh’s name must end with the number of level it use.
eg.: Sphere.L1 (prop owner active object’s mesh name)
Sphere.L2 (inactive object’s mesh name)
Sphere.L3 (optional inactive object’s mesh name)
3.: If you enable Procedural Loading, please make sure all of your objects with LODdist property are located in an inactive layer (else if they are out of LODdist, they will be completely deleted and can’t load them again)
4.: If you enable LOD percentage (level of graphic) you can adjust ALL of your LOD distances with a single number (percent). By default it will be loaded from the “config.txt” in the same directory. But if the file doesn’t exist, the script will load the value from the “userVal” located in the Graph_INIT script.
5.: Don’t worry… this version is backward compatible, and can be work in the exactly same way as it’s Predecessor worked.

Importing to your scene:
1.: Copy both ZLOD_MT.py and Graph_INIT.py files from my .blend to your.
2.: Create an empty.
3.: Create an always sensor to your empty. Enable pulse mode in it. (TRUE level triggering)
4.: Attach a python module controller to it.
5.: Write into the controller: ZLOD_MT.init
6.: (optional) If you want, create LOD configuration properties for the empty, like: “Procedural Loading” or “Use LOD percentage”
7.: (optional) If you enable “Use LOD percentage”, please make sure you have Graph_INIT.py attached to an always sensor WITHOUT pulse.
8.: Voilà! It’s done. Next time you start the BGE it will automatically starts in a new background thread and begins to look for object with LOD game property. (2nddist,3rddist,LODdist)

The LOD algorithm is based on SolarLune’s BGHelper’s LOD solution.

Please feel free to improve it, and/or give advices and new ideas.

Version 0.3 (Minor improvements)
ZLOD_MT_0.3.blend (3 MB)

Version 0.2.2
10.02.2013
(LINK BROKEN) (2,9 MB)

Version 0.1
29.01.2013
ZLOD_MT.blend (540 KB) (0,5MB)

Version optimised for LibLoad by @iFlowProduction
http://iflowproduction.de/blender_opensource_dl

***Changelog:
0.2.2 - Fixed object’s orientation and scale bug during Procedural Loading
0.2 - Added Procedural Loading, Level of Graphics (LOD percentage), Some bugfixes, Thread optimization
0.1 - First release

***Work in progress:

  • Appropriate thread shutdown (if the game killed before the thread, raises an exception)
  • Distance templates (So you won’t need to add 3 different prop)
  • Object size priorities (So bigger objects will automatically loads at a larger distance than smaller ones)
  • Physics LOD
  • Object culling
  • Bugfixes
  • GUI for much more easier use.

***If you will use my script, I would be glad, if you mention my name somewhere :slight_smile:

2

Thanks for this great script! could this threading technique be used for other things like animated loading screens or something else?

3

Good question… Practically the answer is yes, but… loading objects is the main thread’s work, and unfortunately nobody can change this.
The solution is: you have to write a script (inside mine, if you want it to be on a separated thread) which will loads dynamically the objects and puts them in the main scene… so your game won’t stop while it loads new objects and can play animations.

In the next version of my script, I want to implement this function too, so stay tuned :slight_smile:

4

Sounds interesting . Can’t wait to give it a try.

5

Hey Zalán!

I lived in Hungary for 2 years a couple of years ago, Budapest is mad fun, really enjoyed it! :wink:

I am confused about the naming conventions. In your example the sphere seems to work with one mesh on the first layer Sphere, and two others on a hidden layer Sphere.001, Sphere.002.

I do not understand what you mean in 2) “If you use 2nddist and/or 3rddist, your object’s mesh’s name must end with the number of level it use”

Does this simply mean that if I use 2nddist I need: Mesh and Mesh.001, and if I use 3rddist I also need Mesh.002?

6

@martin.hedin:
You named with “Sphere.001 and Sphere.002” the object-names of the replace-objects.
You have to check/ use the mesh-name like Zalán said.
To find this mesh-name click at the properties-window at “Object Data” and you will find the name at the dropdownlist “Browse Mesh Data”. This will be the replace-mesh.

In this expamle it is “Sphere.L2” and “Sphere.L3”.
In fact different “objects” could have the same “mesh” (polygon-structure and all other settings like material).

I hope this would help.

7

@iFlowProductions

Ah I see! Ok now it works, I was able to add new meshes with LODs in the example! Thank you!

@Z4urce

This is awesome, Now I just need to take it for a spin see if it is fast :wink:

8

I’m glad to see that someone is interested in my script :slight_smile:

I’m came too late to answer your question @martin.hedin, but I’m glad to see that @iFlowProduction helped you. :slight_smile:

This script runs on my PC and notebook perfectly, since I wrote it, i got a massive performance boost against the old logic bricks LOD method. And I wrote this one to act as dynamically as it is possible.

If everything will going as I planned, I’m going to release the next version of it in this week, with new features, like dynamic loading, object and physics culling …etc. :slight_smile: so stay tuned.

I’m going to develop and optimize this script to the end of my open world game project. (The Swindler Alley)

9

Maybe the distances for 2nddist, 3rddist and LODdist could be specified in a python file, like different types of setups, and you would just have to specify a Game Property like a Boolean or String, choosing you LODsetup you specified in the python script. Would be easier if I could just add one property like “LODSet1 = (Boolean ticked)” to choose my setup instead of adding all those distances.

And that way you could also setup a graphics settings option, you know, High Graphics, Medium and Low, so that High would mean that 2nddist, 3rddist and LODdist are at 100%, and with Medium and Low would be like 75% and 50%.

10

Okay, I’m going to implement a solution for your proposal :slight_smile:

I give you a quick solution for that until I’m ready with the whole new version :slight_smile:


import bge
import os

def main():

    #you can read out the LOD quotient from a single text file 

    f = open(os.path.abspath('config.txt'), "r")
    userVar = f.read()

    #or you can adjust it manually
    #userVar = 1.0

    # 1.0 is 100% ... 0.5 is 50% ... 0.25 is 25% ... 1.5 is 150% of the distance given in the game properties.

    scene = bge.logic.getCurrentScene()
    for own in scene.objects:
        if "LODdist" in own:
            own['LODdist'] = own['LODdist'] * float(userVar)
        if "2nddist" in own:
            own['2nddist'] = own['2nddist'] * float(userVar)
        if "3rddist" in own:
            own['3rddist'] = own['3rddist'] * float(userVar)

This script only needs an always sensor, without pulse mode (it runs just once, reconfiguring the distances according to the config file [in the same folder, named config.txt] or the manually given userVar)

example:
Blend file with LOD: C:\blend\1.blend
LOD config file: C:\blend\config.txt … it’s content is “0.5” (without quotation marks) (0.5 = 50%)
… gives you the half of all LOD distances (if 30 meters before, 15 now) (if 100 blender units before, 50 now)

… so you can simply adjust the “graphic quality” with a simple external file :smiley:

11

Ah ok! That is very helpful! I am very new to Python, and this is helping alot! :yes:

12

You’re welcome :slight_smile: … and until I update my script don’t forget that you can easily copy any object’s game properties (including LOD props) with Object => Game => Copy properties … to your whole scene :slight_smile:

13

So useful, very clean,easy to use. A really useful resource! Thank you!

14

Thanks for the positive critique! It’s really motivates me :slight_smile: … I hope you all will be satisfied with the new version I just uploaded :slight_smile:

15

Does the new version free its ram on exit?

16

Well, to be honest, still couldn’t figure out how to stop a thread without exception. But I will do my best.

17

I see, thx.

18

The blend crashes on my computer.

19

When? In the opening?

20

In the openning.