Collaborative Render

Collaborative Render

I developed a tool to render in batch mode single or a sequence of images from one or more scenes, using one or many computers (using CPU or GPU).

It works only on Linux and with cycles.
You have to install the Postgres database (very simple in Linux)
You can ‘submit’ a task through an ‘add-on’.
The ‘slaves’ are shell clients that you launch and loop to procedd sub-tasks.
See the .ods file, Installation sheet for details.

Warning:

  • choose temp and dest directory different from your current directory:
    ++ the submit part copy <blende file>_EXR.blend in the destination directory
    ++ a lot of files are generated in the temp directory.

    It is the third beta revision, still a WIP.
    The <blende file>_EXR.blend generated may be used for quick compositing, the render layers are replaced by EXR mutli-layers files.

    To do list:

    • deal with blender version parameter (so that the clients will have a warning if their blender version is older),
    • deal with CPU/GPU limitations. You can launch ‘client’ with -d GPU option, but still lacks how to do when the submiter don’t want GPU renders and how to choose the graphic card
    • deal when one client take too long the render a sub task,
    • add token for clients (see if it’s necessary),

Updated 2014-09-10: Windows and OSX adaptations
> the bldclient.bash has been updated to bldclient.py, to be easier to adapt for windows and OSX. See WARNING in the .py files

If someone can test it in a single PC or network configuration, I will be glad to know if it is working (and how to parameter it for the network part).

Attachments

CollaborativeRenderV1R4.zip (149 KB)

Hi,

See #1 for a new version.
The corrections/improvments are:
- parameter the name of the blend file,
- add number of sample parameter,
- deal with CPU/GPU,
- deal with .blend file values versus parameters value used by shell program,
- assembly automatically render borders parts in one picture,
- assembly automatically render layers parts in one EXR multilayers picture (done with limitations).

The complicated part will be to proceed a subtask compositing, replacing render layer nodes by reference to the EXR multilayers picture generated.
It may be interesting to have in blender an option in the render layer node that let the user choose to use the render layer or a EXR layer picture.

Will be glad to have a feedback from users.

Hi,
See #1 improvments made:

  • render animations,
  • assembly automatically render layers parts in one EXR multilayers picture.
  • proceed a subtask compositing, replacing render layer by reference to EXR multilayers picture.
    Nice to use for compositing even if you have only one PC.
    I tested it with Mike Pan’s BMW file, adding one render layer (just a cube), naming the render layers RL1 and RL2, doing a small animation from frame 1 to 3, then launching 2 process on my PC: 1 with CPU and 1 with GPU.
    The result is OK, 3 EXR multi layers files and 3 PNG generated, the BMW.blend file copied in the dest directory, with render layers replaced by image sequences.

Very interesting ; )

@pitiwazou: thanks!
See #1, the ‘submit’ part is now an ‘add-on’ that write the task to the database, more easy to use.
The ‘client’ is a shell program but very simple.

Attachments


Hi,
I modified the programs to be easier to adapt for Windows and OSX.
See #1 thread.