Distributed single frame render for Blender Cycles / OS X, Linux & Windows

anon55414531 · September 5, 2015, 4:54am

This Python 3.4 script performs a distributed single frame render for Blender Cycles, spreading the workload over locally available machines, or it can simply automate the process of rendering a frame on a single external machine. It benchmarks the remote nodes specified (and caches the results) to determine the number of tiles required to distribute the workload fairly. It can be interrupted and restarted without losing tiles already rendered.

I’ve been using this for a while (it’s under active development), but thought some others might find it useful at the current state of development.

It works on OS X, Linux and Windows, you can get it here:

&gt;&gt; normal start
&gt;&gt; checking if render nodes are responsive
&gt;&gt; benchmarking
using cached benchmarks for all nodes
1920 * 1080, frame 1 with seed 0 is split into 14 tiles, arranged as 7 * 2 distributed over 3 nodes
&gt;&gt; rendering
tile 13 issued to node 192.168.0.129
tile  1 issued to node 127.0.0.1
tile  7 issued to node 192.168.0.128
tile  1 retrieved from 127.0.0.1
tile 12 issued to node 127.0.0.1
tile 13 retrieved from 192.168.0.129
tile  4 issued to node 192.168.0.129
tile 12 retrieved from 127.0.0.1
tile  5 issued to node 127.0.0.1
tile  4 retrieved from 192.168.0.129
tile  8 issued to node 192.168.0.129
tile  8 retrieved from 192.168.0.129
tile  6 issued to node 192.168.0.129
tile  5 retrieved from 127.0.0.1
tile 10 issued to node 127.0.0.1
tile  6 retrieved from 192.168.0.129
tile  3 issued to node 192.168.0.129
tile 10 retrieved from 127.0.0.1
tile  0 issued to node 127.0.0.1
tile  0 retrieved from 127.0.0.1
tile  9 issued to node 127.0.0.1
tile  7 retrieved from 192.168.0.128
node 192.168.0.128 is idle but will not be issued with a new tile, waiting for a faster node to become available
tile  3 retrieved from 192.168.0.129
tile 11 issued to node 192.168.0.129
tile  9 retrieved from 127.0.0.1
tile  2 issued to node 127.0.0.1
tile 11 retrieved from 192.168.0.129
no more work to give to node 192.168.0.129
tile  2 retrieved from 127.0.0.1
no more work to give to node 127.0.0.1
&gt;&gt; result written to: composite_seed_0.png
&gt;&gt; tidying up
&gt;&gt; finished

            node   tiles   mean duration (s)
       127.0.0.1       7                 772
   192.168.0.129       6                 853
   192.168.0.128       1                4185

                              total time (s)
                                        5415

anon55414531 · September 20, 2015, 5:27am

Quick update: the script has now been tested on Linux, and works just fine (Ubuntu 12).

elieli · September 30, 2015, 6:33am

Wow, this is something what we really need. Will be windows version available?

burnin · September 30, 2015, 12:52pm

Congrats on a great work. Very helpful!
+1 for windows

bnzs · September 30, 2015, 4:11pm

Thanks for script!

i to on windows…

anon55414531 · October 2, 2015, 11:32am

I don’t actually have access to any Windows systems, but… after some digging around maybe there’s something I can do with these VMs [1, 2]. I’ll take a look and see what the options are. I think with Cygwin and a few tweaks… it should be possible.

I’m also keeping an eye on Flamenco [3]

[1] http://pclosmag.com/html/issues/201309/page15.html
[2] https://dev.modern.ie/tools/vms/mac
[3] http://www.flamenco.io

marcatore · October 4, 2015, 12:05pm

I can just quote your question.
It could be very useful…

anon55414531 · October 7, 2015, 6:32am

I’ve downloaded one of those VMs; schedule permitting I’ll try and take an initial look at the start of next week.

anon55414531 · October 14, 2015, 1:50am

And the good news is:

Basic operation of the script has been tested and is working on Win XP Pro + Cygwin 2.2.1 courtesy of VirtualBox, with render nodes on XP and OS X running Blender 2.75 (127.0.0.1 and 192.168.0.129 respectively as per the screenshot). Commits have been made to the script on GitLab; documentation will be up in a day or so.

Attachments

anon55414531 · October 14, 2015, 2:08am

And the good news is:

Basic operation of the script has been tested and seems to be working fine on Win XP Pro + Cygwin 2.2.1 courtesy of VirtualBox, with render nodes on XP and OS X running Blender 2.75 (127.0.0.1 and 192.168.0.129 respectively as per the screenshot). Commits have been made to the script on GitLab; documentation will be up in a day or so.

anon55414531 · January 9, 2016, 1:07am

Now tested and working with the Raspberry Pi 2 (Raspbian 8.0), and much friendlier to flash based filesystems.