Open Source Blender Renderfarm: GSORF

Note: If this is not the right forum, please feel free to move this thread.

Hello Blenderheads,

during the last two semesters I was developing a Renderfarm especially for Blender, to be used within my university. It is focused on an easy setup and comfortable use. The software is written in C++, utilizing the great cross-platform Qt Framework and some libraries for video creation (FFMPEG) and security (OpenSSL).
We used it mainly for rendering stereoscopic 3D in 4K, which was not possible with Blenders internal renderfarm. This project was an attempt to make this possible, while using Blender in the background. Moreover, it is possible to use it for distributed tile rendering, for example if you’d like to render a big image over the network of several computers.

Here is a little teaser:
[video]http://www.gsorf.lwtv.de/video/gsorf_movie_english.mp4[/video]


The development is still in the alpha phase, but it might be useful for someone.

This is the project website, where you can download GSORF and look at the documentation:
http://www.gsorf.lwtv.de

The source code is licensed under the GNU GPL license, so feel free to download and compile.

You can also donate, if you like. I’d love to then use the donations to support the Blender Foundation in turn.

Thank you and Cheers,

-Adam

Hi, looks promising.
Got crash after first frame rendered or if I start client before server.
I use the Linux binary.
I cant find place to report on your webside, BT placed:

http://www.pasteall.org/54924

Server was running on same system as in signature or on:

Opensuse 12.3/64
CPU Intel T4200
GPU Intel GN 965

Cheers, mib

This looks cool. I cannot test it right now unfortunately but I will bookmark this for future reference. Thank you for releasing this to the community :slight_smile:

I got this when i ran the exe (the exe is actually there)

win10 x64 amd fx 8350

Hi,

OSX build doesn’t work on 10.9.5 - seems to be problem with link to QT library:

Dyld Error Message:
Library not loaded: /Users/Dominik_Rusiecki/Qt5.2.1/5.2.1/clang_64/lib/QtWebKitWidgets.framework/Versions/5/QtWebKitWidgets
Referenced from: /Users/USER/Desktop/gsorf.app/Contents/MacOS/gsorf
Reason: image not found

Tried the windows binarys and the file libgcc_s_dw2-1.dll is missing,
U other blenerguys can catch it on the link and throw it besides the exe and works…

^^please include the missing dll…

^^ maybe its the wrong dll -> it stopped working after it worked for a couple of minutes :wink:

^^^ is there a commandline ?
it would be nice to start a client with prameters for the settings, also hidden without gui…
-startclient -nogui -pathtobender -cpu/gpu

Hi all,

oh gosh! :frowning: I need a really good team to solve those problems :wink:
But I will try myself:

Hi mib. Okay, we didn’t test it on OpenSuse, it was running on Linux Manjaro (Arch-based distro) smoothly, at least it didn’t crash on the first frame :eek:

Thank you for pasting the bug report. I will work out a solution to implement kind of a bug report system, since so many people have random crashes, sigh. But wait! Just noticed, that github (the place where the source code is hosted offers something like that) offers a bug tracker: https://github.com/GSORF/renderfarm/issues

But I think I’ll point that out on the website as well. Thanks for the heads up!

Following the Blender idea and opening the source for others to learn from :slight_smile:

Windows 10?! :smiley: nice!
You might have the issue, that there is some .dll missing. For example I uploaded the “libgcc_s_dw2-1.dll” about an hour ago, because some users kindly told me they don’t have it on their computer. Please try to redownload the archive now, it is included. Otherwise it may be some firewall blocking.

Hi petrludvik - I was not the one building the Mac version, since I do not have one. I’d need to pass this issue to another person to handle fixing it. Sorry about that.

Yep, the missing .dll is a known issue, which I have already fixed in the currently uploaded Windows archive. :slight_smile:

It was planned initially, but as it is with student projects, some things (e.g. new features) get more important that might deteriorate the project quality and influence the progress. Furthermore I was the only one wanting to implement command line control, so… :wink:

Thank you all for the detailed descriptions and testing. I am very sorry for those errors you get. We did it as a student project and I was hoping to provide something useful to the community. If it doesn’t work on your computer I will try to address as many reports as I can, although I might not have the time to handle them all by myself, I’m afraid. Fixing bugs takes a lot of time.
Since this is an open source project feel free to add your own patches, if you have any suggestions. I will still try to work on it in my spare time (the project is officially over now and most of the team has left). We could turn it into a personal home renderfarm software everyone can use and get a nice time boost for one’s project.

Thank you again! Cheers!

couldnt try it out yet, however it looks quite cool. especially because i always had troubles when trying out the native blender network-rendering (disappearing and idling slaves). will try, as soon as possible.

i noticed that, from where i sit, your page is really, really, really slow. watching the clip without buffering for 5min wasnt possible and i can literally look at the page building itself up.

is it possible to run a client and a server on the same machine?

So it wont work without GUI?

did a test just now, running a server and a client on the same machine. started working just fine, very intuitive the whole thing. I could already see that it rendered a couple of frames, as it displays that in the client and the server (theye were also saved in the gsorf directory). but then it crashed (gsorf.exe has stopped working):

Problem signature:
  Problem Event Name:    APPCRASH
  Application Name:    gsorf.exe
  Application Version:    0.0.0.0
  Application Timestamp:    543c42ae
  Fault Module Name:    ntdll.dll
  Fault Module Version:    6.1.7601.18247
  Fault Module Timestamp:    521ea8e7
  Exception Code:    c0000005
  Exception Offset:    0002e3be
  OS Version:    6.1.7601.2.1.0.256.48
  Locale ID:    2055
  Additional Information 1:    0a9e
  Additional Information 2:    0a9e372d3b4ad19135b953a78882e789
  Additional Information 3:    0a9e
  Additional Information 4:    0a9e372d3b4ad19135b953a78882e789


i then started the server up again. first it looked good because he recognized the still running client. but then crashed again:

Problem signature:
  Problem Event Name:    APPCRASH
  Application Name:    gsorf.exe
  Application Version:    0.0.0.0
  Application Timestamp:    543c42ae
  Fault Module Name:    gsorf.exe
  Fault Module Version:    0.0.0.0
  Fault Module Timestamp:    543c42ae
  Exception Code:    c0000005
  Exception Offset:    0000db02
  OS Version:    6.1.7601.2.1.0.256.48
  Locale ID:    2055
  Additional Information 1:    0a9e
  Additional Information 2:    0a9e372d3b4ad19135b953a78882e789
  Additional Information 3:    0a9e
  Additional Information 4:    0a9e372d3b4ad19135b953a78882e789


i hope this can help you in some way. i’m on windows 7, actually just reinstalled it a few days ago, and updated everything i could think of. also i5-3570k, gtx 660. i also posted this in the bugtracker.

Hi there,

Yeah, I am currently also experiencing slow webserver responses. The reason for that seems to be the recent publishing of the renderfarm (today), so many people might want to see if this is something useful for them. It is a German server, so accessing it from switzerland shouldn’t be a location problem I think. Right now the page is unreachable also… :frowning:

And yes, you can run server and client on the same machine. Although my professor said we do not need to ensure that feature, it was important for me to implement it, since it made testing easier also.

The software relies on a so-called “Display Server”, so it might not start when you are using a text based OS. But you can still use it via the command line to e.g. start the Renderclient over SSH remotely on an other computer. Just the server will not be useful in that mode, since it requires you to drag and drop your .blend file in the GUI (And command line control is not yet implemented - as stated above).

Firstly, glad it worked for you in the beginning and especially, that you find it intuitive.
Though, it’s annoying you got a crash after some frames of course. I’ll investigate this further, trying to find the bug.

Best regards!

Better put a few ads up there then :P.

Yes, compared to the native blender netrenderer it’s very quick, simple and clear. And it works (apart from the crashes of course). I actually like the drag&drop feature. for a small-time user like me that’s perfect. for bigger solutions a command line thingy might be practical though.

@BlenderEi

Okay, verstanden. The master node I am using of course has a GUI, just the nodes I boot over PXE dont.
Looking forward to testing this over the weekend :slight_smile: Vielen Dank!

So is there any way to do this over WAN

Good evening!

Nah, I really just don’t like ads :wink: There wil never be any annoying ads, instructing you to click here or there.
What I would really like people to do, though, is to donate a small amount if they like the software or it being open sourced. This would help really a lot to improve GSORF and fund Blender! :wink:

Yeah, that was the main idea to provide something easy and quick, also usable for smaller / indie studios. The command line is on my feature request radar, and I will try to implement it, as soon as time permits. I have a “Blender job” (animation) now, I will need to finish first.

Alright, I hope you can use it for your project. In my view you should be able to start it via SSH with the “-nohub” option from the command line. It should automatically find it’s GSORF Server, as soon as it is running on the network. I’ve no experience with PXE, so I don’t know if it will start withouth having a display server. I am curious how it will work out in your setup environment. I wish you success!

That is a very interesting question. As our main focus was to make it available at our university, we didn’t have the need to make it available over the internet. Although, we did want to make it available worldwide, it was an interesting goal for us. I’m afraid, that in our current design this is not possible, since the way we handle the transfer of our status messages, is not suited for transmission over several subnets (we use UDP, or ARP respectively, which is not transmitted from router to router). So this is a hardware limitation, due to our initial software design. To implement it would be nice, but would require proxy servers or a dedicated database server. That’s why it currently only works on the local area network (LAN).

Hope I could answer your questions satisfactorily.

Kind regards!

maybe it works, if you connect the client via vpn to the server.

Hi… Is work OK for my… with TheBounty renderer!! Only some issue on my init.py when render is finished.
I downloaded the sources and i want make a look inside the code.
Thank you for an great work.
Greetings…


Has anyone else tried this program? It works fine on my system: one desktop pc connected to LAN cable, and a laptop via wi-fi.

wi-fi.

Hi povmaniaco,

awesome!
So you are sure, that the error comes from your init.py file? Looking at the GSORF client output, it seems so.
If there is no error in your python file, it might help to enable the automatic execution of python scripts?! For this we would need to add the “-y” option to the end of the blender process call (line 124, file blenderrenderengine.cpp).
Do you think this would help?

As for the idea with APCasa - it is quite good. We did try to get this to work, as well. There is a tool called “Hamatchi” [1], where we could establish a kind of VPN connection between a Mac and Linux system connected over the internet, while they remained in the same subnet (which ensures GSORF is still able to recognize nodes on other machines). This would make it possible to render across machines which are connected via the internet, making them think they are still in a local area network.

I am currently rendering a 108 MB file with GSORF. And I realized the renderfarm is working, but very slow in comparison with rendering directly in Blender. It seems like the problem for this is in the division into too many packets for TCP transmission and also we are doing too many UDP Broadcasts, so that they could also lead to duplicate renderings, due to queuing of status messages. Some technical stuff, but we will examine that after sleeping.

Happy Rendering!

[1] = https://secure.logmein.com/products/hamachi/