Cycles Standalone Engine

Hi, just for info, DingTo work on the standalone version of Cycles.
It is now possible to build it easy for Linux and OSX.
On Linux/Mac OS X, simply type “make cycles” inside the Blender source directory, to get a standalone build of the engine.

Here is the mail from him on BF-comitters:

Hi everyone,
I’d like to give you an update on the Cycles standalone renderer.

I worked on the code the past few days, it’s easier to build now and the
XML API has been updated a bit as well.

Wiki Page with infos and example xml files:
http://wiki.blender.org/index.php/Dev:2.6/Source/Render/Cycles/Standalone
Developer Task for discussion and feedback:
https://developer.blender.org/T38279
Screenshot: http://blender.dingto.org/cycles_standalone/standalone.png

Please get in touch with us, if you like to help out, and want to
integrate Cycles into other 3D programs for example.
I’d also like got get feedback on an improved API, I don’t think XML is
the best thing here.

Best regards,
Thomas

Cheers, mib.
EDIT: GPU is working but get some Cuda errors.
Image is saved if you close the render window.

Attachments


wooooooot woooooot

Some comments:

1- currently, scene description files are not XML since they lack a root element;

2 - the root element is also a great place where put a version number, fundamental for handling compatibility;

3 - the mesh description is simply horrible: please consider something like


<mesh>
     <vertices>
           <vertex x="1.000000" y="1.000000" z="-1.000000"/>
           ...
     </vertices>
     <faces>
            <face vertices="0 1 2 3"/>
            ...
     </faces>
</mesh>

which is much much simpler to create, debug and parse (imagine loading in one shot a single string which contains all the vertex position info for the Thai statue of the Stanford mesh repository… <gulp> <gasp> <crash…>).

4 - consider using DTDs/XSDs to specify the file formats (if I remember well, they should also allow you to use “semi-automatically” C++ validating parser engines).

5 - remember to allow external OSL files for defining shaders.

You appear to have doubts about XML; the only sensible alternative is an application of JSON which is surely much more compact. For example


"mesh" : {
    "vertices" : [
        [1.000000, 1.000000, -1.000000],
        ...
    ],
    "faces" : [
        [0, 1, 2, 3],
        ...
    ]
}

Another possibility for the mesh data would be to load natively OBJ files (v, vt, vn, f, g, o, usemtl elements, ignoring everything else considering it as if it was a comment).

Hi jpb06,
thanks for your comment.

The current API was never really intended for general use. Brecht wrote it in the beginning, before there was a Blender integration, for quick testing purposes only. I updated the API a bit , so more of the recent nodes/options can be used, but the core principal remained the same.

So yes, there are 2 paths which we can investigate:

  1. Improve the XML API:
    I like your suggestions, mesh handling can be improved for sure, and I also considered to use something like .obj or even Alembic later.

  2. Use something else:
    Your JSON example is simpler than XML, that is something I hoped for too. Probably easier to manually edit that by people too?

Added a simple Help message (h-key) to the standalone, can be expanded with more info later and/or show statistics.


Hi DingTo, thanks for keep going on this.
I would also go for Json format as is well supported all over and makes easier writing exported for other DCC applications.
(not to mention how easy is to parse it !!)

Is there any chance to build JUST cycle without having to build blender at all ?
Cheers.
L.

p.s
is there any documentation for what concerns the API, from c or c++ ?

Hi! Absolutly stunning and useful stuff. Of course, I’m not developer of applications and programms, but I really want if you’ll make or porting for 3ds Max or to some other 3d app.

Hi,
@hdd: If you are on Linux or Mac OS, you can simply type “make cycles” into the console. (On Windows I still fight some linker errors…) This will compile Cycles without any Blender code. But you need to have a Blender compile environment nevertheless, as Cycles uses some libs (Boost, OIIO, OSL…) so you either need to run the install-deps.sh script on Linux or get the precompiled libs for Mac. See here for how to set this up: http://wiki.blender.org/index.php/Dev:Doc/Building_Blender

When you have this, you can run “make cycles” and are good to go.

As for docs, the current XML API is not documented, but it’s not difficult to understand it.
You can read my example XML files: http://blender.dingto.org/cycles_standalone/

You can look into these 2 files for reference:

  • intern\cycles\app\cycles_xml.cpp, for general infos on the API, settings and node names.
  • intern\cycles\render
    odes.cpp for the Node socket names.

Example:
<glass_bsdf name=“monkey_closure” distribution=“Beckmann” ior=“1.4” roughness=“0.2” />
The node type (glass_bsdf) would be in cycles_xml.cpp, the other properties (ior, roughness…) would be in nodes.cpp. “name” can be chosen by the user.

I hope this helps a bit.

@odil24: Yes it would be nice to see Cycles in other 3D programs, hopefully some developers will be interested. :slight_smile:

Dear DingTo, second part of the brainstorming.

Since you are already thinking about OBJ/Alembic, for the mesh tag I would go for something like


&lt;mesh src="./objects/suzanne.obj" format="obj"/&gt;

or, in case of more complex formats (Alembic looks to me as a Collada-from-hell with the added “feature” of being a binary format :evilgrin:) which require additional parameters


&lt;mesh src="./objects/suzanne.abc" format="almebic"&gt;
    &lt;option name="opt0" value="val0"/&gt;
    .
    .
    .
    &lt;option name="optN" value="valN"/&gt;
&lt;/mesh&gt;

Your example scene_monkey.xml becomes something like


...

&lt;transform rotate="180 0 1 1"&gt;
    &lt;state interpolation="smooth" shader="monkey"&gt;
        &lt;mesh src="./objects/suzanne.obj" format="obj"/&gt;
    &lt;/state&gt;
&lt;/transform&gt;

...

I don’t know your schedules, personally I would whip a skeletal OBJ reader and immediately forget about custom XML file formats for the mesh data. People who are going to write exporters have already access to OBJ export routines so that part of their work is nearly ready (and they are going to thank you). B.t.w., with the above structure, you have an extensible system which can be arbitrarily extended in future to other file formats.

Since we have buried the mesh data issue (the one with size issues), I think that the remaining content (mostly shader code) does not need to go to JSON. XML is a more verbose then JSON but, since we are dealing with files of few KBs the difference is not that significant and you have already the code working.

Hey,
thanks for the suggestions, I will discuss this with Brecht a bit, but I think .obj is the right format for now (and later maybe alembic).

In other news, thanks to Juicyfruit, Cycles Standalone can now be compiled on Windows as well.
You can download it here, among with the example files. http://wiki.blender.org/index.php/Dev:2.6/Source/Render/Cycles/Standalone
Have fun. :smiley:

Someone should develop a Blender exporter for this render engine, I tested it and it is pretty good.

kidding haha. This is cool perhaps one day 3dsmax/maya users can make use of this as an option to alongside options like NOX/ Corona etc (which is pretty popular).

Added basic support for external OSL shaders, just allows 1 output socket to be connected for now.
https://developer.blender.org/rB12109dd18e622a5c469c4fb5b802edfee0d2b2de


Updated the Windows x64 build and the example files for it: http://blender.dingto.org/cycles_standalone/

Alembic is the way to go!

This is pretty neat, I just noticed recently in the build options that a Cycles standalone could be compiled. I never did give it a shot, but I’m glad there’s an x64 Windows build now. Thanks to whoever did that :slight_smile:

Is there any kind of even rudimentary documentation for the .xml file yet? Also, is hair supported yet?

Edit: shoulda read the wiki page first: "Cycles standalone currently has a XML API, which is used for mesh data and shaders/settings. There is no API documentation at the moment, but you can check cycles_xml.cpp in intern/cycles/app to get an understanding"

I have a question on this stand-alone renderer and how Blender handles rendering in general. For network rendering, does every single render node have to have the entire project loaded in order to contribute to the rendering? The reason I ask is because I’m thinking it might be possible to create a wide-area network community render farm using stand-alone render clients. But if a huge scene needs to be sent to all render nodes then that might make it impossible or undesirable. Thoughts?

Pretty much. The renderer needs to have the scene data it will be working on in local RAM in order to do anything at all efficiently. The simplest way to do that is just to have each node open the scene itself, which is usually stored on some kind of network shared storage. For serious scenes it can be awkward without at least gigabit ethernet between render nodes and the storage device.

wow got distracted a second and see what a thread !
thanks for the inputs I’ll have a look at it !
L.

A DLL/.so/dylib build would be wonderful for integration in other applications. A standalone executable is great for network renders, but for tight and responsive interactive viewport rendering, a shared library is IMHO the better path.

If cycles connects with maya and becomes mature it may definitely become the choice of primary render engine for many studios.

I updated the XML API further, almost all nodes are now wrapped (exception RGB Curves, and Color Ramp).
And I added an interactive viewport. https://www.youtube.com/watch?v=G-nqYzgj85M

Some more fixes and improvements will be added, stay tuned. :slight_smile: