[C++/Qt]: Export .blend with procedurally generated mesh?

BlenderEi · February 15, 2015, 10:10am

Hi all,

The title says everything I would like to know how one can write a .blend file within a custom C++ application (in my case with the Qt framework). Of course I did some research already, but I am currently not moving anywhere interesting… Many threads on the internet ask for a solution to read a .blend file and are advised to use other/simpler formats like e.g. .obj-files. This is a rather disapointing solution for me, since I am generating higher dense meshes from point cloud data and importing them into Blender in a different file format than the native .blend would be very slow (Python).

Could anyone of you please guide me in the right direction regarding my question?

My current state:

The Blender source code will be needed for that, I’m pretty sure. There is a Blender file format specification written by Jeroen Bakker in 2009/2010 (http://www.atmind.nl/blender/mystery_ot_blend.html). It gives a great first insight, but using the blender source for writing a .blend is a different story.

Looking inside the source code there is a “blenloader” module that seems to have the desired functions. So far I am trying to get this code running:

#include "blender/source/blender/blenloader/BLO_writefile.h" 

MainWindow::MainWindow(QWidget *parent) : QMainWindow(parent), ui(new Ui::MainWindow)
{
     ui-&gt;setupUi(this);

     Main* mainVar;
     ReportList* reportList;
     const int* thumb = NULL;

     BLO_write_file(mainVar, "Hello.blend", 0, reportList, thumb); //undefined reference
 }

I am stuck with the error message of a “missing reference”, which I would basically solve by adding a library in the project settings. Unfortunately I have no clue how to proceed from here, since I don’t know what the library might be called.

So the steps I need to get solved are:

Using the correct Blender source files in Qt Creator
Linking the source files correctly in Qt Creator
Creating the needed SDNA and SRNA structures for World, Scene and Mesh
Assembling the complete data structure from the scene components
Using the correct blender source code functions to write those data structures into a .blend file (hopefully already my example above)

The discoveries from this thread will be used in my scientific paper about point cloud visualization, so eventually this will also get published. I will also try to document it in a proper way for others to learn from.

Looking forward to some help of our Blender experts

Thank you in advance!

BeerBaron · February 16, 2015, 4:36am

This is a rather disapointing solution for me, since I am generating higher dense meshes from point cloud data and importing them into Blender in a different file format than the native .blend would be very slow (Python).

I think this assumption is highly questionable. How fast do you need the import to be, really? If the simplest case (like just using OBJ) is too slow, consider writing out a more efficient custom binary format and use the Python API to create a mesh from that. There are functions like foreach_set that are fairly efficient.

rebellion · February 17, 2015, 5:33am

This is for importing, but it may help you http://assimp.sourceforge.net/lib_html/importer_notes.html, it said that they have reimplemented a self contained SDNA system, that you might use with Qt, but as BeerBaron suggest, and the page of the link too, it’s better to write a custom binary format…If you don’t need the advanced visualiztion features of blender, you can always write your own viewer; with Qt is very simple, it offer all the class needed for a good OpenGL application.

lordloki76 · February 17, 2015, 5:41am

You can check the 2 main blend external loaders/writers: FBT and bParse. Both loaders/writers were developed for gamekit
https://github.com/gamekit-developers/gamekit/tree/master/Tools/FileTools (code)
http://gamekit.org/forum/viewtopic.php?f=9&t=15 (FBT vs bParse)
https://code.google.com/p/gamekit/source/browse/branches/AnimKit/Samples/BlenderLoader/akBLoader.cpp#176 (little FBT example)
http://gamekit.org/forum/viewtopic.php?f=9&t=185 (general)

BlenderEi · February 20, 2015, 3:18pm

Hi all,

Sorry for my late answer, I have been pretty busy with my PointCloud2Blender Converter.

Well, I’d like to back up my “dream” with some background info. Getting the opportunity to write a .blend file from my c++ program will result in the fastest way of importing into Blender, since this file format is the native one Blender understands. Therefore, of course I am aiming to get the best result / performance possible.
On the other hand, although not being my initial goal, I like your suggestion of using a binary format. Of course, the “plan b” of my development is still to use the alternative formats, but my past experience with importing huge CT-Scan-Data as .ply files for instance, was very disappointing. I was waiting for hours for the import, whereas after the import succeeded the respective .blend file of the high dense mesh opened quite fast. You are right, using a binary format would still be an efficient way while avoiding writing the .blend format, but - on the contrary - it would be yet another custom file format that people could only use with a proper Blender import script. You see the point I am trying to make here? I am not against your idea, it is a great alternative! I just hope that a .blend file would be a much wiser way to go, though much much harder to implement.

Yeah, I “fell in love” with Qt just the way I did with Blender and Python. It is not that much about visualizing things, but rather to cut / shorten the pipeline between doing a laser scan, preprocess and mesh it with my converter tool and get the result into Blender. Having a custom third party file format seems to me like a slowdown of the whole process.

Uh, those links look nice! I came across bParse some time ago, but understood it as a pure .blend importer. Especially the FBT-part looks very promising and worthy to take a closer look at. I think you gave me something I can ponder how to do over the next few weeks.

Thank you everybody for your help!

By the way, since last week my repository is set up. If you are interested how the converter progresses (if you have any suggestions, advices, requests: even better!), you can download the source here: https://github.com/GSORF/PointCloud2Blender

If I get some new insights into this problem, I will share them, naturally.
See you then.

BeerBaron · February 23, 2015, 12:29am

I wouldn’t be so sure about that. It so happens that .blend files are mostly a memory dump, so that might be true. On the other hand, reading raw vertex data from disk and writing it into a mesh datablock using foreach_set has very low overhead (if your source supports the buffer interface, like array).

I haven’t bothered to test which is faster, but either one should be “fast enough”.

On the other hand, although not being my initial goal, I like your suggestion of using a binary format. Of course, the “plan b” of my development is still to use the alternative formats, but my past experience with importing huge CT-Scan-Data as .ply files for instance, was very disappointing. I was waiting for hours for the import, whereas after the import succeeded the respective .blend file of the high dense mesh opened quite fast.

That sounds like the .ply importer is somehow broken, either at an algorithmic level or by using the API in a wrong way. You shouldn’t assume from that experience that this is Python’s fault, which should be on the order of 20x to 100x slower than C code when interpreted - but which can also be near-native speed when predominantly using C extension modules.

You are right, using a binary format would still be an efficient way while avoiding writing the .blend format, but - on the contrary - it would be yet another custom file format that people could only use with a proper Blender import script. You see the point I am trying to make here? I am not against your idea, it is a great alternative! I just hope that a .blend file would be a much wiser way to go, though much much harder to implement.

The “people not needing an addon”-argument does make sense. I don’t think it’s “wise” to spend this kind of effort on that bit of convenience, however.

It is not that much about visualizing things, but rather to cut / shorten the pipeline between doing a laser scan, preprocess and mesh it with my converter tool and get the result into Blender. Having a custom third party file format seems to me like a slowdown of the whole process.

You can just associate your file format with a command that calls blender with the import script running at startup. That way, it will open almost like a native .blend file.

8Observer8 · February 24, 2015, 12:49pm

This is my example how I load sphere by Qt, Assimp, with diffuse+specular lighting: https://github.com/8Observer8/ColorSphere

BlenderEi · February 26, 2015, 4:58am

Hi!

Thank you for your detailed answer. You sound like you are experienced with those things. I will have a look into that specific Python function and see what it can do for me.

Yeah, I am somehow biased here. Initially I was ready to spend effort on that bit of convenience, but today I see how much trouble this gets me into and might eventually even cause problems with meeting the dead line of my thesis. So, yeah, maybe it is wiser to stick to the easier ways than spending most of the time thinking about how to decypher the .blend. Furthermore, the Blender Python API offers a huge set of opportunities to get many things done (I could also choose to connect the apps via TCP or Serial for example). Still, I believe it shouldn’t be a problem to get that simple memory dump out as a .blend file, as long as I have the internal datatypes (which should be available to me, due to Blender’s open nature).

You Sir, just mentioned a killer argument. I like that approach very much, because it’s a really nice way to get efficient data exchange between the two apps and still be very user friendly.
Great, thank you for this helpful idea. Be sure to be quoted in my paper (everyone in this thread) By the way the paper is published at the aforementioned GitHub repository and will constantly be edited.

Though not really what I was researching for, but still a nice example how to use modern (!!!) opengl with qt. This is something I wanted to tackle this weekend anyway (generating a 3D mesh from the beforehand generated 2D panoramic images and displaying it in the app, before - as a last step - exporting them to a (Blender) 3D file format).
//EDIT: Now I now where I know your avatar image from. I saw your tutorials on Youtube. They have been like the only really helpful tutorials showing how to get modern OpenGL working with Qt (though, in russian, but yet quite easy to follow). Thank you for that also, while you are here. (Source: https://www.youtube.com/watch?v=jTHvFGjy5uA&list=PLHlHFavas54e0-05vZcZ0wNGhnpRyFmLc)

I will now mark this thread as solved, though I am a bit sad that it seems to really be that hard to implement a .blend export function.
So, if you have further ideas, please feel free to post them! Others will then be able to learn from your answers.

Thank you and have a great time.