Result got cast to signed short.
It is physically impossible for me to resist answering these sorts of questions.
Platform specific, but typically involves a mutex-protected doubly-linked free list above contiguous chunks allocated by the OS, subject to fragmentation over time that can prevent the OS reclaiming unused memory. As allocators go it’s simple and gets the job done with the assistance of its charming counterpart free(), without attracting the infinite fountain of controversy that is a garbage collector of any kind. (It merely attracts a massive but theoretically bounded amount of controversy.)
…listen, uh, I can sort of see where this is going. You should probably stop asking things.
Well, it would push to a concurrent task queue, then the consumer thread would load and interpret the resources, adding them to synchronised lists, and signal success.
Oh. I… I’m sorry. I drifted off for a second into a lovely daydream. Forgot where I was. What was your question again?
reads
…
for the love of all that is good and beautiful in this world what have you done
Okay. I’m going to understand this. All of it. Just remember, you asked me to do this. In case I die of malnutrition, it’s been an honour to develop games with you. All of you. Here we go!
LibLoad starts off with a typical python wrapper resembling tubifex roadkill, encapsulating a C function gLibLoad from KX_PythonInit.cpp. We call one of two C++ methods depending on whether it’s loading from memory or from file: LinkBlendFileMemory from KX_BlenderSceneConverter.cpp, which calls C function BLO_blendhandle_from_memory from readblenentry.c, which calls blo_openblendermemory from readfile.c, or LinkBlendFilePath, which calls BLO_blendhandle_from_file (‘path’ went on sick leave), which calls blo_openblenderfile. The only slightly marred naming symmetry of these invocation chains is also seen in the implementations, since the first and second pairs of functions are 95% character-for-character identical in their implementation.
Anyway, moving on. Our LinkBlendFile variants both have handles to their respective .blend files, and conclude by calling … well, LinkBlendFile, which emphatically denies it is going to measure how long anything takes with //TIMEIT_START(bge_link_blend_file);. It goes on to say /* only scene and mesh supported right now */, which the official API documentation helpfully clarifies as actually meaning “only “Action”, “Mesh” and “Scene” are supported”. After a little book-keeping it calls load_datablocks three times so things are ‘properly loaded’ (functions are like that sometimes, you gotta keep hammering until they stick), and then it, well, uhm…
… then it closes the handle. And says /* done linking */.
So, to summarise, at this point, it has read the data from the .blend and interpreted it into real, usable Blender structures. All of that happens regardless of the promise of an asynchronous LibLoad; the file is opened and closed before LibLoad returns. Intruiging! …yes, definitely intruiging! Keeping calm and carrying on.
After the actual task of loading the library from file is accomplished, we create a KX_LibLoadStatus and start getting excited about the promise of at least doing something in another thread. What is left is to convert the Blender structures into BGE structures. There are three possible things to convert: Meshes, Actions, or Scenes. Of these, meshes and actions are converted immediately, presumably because scenes were the most requested type (fair enough). Scenes must surely link to the previous datatypes anyway, but I won’t digress. Forward unto concurrency!
We begin by pushing all scenes to a list, then create a new thread that runs async_convert. This function creates BGE scenes from that list; each time it finishes a scene, it adds an amount of progress. Thus, the usefulness of the progress percentage increases with the number of scenes you are loading, and the speed at which the real library load occurs. Therefore it is recommended you subdivide libraries into as many scenes as you can, and bundle an SSD and ramdisk software with your games.
After all the scenes are converted, we deallocate the list created in the main thread, and in turn, leave one for the main thread to deallocate. Cross-thread memory management: for fun and profit and furious tears of regret.
Back in the main thread, in KX_KetsjiEngine::NextFrame, we poll for scenes to merge because software design. According to documentation, merging scenes is estimated (or rather, forced) to account for 10% of the work actually tracked by the progress percentage, and is finished within a single frame. This means that creating a heavyweight thread, converting the scenes, and destroying the thread, must theoretically be done within 9 frames, or 0.15 seconds. That’s… that’s a really fast scene load. We couldn’t have intentionally created a concurrent loading mechanism that causes a major frame freeze regardless, right? That would make it useless for dynamic loading! a… all those thousands of lines of code… well, regardless, it’s theoretical time well spent, since it was in another thread, and multithreading is good. And loading screens can be useful and let the player know how much time is left with a progress b- … well, it can entertain them with a little spinning doodad.
We can make the loss back, though: as bad as the progress feedback is and always will be until a major rewrite, there might be Pythonic progress callbacks in future, allowing you to receive the progress metric whenever it changes, because optimising your progress bars to only update when the value changes represents a performance boost worthy of a few more dead tubifex colonies. Especially since it changes so infrequently! And we’ll get right on that, when Python’s Global Interpreter Lock stops being so global.
But of course, we should now discuss how the LibLoad functionality infects almost every other function associated with BGE data conversion with a ‘libloading’ argument to tell it what context it’s operating in, because cross-cutting concerns are a truly fantastic opportunity to weed out the people who can’t keep more than twenty files open at once what am I doing with my life.
…seriously people, I have nothing but respect for the BGE developers. They have made brilliant choices under the circumstance of building on an existing codebase, especially as earlier developers did not plan for the BGE’s current direction, much less actual existence. However, as always, the intersection of multiple brilliant development efforts operating with different goals and awareness of problems has … not panned out. And the consequences are evident. My criticism is intended to be affectionate and productive if at all possible. Also, if I got anything wrong about how libload works, I apologise immediately and profusely. I didn’t dig especially deep.
just a bit
Something for everyone to think about:
is a full-featured FPS in 96 kilobytes.
The BGE can present absolutely nothing in 107 megabytes. In case the 7 is distracting you, that’s more than a tenth of a gigabyte, for the minimum engine distributable. With 7z Ultra compression it becomes 23.8 megabytes. ‘Towering pillar’ is insufficient. Perhaps ‘hulking planetoid’?
[SUB]I AM THE ALPHA GRINCH grinds teeth at happy things[/SUB]
