Up to now, network programming in blender has been a nightmare and virtually absent. Python threading did not work reliably inside blender, and so any script running for extended periods of time would block the user interface. Any code running outside the main process would not have full or reliable access to the API.
The larger Python community solved this problem using asyncio. In fact, Guido van Rossum himself designed a standard event loop library for concurrency and asynchronous input/output. Codenamed Tulip, this library is already included in the standard library and current distributions of Blender.
I’m not using any external renderers, so they would have to talk to me directly on what they want to accomplish. But yes, communicating with external processes could be a good use-case.
My next step is to work on a basic Http Framework to hook small web apps into. For example a button panel, a text editor or a touch enabled timeline controller. It would also include a communication channel using websockets, so that addons and apps can interchange information with relatively little latency.
My main interest in Blender is around video editing and creating simple 2D-ish animations for videos. So that’s what I am focussing on mostly.
I could imagine that people working on more professional projects would find asyncio more interesting for communicating with an asset manager. That could be a server running in a cloud, or even on the desktop, and multiple instances of blender from one or multiple work stations could communicate with that server.
If there is interest in making more out of asyncio for blender, it could be a good idea to polish up the asyncio-bridge plus the mini web framework and package it as an addon.
I’m currently working on a mini-webframework for blender instances. It will contain templating, static files, and a REST API. The REST API will allow web apps or external processes to access some of blender’s internals and data structures, while the web interface is geared towards building user interfaces that blender is naturally bad at: Text/Script editing, browsing data structures, “remote-control” through mobile devices, and human-readable reports.
I’m really looking forward to your development for the web framework it’ll be very useful !
Unrelated to this topic but I’ve stumbled upon this the other day http://verse.github.io/
Since my first message I’ve been reading about asyncio and I’ve got the gist of it but still there are many way to manage coroutines and I’m a bit confused.
As an example I wanted to integrate irc3 asyncio as a plugin.
Like you did with the blendernation feed I want to feed text object with the channel chat and be able to post to the irc channel.
So far I’ve built the UI and successfully connected to a room in a blocking way but I have no idea how to integrate the loops and start a forever running task from an operator and also be able to disconnect following this exampleto use a foreign loop the irc client get instantiated like this
loop = asyncio.get_event_loop()
bot = irc3.IrcBot.from_argv(loop=loop)
where asyncio was imported “from irc3.compat import asyncio”
which one should I use? Here is the gist with the code I wrote so far
it fails by giving me
RuntimeError: class CHAT_OT_connect, function execute: incompatible return value , str(, Function.result expected a set, not a generator)
To try out the code you need to add to your script/modules blender directory
irc3
venusian
docopt.py
open the gist in the text editor and run when a text object is active, a panel will appear
I really hope you have the time to take a look at it, thanks in advance
Uhm, yes, you are exactly running into the reason why I had to write a “bridge”. Look in the github repo mentioned in the original post. You will need the asyncio_bridge.py module. It implements a modal operator which emulates the asyncio event loop. I also haven’t tried it on Windows yet, but it should still work.
You are using “loop.run_forever()”, and then the loop, well, runs forever. But as I said, python code must never run for long periods of time without yielding to Blender’s event loop. That is true both for programs which are based on asyncio and those which are not. Operators are pretty much like asyncio “tasks” or concurrent threads, because they provide callbacks, they can consume input events or run loops by having certain callbacks called repeatedly and so on.
asyncio_bridge replaces “loop.run_forever()” with a modal operator…
I still haven’t gotten the code into a state that I want to share it. But I am pretty close, I think.
Also, an IRC Plugin may be quite hard to implement. Especially the input line… What I would probably do is use a text file in the text editor, and try to manipulate that…
Yes, the problem is that I don’t know how to use it How can I make a program that requires to be run in the background, like an irc client ongoing connection, be started by an operator that, like you said are tasks that have to return and die.
The example you provided of the http_server_demo use the asyncio.get_event_loop() so is not using your bridge with loop.create_server() right? What should be the right way to integrate irc3 into the blender bridge loop and still be able to “interact” with the process and send commands? I really need your help to integrate it
You mean that is not possible yet?
I thought about it, I’ve already have some working code to use the text editor, but for now a single line StringProperty is enough to start with and maybe enough in general since blender will never be a full fledged irc client, but the possibility to track live conversation inside the interface can be useful and fun beside the single line input line.
And then you can just start your Server “thread”. In your gist I think you only need to delete the “run_forever” line in the irc_connect function. Also that doesn’t need to be a coroutine or yielded from. You should only use yield/yield from when you want to either write a generator, or a coroutine that will run in the asyncio event loop. The operator’s “execute” however, is run in Blender’s “event loop” if you can call it that. Your whole code doesn’t contain a single coroutine, it just starts such threads using the irc3 library which I am not familiar with.
I’d like to add that asyncio isn’t easy to grasp. I also haven’t made up my mind how to package all of this as an addon. The first problem is that multiple addons should “share” the event loop, which isn’t so much a problem for asyncio, but rather for the addons which need to rely on the same code. So you can’t just distribute your version of asyncio_bridge.py, and hope it works with mine. Second problem is that distributing and installing blender addons isn’t as clear as it could be.
A solution for this packaging trouble would be if there were blender builds which are linked with python distributions and their dynamic libraries, for example the excellent Anaconda from Continuum Analytics. Then you would get binary packages and pip instantly. Having access to a compiler would also be great, especially for stuff like cython and the like. But “unfortunately” or “fortunately”, depending on whom you ask, Blender is predominantly used on Windows, where it can be a major pain to get a compiler running…
It works, it connects and joins the room in a non blocking manner, had to use run(forever=False) to instanciate the client see the updated gist. However now I don’t know how to access the running client since it doesn’t return a reference…!
I’m still confused when to use yield from since asyncio.get_event_loop() it’s been used in your server example like in the irc case.
About the packaging problem: Your bridge it’s just a pure module and all the other addon should refer to it and share the “loop”
I agree there is no good way to integrate with external modules.
As far as I know if you need to include an external module in your addon you either have to tell the user to put it in the blender or user-defined scripts/modules directory or sys append the path in your code. I would love that someone correct me if there are other way to do it.
And of course no way to easily manage them like with pip. Yeah Anaconda rocks
In asyncio, you need to use “yield from” in a coroutine, when you want to “call” another coroutine. Documentation for the irc library and for asyncio itself should contain information on which methods are coroutines.
You are not assigning an IrcBot instance to your “bot” variable but the result of calling “run” on a newly created instance of IrcBot, which is then lost without a reference.
Another important thing to note about “yield from” is that this is the only way to let other code be executed. Anything between two yield from statements (or in a @coroutine decorated function without a yield from statement) is executed in one go, blocking the whole process, like any other script in blender. So there is also “yield from asyncio.sleep(seconds)”, where you can let the rest of blender do its thing and resume after “seconds” seconds.
In the asyncio docs they say you should never use yield instead of yield from, at least in the context of coroutines. If you are using generators for other stuff, you are of course free to use it for that.
Maybe you could collaborate with some external renderer devs, like those from LuxCore, to help them make their integration better?
How can I make a program that requires to be run in the background, like an irc client ongoing connection, be started by an operator that, like you said are tasks that have to return and die.