I wonder if that’s something Houdini offers out of the box, or if it’s all custom modules written for specific games ?
Those examples are all custom written for their respective game projects and are infused into the game engine via the Houdini Engine.
Proceduralism is the essence of Houdini
There is not much that is built-in in Houdini, just “basic” nodes out of which you makes complex networks that do crazy things.
Yup! That it is for sure. In the video they demonstrate tools that was made using Houdini’s core node based proceduralism exported as Houdini Engine for UE4/Unity. Awesome stuff!
I started digging deeper into Houdini some 6 months ago and in january this year, I had migrated fully from 3ds Max / Vray to Houdini Indie and never looked back. It’s a great application, immensly powerful, but for me the biggest gain is how much fun I’m having building the setups… 
About the game connection it’s basically a part of Houdini, creating OTL assets, that SideFX broke out of Houdini together with a runtime version of the main app for use in host applications, be it in Maya or in Unity or Unreal. So you create your setup in Houdini, collapse it into a subnet, create an UI for the controls you want to expose and save it as an OTL. This “asset” can now be used in Houdini or any of the engine host app’s. SideFX also offer a marketplace for these assets, so you can build them and sell them there, or buy one to implement in your host app - http://www.orbolt.com/
it’s just standard procedure