Live video effects in fishjam
Hey, I am curious about how possible it is to implement live effects into fishjam webrtc streams from the server using a custom endpoint with something like video-compositor in real time. How possible is this with membrane currently?
Excuse the video but here is an example of an effect I am talking about https://www.youtube.com/shorts/4YaE5u1mjlc
3 Replies
Hi,
with Membrane itself, it should be fairly simple to implement such an effect using membrane_live_compositor_plugin.
You must construct a pipeline with LiveCompositor Plugin, register your shader with that effect, and use it on the input stream (input pad).
You can find a similar pipeline example here. We have a very similar shader in the LiveCompositor repo.
For info on shaders themselves, refer to this intro.
I'm not sure about integration with FishJam. @Jellyfish/WebRTC you help with that?
cc @mickel8 @kamilstasiak
You would have to create a new endpoint that will use membrane_live_compositor_plugin under the hood. This plugin. AFAIK, will send data to the Live Compositor, which will create an animation and send the video back to the fishjam endpoint. Then you just publish this returned video as a new track and it will be consumed by the other endpoints
More or less 😄
Great I appreciate the links and examples. I think I have a decent idea of what to do after both of your guys help. Cheers