tintin Posts - Answer Overflow

tintin

•Created by tintin on 7/6/2024 in #membrane-help

Does Overlay plugin support 30 fps?

I'm interesting in using Membrane Overlay Plugin to do real-time video processing. However, after reading this discussion, the maintainer mentioned that Image library is not suitable for real-time frame processing (I assume this meant 30 fps because it is taking ~40ms per frame.

Full encode: 43ms

    Convert to YUV: 4.3ms
    3 x write to binary (each plane: 10.8 x 3 = 32.4ms
    So far 36.7ms (unaccounted for 7ms)

![reference](https://github.com/elixir-image/image/discussions/121#discussioncomment-8074166)

Full encode: 43ms

    Convert to YUV: 4.3ms
    3 x write to binary (each plane: 10.8 x 3 = 32.4ms
    So far 36.7ms (unaccounted for 7ms)

![reference](https://github.com/elixir-image/image/discussions/121#discussioncomment-8074166)

My input is 30 fps and I would like to overlay images then encode it to H264 for live stream. Is there any workaround so I can achieve 30fps? Could this plugin (https://github.com/membraneframework/membrane_live_compositor_plugin) work?

4 replies

SMSoftware Mansion

•Created by tintin on 6/25/2024 in #membrane-help

Modifying pipeline after it has been started

I would like to create/remove additional children to the pipeline after it has start been started and is :playing. I was able to create children after pipeline started but I was wondering if it's fundamentally wrong to do so. Let's say I have this simple membrane pipeline where it reads from a file, pass it to a tee, and have a sink that writes to a file.

defmodule SimplePipeline do
  use Membrane.Pipeline

  @impl true
  def handle_init(_context, _options) do
    spec = [
      child(:input, %MyFileSource{location: "input.txt"})
      |> child(:tee, Membrane.Tee.Parallel)
      |> child(:output, %MyFileSink{location: "output.txt"})
    ]

    {[spec: spec], %{}}
  end

defmodule SimplePipeline do
  use Membrane.Pipeline

  @impl true
  def handle_init(_context, _options) do
    spec = [
      child(:input, %MyFileSource{location: "input.txt"})
      |> child(:tee, Membrane.Tee.Parallel)
      |> child(:output, %MyFileSink{location: "output.txt"})
    ]

    {[spec: spec], %{}}
  end

Then I start this pipeline and it starts reading from a file. While it is still reading from a file, is it fundamentally wrong if I invoke a new spec action to add additional children?

# in the same SimplePipeline module
def handle_info(:add_new_sink, _ctx, state) do  
  spec = [
    get_child(:tee)
    |> child(:new_output, %MyFileSink{location: "output_2.txt"})
  ]
  {[spec: spec], state}
end

# in the same SimplePipeline module
def handle_info(:add_new_sink, _ctx, state) do  
  spec = [
    get_child(:tee)
    |> child(:new_output, %MyFileSink{location: "output_2.txt"})
  ]
  {[spec: spec], state}
end

(I do that by send(<Pipeline_PID>, :add_new_sink, nil) ) I tested this and it worked fine. I also looked at the source code of how Membrane.Core.Pipeline initializes its children and I don't see anything obvious that would break. I just want to make sure that this won't impact performance and break benefits that membrane gives (eg. shared memory buffer etc.) Follow-up question, does Tee use shmex to share buffer across multiple elements or each element will get a duplicate copy of the same buffer? Thanks!

4 replies

Gaming

Programming