enzoqtvf Posts - Answer Overflow

enzoqtvf

•Created by enzoqtvf on 1/10/2025 in #membrane-help

How to split a Raw Audio Buffer with 2 channels within frame into two different buffer

Hey team ! I'm trying to process an FLV stream with AAC codec for the audio, and the audio part has 2 channels that I would like to treat separately, is there a way to split my pipeline in order to handle the 2 channels differently? Here is an overview of the pipeline:

      child(:source, %Membrane.RTMP.Source{
        socket: socket
      })
      |> child(:demuxer, Membrane.FLV.Demuxer)
      |> via_out(Pad.ref(:audio, 0))
      |> child(:audio_parser, %Membrane.AAC.Parser{})
      |> child(:audio_decoder, Membrane.AAC.FDK.Decoder)
      |> child(:converter, %Membrane.FFmpeg.SWResample.Converter{
        output_stream_format: %Membrane.RawAudio{
          sample_format: :f32le,
          sample_rate: 16_000,
          channels: 2
        }
      })

      child(:source, %Membrane.RTMP.Source{
        socket: socket
      })
      |> child(:demuxer, Membrane.FLV.Demuxer)
      |> via_out(Pad.ref(:audio, 0))
      |> child(:audio_parser, %Membrane.AAC.Parser{})
      |> child(:audio_decoder, Membrane.AAC.FDK.Decoder)
      |> child(:converter, %Membrane.FFmpeg.SWResample.Converter{
        output_stream_format: %Membrane.RawAudio{
          sample_format: :f32le,
          sample_rate: 16_000,
          channels: 2
        }
      })

So I would like after that (or maybe the split should happen before the decoding and resampling), to have an element that provide 2 output with one channel each. Thanks ! 🙂

3 replies

SMSoftware Mansion

•Created by enzoqtvf on 6/20/2023 in #membrane-help

How to pass some client side parameters to an RTMP pipeline

Hey team I have an RTMP pipeline (which I simplified for the purpose of the question):

  def handle_init(ctx, socket: socket) do
    Logger.info("Starting RTMP pipeline")

    structure = [
      child(:source, %Membrane.RTMP.Source{
        socket: socket,
        validator: Membrane.RTMP.DefaultMessageValidator
      })
      |> child(:demuxer, Membrane.FLV.Demuxer)
      |> via_out(Pad.ref(:audio, 0))
      # we use a fake sink to terminate the pipeline
      |> child(:fake_sink, Membrane.Fake.Sink.Buffers)
    ]

    {[spec: structure, playback: :playing], %{}}
  end

  def handle_init(ctx, socket: socket) do
    Logger.info("Starting RTMP pipeline")

    structure = [
      child(:source, %Membrane.RTMP.Source{
        socket: socket,
        validator: Membrane.RTMP.DefaultMessageValidator
      })
      |> child(:demuxer, Membrane.FLV.Demuxer)
      |> via_out(Pad.ref(:audio, 0))
      # we use a fake sink to terminate the pipeline
      |> child(:fake_sink, Membrane.Fake.Sink.Buffers)
    ]

    {[spec: structure, playback: :playing], %{}}
  end

Which is served by an TCP server:

 {Membrane.RTMP.Source.TcpServer, rtmp_server_options()},

 {Membrane.RTMP.Source.TcpServer, rtmp_server_options()},

defp rtmp_server_options do
    %Membrane.RTMP.Source.TcpServer{
      port: rtmp_port(),
      listen_options: [
        :binary,
        packet: :raw,
        active: false,
        ip: rtmp_ip()
      ],
      socket_handler: fn socket ->
        # On new connection a pipeline is started
        {:ok, _supervisor, pipeline} = MyPipeline.Pipeline.start_link(socket: socket)
        {:ok, pipeline}
      end
    }
  end

defp rtmp_server_options do
    %Membrane.RTMP.Source.TcpServer{
      port: rtmp_port(),
      listen_options: [
        :binary,
        packet: :raw,
        active: false,
        ip: rtmp_ip()
      ],
      socket_handler: fn socket ->
        # On new connection a pipeline is started
        {:ok, _supervisor, pipeline} = MyPipeline.Pipeline.start_link(socket: socket)
        {:ok, pipeline}
      end
    }
  end

This works great, I can initiate an rtmp session using this kind of url for example: rtmp://localhost:5000 Now I'm trying to use url like this: rtmp://localhost:5000?rtmp_id=xxxx or even something like this: rtmp://localhost:5000/xxxx/ So within my pipeline I can get this xxxx and use if for different things (maybe you can think of a filename which could be anice usecase) So far I don't manage to find how I can get this xxxx within my pipeline, happy to know if there is anotherway to do that 😄 ! Thanks

4 replies

SMSoftware Mansion

•Created by enzoqtvf on 5/9/2023 in #membrane-help

Pipeline with RTMP source and AAC decoder

Hi everyone 🙂 (message can be redundant with the one I posted on slack, sorry about that) I'm currently building a POC for live transcription using whisper model. I checked Lawik example already which uses either the mic as or a file as a source to achieve it. In my case I do have RTMP source with AAC codec for audio and flv format. I've built a pipeline which seems to be "ok":

  def handle_init(_ctx, socket: socket) do
    structure = [
      child(:source, %Membrane.RTMP.SourceBin{
        socket: socket,
        validator: Membrane.RTMP.DefaultMessageValidator
      })
      |> via_out(:audio)
      |> child(:decoder, Membrane.AAC.FDK.Decoder)
      |> child(:converter, %Converter{
        output_stream_format: %RawAudio{
          sample_format: :f32le,
          sample_rate: 16000,
          channels: 1
        }
      })
      |> child(:fake_sink, Membrane.Fake.Sink.Buffers)
    ]

    {[spec: structure, playback: :playing], %{}}}
  end

  def handle_init(_ctx, socket: socket) do
    structure = [
      child(:source, %Membrane.RTMP.SourceBin{
        socket: socket,
        validator: Membrane.RTMP.DefaultMessageValidator
      })
      |> via_out(:audio)
      |> child(:decoder, Membrane.AAC.FDK.Decoder)
      |> child(:converter, %Converter{
        output_stream_format: %RawAudio{
          sample_format: :f32le,
          sample_rate: 16000,
          channels: 1
        }
      })
      |> child(:fake_sink, Membrane.Fake.Sink.Buffers)
    ]

    {[spec: structure, playback: :playing], %{}}}
  end

When I start it it initialise without issues however it doesn't do anything (from what I can see) (I see debug logs showing elements received the play request) I would be happy to get advises on that one :D, thanks !

10 replies

Gaming

Programming