nickdichev_fw Posts - Answer Overflow

nickdichev_fw

•Created by nickdichev_fw on 5/9/2023 in #membrane-help

Intermittent Failures with RTMP Sink

We are running into some intermittent failures with the RTMP sink. What we notice is that sometimes a given pipeline will stream all the way through and sometimes the RTMP sink will raise when writing frames and crash the pipeline.

22:01:58.807 [error] <0.5365.0>/:rtmp_sink/ Error handling action {:split, {:handle_write, [[:video, %Membrane.Buffer{payload: <<0, 0, 139, 207, 65, 154, 128, 54, 188, 23, 73, 255, 152, 130, 98, 10, 43, 88, 28, 64, 176, 10, 39, 247, 233, 179, 54, 27, 17, 168, 97, 24, 82, 152, 175, 21, 138, 252, 216, 108, 205, 134, ...>>, pts: 27240000000, dts: 27240000000, metadata: %{h264: %{key_frame?: false}, mp4_payload: %{key_frame?: false}}}]]}} returned by callback Membrane.RTMP.Sink.handle_write_list


22:01:58.808 [error] GenServer #PID<0.5369.0> terminating
** (RuntimeError) writing audio frame failed with reason: "End of file"
    (membrane_rtmp_plugin 0.11.2) lib/membrane_rtmp_plugin/rtmp/sink/sink.ex:269: Membrane.RTMP.Sink.write_frame/3

22:01:58.807 [error] <0.5365.0>/:rtmp_sink/ Error handling action {:split, {:handle_write, [[:video, %Membrane.Buffer{payload: <<0, 0, 139, 207, 65, 154, 128, 54, 188, 23, 73, 255, 152, 130, 98, 10, 43, 88, 28, 64, 176, 10, 39, 247, 233, 179, 54, 27, 17, 168, 97, 24, 82, 152, 175, 21, 138, 252, 216, 108, 205, 134, ...>>, pts: 27240000000, dts: 27240000000, metadata: %{h264: %{key_frame?: false}, mp4_payload: %{key_frame?: false}}}]]}} returned by callback Membrane.RTMP.Sink.handle_write_list


22:01:58.808 [error] GenServer #PID<0.5369.0> terminating
** (RuntimeError) writing audio frame failed with reason: "End of file"
    (membrane_rtmp_plugin 0.11.2) lib/membrane_rtmp_plugin/rtmp/sink/sink.ex:269: Membrane.RTMP.Sink.write_frame/3

this one crashed ~20 seconds into a 1 hour long stream, so I don't understand how it could have been the end of file. Additionally, in the error log I see an error handling a video buffer, however, the crash is in the write audio frame path.

22:57:02.461 [error] <0.3134.0>/:rtmp_sink/ Error handling action {:split, {:handle_write, [[:video, %Membrane.Buffer{payload: <<0, 0, 0, 7, 65, 154, 0, 122, 0, 63, 204>>, pts: 3626320000000, dts: 3626320000000, metadata: %{h264: %{key_frame?: false}, mp4_payload: %{key_frame?: false}}}]]}} returned by callback Membrane.RTMP.Sink.handle_write_list


22:57:02.461 [error] GenServer #PID<0.3138.0> terminating
** (RuntimeError) writing video frame failed with reason: "Invalid argument"
    (membrane_rtmp_plugin 0.11.2) lib/membrane_rtmp_plugin/rtmp/sink/sink.ex:285: Membrane.RTMP.Sink.write_frame/3

22:57:02.461 [error] <0.3134.0>/:rtmp_sink/ Error handling action {:split, {:handle_write, [[:video, %Membrane.Buffer{payload: <<0, 0, 0, 7, 65, 154, 0, 122, 0, 63, 204>>, pts: 3626320000000, dts: 3626320000000, metadata: %{h264: %{key_frame?: false}, mp4_payload: %{key_frame?: false}}}]]}} returned by callback Membrane.RTMP.Sink.handle_write_list


22:57:02.461 [error] GenServer #PID<0.3138.0> terminating
** (RuntimeError) writing video frame failed with reason: "Invalid argument"
    (membrane_rtmp_plugin 0.11.2) lib/membrane_rtmp_plugin/rtmp/sink/sink.ex:285: Membrane.RTMP.Sink.write_frame/3

4 replies

SMSoftware Mansion

•Created by nickdichev_fw on 3/28/2023 in #membrane-help

Problems specifying toilet capacity of Realtimer

I have a pipeline which I see fail intermittently on startup due to a toilet overflow of a realtimer element. The downstream element of the realtimer is a rtmp sink. In my pipeline I have specified the toilet capacity with via_in to the realtimer as shown in the docs:

      # shortened for brevity
      |> child(:h264_encoder, %Membrane.H264.FFmpeg.Encoder{preset: :ultrafast})
      |> child(:h264_parser, %Membrane.H264.FFmpeg.Parser{
        framerate: {30, 1},
        alignment: :au,
        attach_nalus?: true,
        skip_until_keyframe?: false
      })
      |> child(:video_payloader, Membrane.MP4.Payloader.H264)
      |> via_in(:input, toilet_capacity: 500)
      |> child(:video_realtimer, Membrane.Realtimer)
      |> via_in(:video)
      |> get_child(:rtmp_sink),
      ################################################################
      # shortened for brevity
      |> child(:aac_encoder, Membrane.AAC.FDK.Encoder)
      |> child(:aac_parser, %Membrane.AAC.Parser{
        in_encapsulation: :ADTS,
        out_encapsulation: :ADTS
      })
      |> via_in(:input, toilet_capacity: 500)
      |> child(:audio_realtimer, Membrane.Realtimer)
      |> via_in(:audio)
      |> get_child(:rtmp_sink)

      # shortened for brevity
      |> child(:h264_encoder, %Membrane.H264.FFmpeg.Encoder{preset: :ultrafast})
      |> child(:h264_parser, %Membrane.H264.FFmpeg.Parser{
        framerate: {30, 1},
        alignment: :au,
        attach_nalus?: true,
        skip_until_keyframe?: false
      })
      |> child(:video_payloader, Membrane.MP4.Payloader.H264)
      |> via_in(:input, toilet_capacity: 500)
      |> child(:video_realtimer, Membrane.Realtimer)
      |> via_in(:video)
      |> get_child(:rtmp_sink),
      ################################################################
      # shortened for brevity
      |> child(:aac_encoder, Membrane.AAC.FDK.Encoder)
      |> child(:aac_parser, %Membrane.AAC.Parser{
        in_encapsulation: :ADTS,
        out_encapsulation: :ADTS
      })
      |> via_in(:input, toilet_capacity: 500)
      |> child(:audio_realtimer, Membrane.Realtimer)
      |> via_in(:audio)
      |> get_child(:rtmp_sink)

However, I see the reported toilet size as 200 instead of the specified value when the pipeline. crashes:

10:58:09.755 [error] <0.591.0>/:audio_realtimer/ Toilet overflow.

Reached the size of 201, which is above toilet capacity (200)
when storing data from output working in push mode. It means that some element in the pipeline
processes the stream too slow or doesn't process it at all.
To have control over amount of buffers being produced, consider using output in pull mode
(see `Membrane.Pad.mode_t`).
You can also try changing the `toilet_capacity` in `Membrane.ChildrenSpec.via_in/3`.

10:58:09.755 [error] <0.591.0>/:audio_realtimer/ Toilet overflow.

Reached the size of 201, which is above toilet capacity (200)
when storing data from output working in push mode. It means that some element in the pipeline
processes the stream too slow or doesn't process it at all.
To have control over amount of buffers being produced, consider using output in pull mode
(see `Membrane.Pad.mode_t`).
You can also try changing the `toilet_capacity` in `Membrane.ChildrenSpec.via_in/3`.

Any thoughts on why the config isn't bubbling down to the realtimer?

4 replies

SMSoftware Mansion

•Created by nickdichev_fw on 3/22/2023 in #membrane-help

Confusion on usage of MP4.Demuxer.ISOM

Hi, I'm trying to write a simple pipeline which take in an mp4 source and streams it out via the RTMP sink. I have some general confusion on how to properly wire up the MP4.Demuxer.ISOM. 1. How do I decode/parse the output pads stream format downstream from the source? I haven't seen any examples or demos of transforming the MP4.Payload.{AAC.AVC1} 2. How to properly handle dynamically attaching the output pads for each track to downstream elements? If I want to handle the :new_track message to attach to some sink with pads in the :always availability mode (such as the RTMP sink) I can't attach that track to some grouping of elements which end at the sink temporarily. For example, if I get the :new_track notification for an AAC track I can't attach just the :audio pad of the RTMP sink, because when handling that callback there is no video pad to attach. 3. Is it better to handle the track determination statically? IE, should I ffprobe the mp4 file and parse the available tracks beforehand? 4. Does the demuxer only handle mp4 with embedded h264/aac? A container with h264/mp3 won't be able to be demuxed? Thanks!

12 replies

SMSoftware Mansion

•Created by nickdichev_fw on 1/18/2023 in #membrane-help

Confusion on video compositor timestamp offset

Hi 🙂 I'm working on a POC app and evaluating membrane for a project at work and I have some questions about how the timestamp offset for the video compositor plugin is supposed to work. First the goal of the POC app is to take an arbitrary list of videos and time offsets and dynamically stitch them together into one continuous video (and then stream the result somewhere but I haven't gotten that far yet). Some other members of my team chatted with some of the membrane contributors and they suggested we take a look at the video compositor plugin and provided us some skeleton code, however, I'm struggling to understand how the timestamp offset option that the compositor takes can be used to seek through the input. No matter what input I give for the timestamp offset (except a negative number which raises as expected) the video starts playing from the first frame. Does the timestamp offset not do what I am expecting it to do? My approach for the POC is to dynamically spawn (and remove previous children) at each cue point and attempt to provide the compositor timestamp offset to seek the new video to the correct offset. Is this the right approach? Thanks! Sorry for the long post 🙂

6 replies

Gaming

Programming