BrainTrain3000 Comments - Answer Overflow

BrainTrain3000

Explore posts from servers

•Created by BrainTrain3000 on 8/29/2024 in #membrane-help

WebRTC TURN TCP/TLS configuration issue

Sorry for the long delay - getting a ton of logs on the server side, but I'm not sure what to look for. I'm not seeing anything pertinent

4 replies

WWindmill

•Created by BrainTrain3000 on 4/2/2024 in #help

Setting content-type in S3 File Uploader

Fair enough - thanks for clarifying!

4 replies

SMSoftware Mansion

•Created by BrainTrain3000 on 4/28/2023 in #membrane-help

Membrane.Source example for RTC Engine

I dunno if this helps, but here's our handle_pad_added:

@impl true
  def handle_pad_added(Pad.ref(:output, {_track_id, _rid}) = pad, _ctx, state) do
    structure = [
      child(:synthesizer_source, %SynthesizerElement{
        room_id: state.room_id
      })
      |> child(:decoder, Membrane.MP3.MAD.Decoder)
      |> child(:converter, %Membrane.FFmpeg.SWResample.Converter{
        input_stream_format: %Membrane.RawAudio{
          channels: 1,
          sample_format: :s24le,
          sample_rate: 44_100
        },
        output_stream_format: %Membrane.RawAudio{
          channels: 1,
          sample_format: :s16le,
          sample_rate: 48_000
        }
      })
      |> child(:encoder, %Membrane.Opus.Encoder{
        input_stream_format: %Membrane.RawAudio{
          channels: 1,
          sample_rate: 48_000,
          sample_format: :s16le
        }
      })
      |> child(:parser, %Membrane.Opus.Parser{})
      |> child(:payloader, %Membrane.RTP.PayloaderBin{
        payloader: Membrane.RTP.PayloadFormat.get(state.track.encoding).payloader,
        ssrc: state.ssrc,
        payload_type: state.payload_type,
        clock_rate: state.track.clock_rate
      })
      |> via_in(Pad.ref(:input, {state.track.id, :high}))
      |> child(:track_sender, %TrackSender{track: state.track, variant_bitrates: %{high: 48_000}},
        get_if_exists: true
      )
      |> via_out(pad)
      |> bin_output(pad)
    ]

    {[spec: structure], state}
  end

@impl true
  def handle_pad_added(Pad.ref(:output, {_track_id, _rid}) = pad, _ctx, state) do
    structure = [
      child(:synthesizer_source, %SynthesizerElement{
        room_id: state.room_id
      })
      |> child(:decoder, Membrane.MP3.MAD.Decoder)
      |> child(:converter, %Membrane.FFmpeg.SWResample.Converter{
        input_stream_format: %Membrane.RawAudio{
          channels: 1,
          sample_format: :s24le,
          sample_rate: 44_100
        },
        output_stream_format: %Membrane.RawAudio{
          channels: 1,
          sample_format: :s16le,
          sample_rate: 48_000
        }
      })
      |> child(:encoder, %Membrane.Opus.Encoder{
        input_stream_format: %Membrane.RawAudio{
          channels: 1,
          sample_rate: 48_000,
          sample_format: :s16le
        }
      })
      |> child(:parser, %Membrane.Opus.Parser{})
      |> child(:payloader, %Membrane.RTP.PayloaderBin{
        payloader: Membrane.RTP.PayloadFormat.get(state.track.encoding).payloader,
        ssrc: state.ssrc,
        payload_type: state.payload_type,
        clock_rate: state.track.clock_rate
      })
      |> via_in(Pad.ref(:input, {state.track.id, :high}))
      |> child(:track_sender, %TrackSender{track: state.track, variant_bitrates: %{high: 48_000}},
        get_if_exists: true
      )
      |> via_out(pad)
      |> bin_output(pad)
    ]

    {[spec: structure], state}
  end

15 replies

SMSoftware Mansion

•Created by BrainTrain3000 on 4/28/2023 in #membrane-help

Membrane.Source example for RTC Engine

My best guess is that the following is happening: 1. The API we're calling for the audio responds with a stream of chunks. 2. Those chunks are received very quickly (faster than the duration of the audio they contain), and we process them as they arrive. 3. We package these chunks into an RTP stream via Membrane.RTP.PayloaderBin. 4. Membrane forwards these packets to the client as soon as they arrive. 5. Because they're received by the client faster than it can play them back, it starts dropping packets. (see section 3.2 of https://www.rfc-editor.org/rfc/rfc7002) So...do we need to control the rate of playback on the Membrane side in order to make this work? I guess I assumed it would measure things out on its own.

15 replies

SMSoftware Mansion

•Created by BrainTrain3000 on 4/28/2023 in #membrane-help

Membrane.Source example for RTC Engine

Interesting, thanks! We're seeing no packets lost, but the majority of packets are discarded. That might speak to a timestamp issue...do you know how that might come about, and what we could do about it? Membrane Videoroom sounded just fine with two tabs!

15 replies

SMSoftware Mansion

•Created by BrainTrain3000 on 4/28/2023 in #membrane-help

Membrane.Source example for RTC Engine

@mickel8. Yep, that's mostly correct. I've made my own Source, similar to Hackney, but actually forwarding a stream of audio chunks from successive API calls. I've added that endpoint to the engine, and that track is being listened to by 2 endpoints: 1. The WebRTC endpoint from the user 2. A recording endpoint that writes the audio to disk. The recording endpoint records the audio just fine, it sounds exactly as it should. However, the WebRTC endpoint (or at least the audio element in the browser) receives the audio all messed up, missing large sections of it and generally sped up. Do you know why that might happen? I don't think the issue is with my Source/source endpoint, given that the recorder picks it up fine. The issue must either lie somewhere inside the WebRTC endpoint, or in the browser, I'd imagine. Any help would be much appreciated!

15 replies

Gaming

Programming