spscream Comments - Answer Overflow

spscream

•Created by spscream on 3/15/2024 in #membrane-help

bundlex nifs and libasan

e.g. for ex_dtls it found some leak, maybe false positive:

    #0 0x7f319eb1fa57 in __interceptor_calloc ../../../../src/libsanitizer/asan/asan_malloc_linux.cpp:154
    #1 0x7f3190536ccf in export_keying_material /home/amalaev/work/sip_trunk/deps/ex_dtls/c_src/ex_dtls/dtls.c:96

-----------------------------------------------------
Suppressions used:
  count      bytes template
     70     945280 sys_thread_init_signal_stack
-----------------------------------------------------

SUMMARY: AddressSanitizer: 10002 byte(s) leaked in 171 allocation(s).

    #0 0x7f319eb1fa57 in __interceptor_calloc ../../../../src/libsanitizer/asan/asan_malloc_linux.cpp:154
    #1 0x7f3190536ccf in export_keying_material /home/amalaev/work/sip_trunk/deps/ex_dtls/c_src/ex_dtls/dtls.c:96

-----------------------------------------------------
Suppressions used:
  count      bytes template
     70     945280 sys_thread_init_signal_stack
-----------------------------------------------------

SUMMARY: AddressSanitizer: 10002 byte(s) leaked in 171 allocation(s).

but for my plugin I got no leaks found even if I put it intentionally

2 replies

SMSoftware Mansion

•Created by spscream on 3/11/2024 in #membrane-help

unifex seg fault on handle_destroy_state

final fix:

UNIFEX_TERM create(UnifexEnv *env, int bit_rate, int sample_rate) {
  UNIFEX_TERM res;

  State *state = unifex_alloc_state(env);

  state->decode_state = g722_1_decode_init(NULL, bit_rate, sample_rate);

  if (state->decode_state == NULL) {
    res = create_result_error(env, "decode_init");
    goto exit_create;
  }

  if (state->decode_state == NULL) {
    res = create_result_error(env, "decode_init");
    goto exit_create;
  }

  res = create_result_ok(env, state);
exit_create:
  unifex_release_state(env, state);
  return res;
}

UNIFEX_TERM create(UnifexEnv *env, int bit_rate, int sample_rate) {
  UNIFEX_TERM res;

  State *state = unifex_alloc_state(env);

  state->decode_state = g722_1_decode_init(NULL, bit_rate, sample_rate);

  if (state->decode_state == NULL) {
    res = create_result_error(env, "decode_init");
    goto exit_create;
  }

  if (state->decode_state == NULL) {
    res = create_result_error(env, "decode_init");
    goto exit_create;
  }

  res = create_result_ok(env, state);
exit_create:
  unifex_release_state(env, state);
  return res;
}

If I put NULL as initial parameter - g7221_1_decode_init initializes memory itself and it is correctly freed on destroy

8 replies

SMSoftware Mansion

•Created by spscream on 3/11/2024 in #membrane-help

unifex seg fault on handle_destroy_state

but may be I fixed it incorrectly, thanks for directions btw

8 replies

SMSoftware Mansion

•Created by spscream on 3/11/2024 in #membrane-help

unifex seg fault on handle_destroy_state

My bad. I made mistake on state initialization, made changes to create:

UNIFEX_TERM create(UnifexEnv *env, int bit_rate, int sample_rate) {
  UNIFEX_TERM res;

  State *state = unifex_alloc_state(env);
  g722_1_decode_state_t decoder_state;
  state->decode_state = g722_1_decode_init(&decoder_state, bit_rate, sample_rate);

  if (state->decode_state == NULL) {
    res = create_result_error(env, "decode_init");
    goto exit_create;
  }

  res = create_result_ok(env, state);
exit_create:
  unifex_release_state(env, state);
  return res;
}

UNIFEX_TERM create(UnifexEnv *env, int bit_rate, int sample_rate) {
  UNIFEX_TERM res;

  State *state = unifex_alloc_state(env);
  g722_1_decode_state_t decoder_state;
  state->decode_state = g722_1_decode_init(&decoder_state, bit_rate, sample_rate);

  if (state->decode_state == NULL) {
    res = create_result_error(env, "decode_init");
    goto exit_create;
  }

  res = create_result_ok(env, state);
exit_create:
  unifex_release_state(env, state);
  return res;
}

and now is working correctly. I'm new to C programming, reread about pointers and it seems clear for me now.

8 replies

SMSoftware Mansion

•Created by spscream on 3/11/2024 in #membrane-help

unifex seg fault on handle_destroy_state

I hope this is right place to post unifex questions, since Membrane framework is the owner of unifex.

8 replies

SMSoftware Mansion

•Created by spscream on 3/5/2024 in #membrane-help

h264 encoder problems

I also should mention, what i'm using swscale directly on decoder output in c code. I have a special case with switch stream on janus subscription - frame size could be different for published videos, so I had to add it in decoder. I tried to use swscale plugin in pipeline with no success and faster for my case was to implement it in decoder. In future I'll move this logic to separate membrane element, but I have deadlines now and it works for me now - decoder gives me constant frame format for outputs.

15 replies

SMSoftware Mansion

•Created by spscream on 3/5/2024 in #membrane-help

h264 encoder problems

thank you for insights, I moved repo in public

15 replies

SMSoftware Mansion

•Created by spscream on 3/7/2024 in #membrane-help

toilet capacity of outbound_rtx_controller

it might be so. Should I handle toilet overflow some way on my side, e.g. dropping input buffers? I don't want to have very large value for toilet_capacity

9 replies

SMSoftware Mansion

•Created by spscream on 3/7/2024 in #membrane-help

toilet capacity of outbound_rtx_controller

https://gist.github.com/spscream/a978129cf3d99bc2d1d5cb8b1eefe944 this is code for my UDP.Endpoint

9 replies

SMSoftware Mansion

•Created by spscream on 3/7/2024 in #membrane-help

toilet capacity of outbound_rtx_controller

I set toilet_capacity to 10000 and stream is flowing now. How can I debug the current buffer state of :sip_video? add debug to handle_buffer?

9 replies

SMSoftware Mansion

•Created by spscream on 3/7/2024 in #membrane-help

toilet capacity of outbound_rtx_controller

Ok, you give me some directions for debug, thank you btw I'm getting the same error with 1000 now:

21:01:43.622 pid=<0.1975.0> [error] <0.1286.0>/:sip_rtp/{:outbound_rtx_controller, 4064417025} Toilet overflow.

Atomic demand reached the size of -1001, which means that there are 1001
buffers sent without demanding it, which is above toilet capacity (1000)
when storing data from output working in push mode. It means that some element in the pipeline
processes the stream too slow or doesn't process it at all.
To have control over amount of buffers being produced, consider using output in :auto or :manual
flow control mode. (see `Membrane.Pad.flow_control`).
You can also try changing the `toilet_capacity` in `Membrane.ChildrenSpec.via_in/3`.


21:01:43.626 pid=<0.1286.0> [error] <0.1286.0>/ Terminating with reason: {:membrane_child_crash, :sip_video, :killed}

21:01:43.622 pid=<0.1975.0> [error] <0.1286.0>/:sip_rtp/{:outbound_rtx_controller, 4064417025} Toilet overflow.

Atomic demand reached the size of -1001, which means that there are 1001
buffers sent without demanding it, which is above toilet capacity (1000)
when storing data from output working in push mode. It means that some element in the pipeline
processes the stream too slow or doesn't process it at all.
To have control over amount of buffers being produced, consider using output in :auto or :manual
flow control mode. (see `Membrane.Pad.flow_control`).
You can also try changing the `toilet_capacity` in `Membrane.ChildrenSpec.via_in/3`.


21:01:43.626 pid=<0.1286.0> [error] <0.1286.0>/ Terminating with reason: {:membrane_child_crash, :sip_video, :killed}

9 replies

SMSoftware Mansion

•Created by spscream on 3/7/2024 in #membrane-help

toilet capacity of outbound_rtx_controller

should I to link somethink to SessionBin to handle rtx, if yes, what exactly?

9 replies

SMSoftware Mansion

•Created by spscream on 3/5/2024 in #membrane-help

h264 encoder problems

problem is solved, thanks 😄

15 replies

SMSoftware Mansion

•Created by spscream on 3/5/2024 in #membrane-help

h264 encoder problems

I set zerolatency tune for h264 encoder and framerate to {0,1} for vp8 decoder and now it looks good

15 replies

SMSoftware Mansion

•Created by spscream on 3/5/2024 in #membrane-help

h264 encoder problems

may be different frame rates in incoming videos could be the reason...

15 replies

SMSoftware Mansion

•Created by spscream on 3/5/2024 in #membrane-help

h264 encoder problems

00:18:21.171 pid=<0.2013.0> [info] <0.1285.0>/:video_vr_subscriber/:rtp/{:stream_receive_bin, 2635607903}/:depayloader/:depayloader timestamp: 2974099 diff: 1350
00:18:21.192 pid=<0.2013.0> [info] <0.1285.0>/:video_vr_subscriber/:rtp/{:stream_receive_bin, 2635607903}/:depayloader/:depayloader timestamp: 2975449 diff: 1350
00:18:21.215 pid=<0.2013.0> [info] <0.1285.0>/:video_vr_subscriber/:rtp/{:stream_receive_bin, 2635607903}/:depayloader/:depayloader timestamp: 2976889 diff: 1440
00:18:21.227 pid=<0.1401.0> [warning] calling to switch subscriber stream
...
00:18:21.228 pid=<0.1401.0> [info] JanusSubscriberSession janus_ws: %{"janus" => "event", "plugindata" => ...
00:18:21.239 pid=<0.2013.0> [info] <0.1285.0>/:video_vr_subscriber/:rtp/{:stream_receive_bin, 2635607903}/:depayloader/:depayloader timestamp: 2979057 diff: 2168
00:18:21.240 pid=<0.2013.0> [info] <0.1285.0>/:video_vr_subscriber/:rtp/{:stream_receive_bin, 2635607903}/:depayloader/:depayloader timestamp: 2979057 diff: 0
00:18:21.240 pid=<0.2013.0> [info] <0.1285.0>/:video_vr_subscriber/:rtp/{:stream_receive_bin, 2635607903}/:depayloader/:depayloader timestamp: 2979057 diff: 0
00:18:21.240 pid=<0.2013.0> [info] <0.1285.0>/:video_vr_subscriber/:rtp/{:stream_receive_bin, 2635607903}/:depayloader/:depayloader timestamp: 2979057 diff: 0
00:18:21.240 pid=<0.2013.0> [info] <0.1285.0>/:video_vr_subscriber/:rtp/{:stream_receive_bin, 2635607903}/:depayloader/:depayloader timestamp: 2979057 diff: 0
00:18:21.240 pid=<0.2013.0> [info] <0.1285.0>/:video_vr_subscriber/:rtp/{:stream_receive_bin, 2635607903}/:depayloader/:depayloader timestamp: 2979057 diff: 0
00:18:21.240 pid=<0.2013.0> [info] <0.1285.0>/:video_vr_subscriber/:rtp/{:stream_receive_bin, 2635607903}/:depayloader/:depayloader timestamp: 2979057 diff: 0
00:18:21.283 pid=<0.2013.0> [info] <0.1285.0>/:video_vr_subscriber/:rtp/{:stream_receive_bin, 2635607903}/:depayloader/:depayloader timestamp: 2981937 diff: 2880

00:18:21.171 pid=<0.2013.0> [info] <0.1285.0>/:video_vr_subscriber/:rtp/{:stream_receive_bin, 2635607903}/:depayloader/:depayloader timestamp: 2974099 diff: 1350
00:18:21.192 pid=<0.2013.0> [info] <0.1285.0>/:video_vr_subscriber/:rtp/{:stream_receive_bin, 2635607903}/:depayloader/:depayloader timestamp: 2975449 diff: 1350
00:18:21.215 pid=<0.2013.0> [info] <0.1285.0>/:video_vr_subscriber/:rtp/{:stream_receive_bin, 2635607903}/:depayloader/:depayloader timestamp: 2976889 diff: 1440
00:18:21.227 pid=<0.1401.0> [warning] calling to switch subscriber stream
...
00:18:21.228 pid=<0.1401.0> [info] JanusSubscriberSession janus_ws: %{"janus" => "event", "plugindata" => ...
00:18:21.239 pid=<0.2013.0> [info] <0.1285.0>/:video_vr_subscriber/:rtp/{:stream_receive_bin, 2635607903}/:depayloader/:depayloader timestamp: 2979057 diff: 2168
00:18:21.240 pid=<0.2013.0> [info] <0.1285.0>/:video_vr_subscriber/:rtp/{:stream_receive_bin, 2635607903}/:depayloader/:depayloader timestamp: 2979057 diff: 0
00:18:21.240 pid=<0.2013.0> [info] <0.1285.0>/:video_vr_subscriber/:rtp/{:stream_receive_bin, 2635607903}/:depayloader/:depayloader timestamp: 2979057 diff: 0
00:18:21.240 pid=<0.2013.0> [info] <0.1285.0>/:video_vr_subscriber/:rtp/{:stream_receive_bin, 2635607903}/:depayloader/:depayloader timestamp: 2979057 diff: 0
00:18:21.240 pid=<0.2013.0> [info] <0.1285.0>/:video_vr_subscriber/:rtp/{:stream_receive_bin, 2635607903}/:depayloader/:depayloader timestamp: 2979057 diff: 0
00:18:21.240 pid=<0.2013.0> [info] <0.1285.0>/:video_vr_subscriber/:rtp/{:stream_receive_bin, 2635607903}/:depayloader/:depayloader timestamp: 2979057 diff: 0
00:18:21.240 pid=<0.2013.0> [info] <0.1285.0>/:video_vr_subscriber/:rtp/{:stream_receive_bin, 2635607903}/:depayloader/:depayloader timestamp: 2979057 diff: 0
00:18:21.283 pid=<0.2013.0> [info] <0.1285.0>/:video_vr_subscriber/:rtp/{:stream_receive_bin, 2635607903}/:depayloader/:depayloader timestamp: 2981937 diff: 2880

15 replies

SMSoftware Mansion

•Created by spscream on 3/5/2024 in #membrane-help

h264 encoder problems

also I see what for first video stream is coming with full vp8 frame per packet, after switch is coming splitted, timestamps on incoming rtp from subscriber stream:

15 replies

SMSoftware Mansion

•Created by spscream on 3/5/2024 in #membrane-help

h264 encoder problems

22:28:10.208 pid=<0.1953.0> [info] <0.1281.0>/:my_sink handle_buffer: pts: 97870544444 dts: , diff: 32000000 len: 3686400
22:28:10.257 pid=<0.1953.0> [info] <0.1281.0>/:my_sink handle_buffer: pts: 97902544444 dts: , diff: 32000000 len: 3686400
22:28:10.287 pid=<0.1953.0> [info] <0.1281.0>/:my_sink handle_buffer: pts: 97950544444 dts: , diff: 48000000 len: 3686400
22:28:10.304 pid=<0.1953.0> [info] <0.1281.0>/:my_sink handle_buffer: pts: 97982544444 dts: , diff: 32000000 len: 3686400
22:28:10.351 pid=<0.1953.0> [info] <0.1281.0>/:my_sink handle_buffer: pts: 97998544444 dts: , diff: 16000000 len: 3686400
22:28:10.375 pid=<0.1397.0> [warning] calling to switch subscriber stream
22:28:10.375 pid=<0.1397.0> [info] [%{"feed" => 1515747746186501, "mid" => "1"}]
22:28:10.375 pid=<0.1397.0> [info] [%{"feed" => 1515747746186501, "mid" => "1", "sub_mid" => "0"}]
22:28:10.376 pid=<0.1397.0> [info] JanusSubscriberSession janus_ws: %{"janus" => "ack", "session_id" => 416529640432331, "transaction" => "bgs1t3lt2No"}
22:28:10.376 pid=<0.1397.0> [info] JanusSubscriberSession janus_ws: %{"janus" => "event", "plugindata" => ...}
22:28:10.382 pid=<0.1953.0> [info] <0.1281.0>/:my_sink handle_buffer: pts: 98046544444 dts: , diff: 48000000 len: 3686400
22:28:10.397 pid=<0.1953.0> [info] <0.1281.0>/:my_sink handle_buffer: pts: 98075566666 dts: , diff: 29022222 len: 3686400
22:28:10.411 pid=<0.1953.0> [info] <0.1281.0>/:my_sink handle_buffer: pts: 98091566666 dts: , diff: 16000000 len: 3686400
22:28:10.636 pid=<0.1985.0> [info] <0.1281.0>/:video_vr_subscriber/:vp8_decoder generate_stream_format_change: 320 180
22:28:10.637 pid=<0.1953.0> [info] <0.1281.0>/:my_sink handle_buffer: pts: 98106566666 dts: , diff: 15000000 len: 3686400
22:28:10.637 pid=<0.1953.0> [info] <0.1281.0>/:my_sink handle_buffer: pts: 98123566666 dts: , diff: 17000000 len: 3686400

22:28:10.208 pid=<0.1953.0> [info] <0.1281.0>/:my_sink handle_buffer: pts: 97870544444 dts: , diff: 32000000 len: 3686400
22:28:10.257 pid=<0.1953.0> [info] <0.1281.0>/:my_sink handle_buffer: pts: 97902544444 dts: , diff: 32000000 len: 3686400
22:28:10.287 pid=<0.1953.0> [info] <0.1281.0>/:my_sink handle_buffer: pts: 97950544444 dts: , diff: 48000000 len: 3686400
22:28:10.304 pid=<0.1953.0> [info] <0.1281.0>/:my_sink handle_buffer: pts: 97982544444 dts: , diff: 32000000 len: 3686400
22:28:10.351 pid=<0.1953.0> [info] <0.1281.0>/:my_sink handle_buffer: pts: 97998544444 dts: , diff: 16000000 len: 3686400
22:28:10.375 pid=<0.1397.0> [warning] calling to switch subscriber stream
22:28:10.375 pid=<0.1397.0> [info] [%{"feed" => 1515747746186501, "mid" => "1"}]
22:28:10.375 pid=<0.1397.0> [info] [%{"feed" => 1515747746186501, "mid" => "1", "sub_mid" => "0"}]
22:28:10.376 pid=<0.1397.0> [info] JanusSubscriberSession janus_ws: %{"janus" => "ack", "session_id" => 416529640432331, "transaction" => "bgs1t3lt2No"}
22:28:10.376 pid=<0.1397.0> [info] JanusSubscriberSession janus_ws: %{"janus" => "event", "plugindata" => ...}
22:28:10.382 pid=<0.1953.0> [info] <0.1281.0>/:my_sink handle_buffer: pts: 98046544444 dts: , diff: 48000000 len: 3686400
22:28:10.397 pid=<0.1953.0> [info] <0.1281.0>/:my_sink handle_buffer: pts: 98075566666 dts: , diff: 29022222 len: 3686400
22:28:10.411 pid=<0.1953.0> [info] <0.1281.0>/:my_sink handle_buffer: pts: 98091566666 dts: , diff: 16000000 len: 3686400
22:28:10.636 pid=<0.1985.0> [info] <0.1281.0>/:video_vr_subscriber/:vp8_decoder generate_stream_format_change: 320 180
22:28:10.637 pid=<0.1953.0> [info] <0.1281.0>/:my_sink handle_buffer: pts: 98106566666 dts: , diff: 15000000 len: 3686400
22:28:10.637 pid=<0.1953.0> [info] <0.1281.0>/:my_sink handle_buffer: pts: 98123566666 dts: , diff: 17000000 len: 3686400

15 replies

SMSoftware Mansion

•Created by spscream on 3/5/2024 in #membrane-help

h264 encoder problems

Is it reason could be in non-monotonically growing pts in incoming buffers? I have subscriber connection to janus, which is calling "switch" to janus on on "talking" events from janus, to switch video to new speaker if it is talking right now. I added sink to debug buffers and it is showing, what pts isn't monotonically increased, in log diff is the prev buffer pts - current pts

15 replies

SMSoftware Mansion

•Created by spscream on 3/5/2024 in #membrane-help

h264 encoder problems

full logic of my pipeline is the following: webrtc subscription for video from janus gateway -> vp8 decode -> h264 encode -> rtp output to sip client I implemented vp8 encoder/decoder plugin (https://github.com/spscream/membrane_vp8_ffmpeg_plugin it is in WIP state now) for membrane to decode vp8, using membrane_h264_ffmpeg_plugin as an example, it makes decoding and encoding of vp8 for my case.

15 replies

Gaming

Programming