Demuxing Safari MP4
Hi! I'm trying to use a MediaRecorder to record audio/mp4 on Safari, and then handle it using Membrane.
Membrane.MP4.Demuxer.ISOM gives me an error:
I suspect this is because the recording does not have fast start enabled, as the docs for this plugin say it must be.
Is there any Membrane plugin I can use to demux this media container? I don't think it's possible to record fast start mp4 using MediaRecorder on Safari.2 Replies
Hi there, the plugin supports non-fast-start, it even has an option to optimize for that scenario, see https://hexdocs.pm/membrane_mp4_plugin/Membrane.MP4.Demuxer.ISOM.html#module-element-options. However, your case is probably not about fast-start, but safari producing (probably) something called fragmented mp4, which is not supported.
However, I have (a bit hacky) idea for a cross-browser way to get media that may work 🙂 The idea is to start a WebRTC Peer Connection and use insertable streams to extract the audio and send it wherever you want. Then, you probably need another Peer Connection that would serve as the receiver part of the connection - it wouldn't actually receive anything, but is needed for the stream to start. Insertable streams are usually used for e2ee, see https://github.com/webrtc/samples/tree/gh-pages/src/content/insertable-streams/endtoend-encryption, but in fact they give you direct access to the stream.
That gives you a stream of Opus packets that are understandable by Membrane and most other tools, no matter what browser you're using 🙂
That said, it's a pretty hacky way and I'm not sure it will work. It would be nice to have a fragmented mp4 demuxer in Membrane, but can't tell when it happens
Ah interesting! That sounds like a lot 😅 I think I'm just gonna use an AudioWorkletProcessor to extract raw audio. Was hoping to be able to use an encoding to reduce bandwidth, but I think it should be fine.
Thanks for the context and ideas!