Background loop with sound effect playback on event
Hello,
I'm using Membrane for background music playing locally (portaudio), and I want to be able to play sound effects at will (as in, not related to any timer, just events like pressing a key, for example). Would it make sense to have multiple pipelines, one for each sound effect? I think it would make more sense to have my background loop pipeline and a separate sound effect pipeline, which can play any of the various sound effects, but I'm not sure how that would work. Any help would be great, thanks!
7 Replies
I guess I'm also confused on how to keep the background loop playing when the sound effects play. I suppose multiple sound effects could play overlapped as well.
I'd still like some help with this if anyone has any suggestions, but what I'm doing for now is I have my looping background as it's own pipeline, and another sound effect pipeline that takes a path. in places where I want to play a sound effect, I just call
Membrane.Pipeline.start/3. the background pipeline is in my application's supervision tree but the sound effect pipeline isn't.
if anyone wants to see the code, https://github.com/harrisi/elixir_breakout/pull/1/files is just membrane stuff, shouldn't need to look at the rest of the project to understand.One thing that made it a lot easier for me was to not use MP3s as input, but RAW audio instead. When you seek in the MP3 file you risk incomplete data being emitted to your MP3 decoder.
To "merge" multiple audios you need the
Membrane.AudioMixerThat's true, I do have some various warnings about that. Good tip!
Is this actually needed? If I have two different pipelines to play the audio, it seems to play things correctly. It does seem like mixing the audio could improve things, but I'm not sure how to do that with the background loop. Maybe dynamic pads or something?
In our pipeline we have implemented some similar mechanisms. At a high level our elements work as follows:
We wrote a "LoopingBufferQueue" element. it is placed after a source and will read the entire input until it sees end_of_stream. Every time it sees a buffer before end_of_stream it stores in an in-memory ring buffer. The idea then is when the LoopingBufferQueue gets demand it just serves it from its ring buffer forever instead of propogating the end_of_stream from the source.
Then we feed this LoopingBufferQueue input into a mixer (`custom element) which has a "primary" static pad and dynamic pads. Then you can wire it up such that the mixer will produce buffers from the primary input until an extra pad is added to the mixer. When the pad is added to the mixer it undergoes a state change and starts producing buffers from the extra input pad.
The tricky thing to keep in mind are that generally you should produce buffers with a monotonicly increasing presentation time stamp. You must take that into account in several places with this architecture for example every time the LoopingBufferQueue loops the timestamp resets to 0. Similarly, when an extra pad is attached its timestamp will reset to 0.
ohh, so you have something like this?:
if I understand correctly, the
Mixer would have dynamic pads which would mix (somehow) the looping buffer when linked. I haven't used dynamic pads (I just started with membrane yesterday), but that flow does make sense to me. there's some hand waving, but I think I kinda get it.You could decompose it by using the existing source elements like this:
but yes that is the general idea. Sorry I cant share the code its from an app at work 🙂 I think the membrane guides have an introduction to creating elements (and iirc dynamic pads are covered).
that makes sense. thanks a lot for the suggestion!