RunPod•6mo ago

Slow I/O

Hey, I am trying to download a 7GB file and run a ffmpeg process to extract an audio from that file (its a video). Locally it takes on average around 5 minutes, but when I try it on the cloud (I chose the CPU, general purpose since a GPU doesn't seem to give any advantage here) and it looks like the I/O is SUPER SLOW. Is there anything I can do to speed up the Disk I/O?

4 Replies

nerdylive•6mo ago

Don't use network volumes

Itay ElgazarOP•6mo ago

What do you mean? I didn't configure any network volume - I simply use an URL (of an acceleerated S3 bucket) to extract the audio from that video It looks like that:

def video_to_audio_url(video_url, output):
    # Use ffmpeg to extract audio from the video
    if os.path.exists(output):
        os.remove(output)
    process = (
        ffmpeg
        .input(video_url)
        .output(output, format='mp3', acodec='libmp3lame')
        .overwrite_output()
        .run_async(pipe_stdin=True, pipe_stdout=True, pipe_stderr=True)
    )

    stdout, stderr = process.communicate()

    if os.path.exists(output):
        print("Audio extracted successfully")
        return True
    else:
        print(f"Failed to extract audio: {stderr.decode('utf-8')}")
        return False

def video_to_audio_url(video_url, output):
    # Use ffmpeg to extract audio from the video
    if os.path.exists(output):
        os.remove(output)
    process = (
        ffmpeg
        .input(video_url)
        .output(output, format='mp3', acodec='libmp3lame')
        .overwrite_output()
        .run_async(pipe_stdin=True, pipe_stdout=True, pipe_stderr=True)
    )

    stdout, stderr = process.communicate()

    if os.path.exists(output):
        print("Audio extracted successfully")
        return True
    else:
        print(f"Failed to extract audio: {stderr.decode('utf-8')}")
        return False

nerdylive•6mo ago

Hmm I'm not sure about that then, but if you think it's an hardware issue, make a support ticket on the website then

Encyrption•6mo ago

Have you tried in a region outside of EU-RO-1? It's the only region with CPU but from personal experience it is broken with a capital B. My app was failing to download the checkpoints files, timing out after 10 minutes. I moved my app into CA-MTL-1 and the checkpoints downloaded in under a minute without any issues. You'll have to put your work on a GPU instance but I would bet money on that fixing your problem. If you do find that fixes your problem please open a ticket. Maybe if they get enough people complaining / moving out of EU-RO-1 they will actually fix it.

Gaming

Programming

Slow I/O