garg-aayush
RRunPod
•Created by garg-aayush on 2/25/2025 in #⚡｜serverless
How to Speed Up S3 Upload or Make it Async in RunPod Serverless Deployments
I am currently exploring using RunPod as our primary in-house model deployment platform instead of Replicate (our current preferred platform). Our in-house models mostly are txt2img/img2img custom models.
One of the issues I'm facing while testing RunPod is long S3 upload times. For example, for one of our processes, the prediction time is ~1 second, but the S3 upload is taking up to 4-5 seconds (depending on image size), significantly increasing the overall prediction time.
This causes two main problems:
- Long prediction times despite the GPU being free after just 1 second of actual processing
- Increased queue times as workers remain occupied during these long uploads
Is there a way to speed up S3 uploads? Is there a way to make the S3 upload async so that the server can handle multiple concurrent requests? For comparison, Replicate provides temporary File URLs (persistent for ~30 mins) that avoid the S3 upload overhead, resulting in much faster overall request times.
Note: I am already using RunPod's specific S3 upload function that uses chunking.
3 replies