How to Minimize I/O Waiting Time?
Hello,
I’m using serverless Runpod for ComfyUI, where I send and return image URLs, leveraging the Google Cloud Bucket SDK. My current flow is:
Runpod handler downloads the image using the URL.
The image is converted to base64 and sent to ComfyUI for processing.
ComfyUI processes the image and returns it as base64.
The Runpod handler uploads the processed image to Google Cloud and returns the URL.
This process involves two I/O operations: downloading and uploading. My serverless pod is idle while waiting for these I/O trips. Could you provide any advice on how to optimize this flow to reduce the I/O waiting time and allow the GPU to start processing faster?
Thank you for your help!
3 Replies
pass base64 as input of the job, return base64 as output of the job to avoid downloading and uploading within the worker
another way is to use network volumes + cpu endpoint + gpu endpoint, have cpu endpoint download and upload while gpu workers handle the actual work, use network volumes to share data between cpu and gpu endpoint
Thanks. I thought Async magic with handler could be one of the solution, but I wasn't sure really about. I will switch off to base64 instead of URL. I was losing about 2s on trips. Network storage is interesting. Do you have rough idea which one would yield faster result? My speculation is that since network storage is disk based, there will be read and write operation. And on top of that, there will another api communication between instances. Probably base64 would be faster...
yes base64 will be faster, latency with back end forth events with network storage with 2 endpoints will cause more latencies