ToonyGen
RRunPod
•Created by ToonyGen on 10/23/2024 in #⚡｜serverless
Multi-Region Support and Expansion Plans
Hello,
Currently, the serverless worker system distributes containers randomly between the US and EU. I’m wondering if there are any plans to allow assigning a specific number of workers to each region (e.g., x workers in the US and x workers in the EU) under a single endpoint in the future.
Additionally, would it be possible to implement automatic routing of requests to the nearest region if this feature becomes available? For instance, if an edge function is called from the EU, it would be ideal to route the request to an EU-deployed worker to reduce latency.
At the moment, to handle regional routing, we need to deploy on separate pods—one for the EU and one for the US—and have our backend determine which endpoint to use.
Lastly, are there any plans to expand into Asia? Latency for Australia and New Zealand is currently isn't optimal at all.
Thank you!
5 replies
RRunPod
•Created by ToonyGen on 10/23/2024 in #⚡｜serverless
How to Minimize I/O Waiting Time?
Hello,
I’m using serverless Runpod for ComfyUI, where I send and return image URLs, leveraging the Google Cloud Bucket SDK. My current flow is:
Runpod handler downloads the image using the URL.
The image is converted to base64 and sent to ComfyUI for processing.
ComfyUI processes the image and returns it as base64.
The Runpod handler uploads the processed image to Google Cloud and returns the URL.
This process involves two I/O operations: downloading and uploading. My serverless pod is idle while waiting for these I/O trips. Could you provide any advice on how to optimize this flow to reduce the I/O waiting time and allow the GPU to start processing faster?
Thank you for your help!
5 replies