R
RunPod3mo ago
papanton

Poor upload speed

I am uploading a large files (2GB) from a serverless endpoint in CA-MTL-1. I get consistenly low speeds and timeouts. The GPU is attached to a network volume if that matters. I copy the file from the network volume to the tmp directory before uploading. I measured by upload speed and its about 30MB/s which is low considering the server advertises 1000mbps+ upload speeds Please help
9 Replies
papanton
papanton3mo ago
its actually even worse in subsequent invocations: Upload rate: 4.85 MB/s", "level": "INFO"} { "delayTime": 4870, "id": "2bd90efe-b300-41d9-badd-c9c5ba04d960-u1", "status": "IN_PROGRESS" } workerId: bkunbkb6ko3tu8
nerdylive
nerdylive3mo ago
Upload speeds aren't just from runpod, I'm sure they are around hundreds of mb/s if not wrong It depends also from the receiver's network Where are you uploading this to?
papanton
papanton3mo ago
uploading to google file storage
nerdylive
nerdylive3mo ago
Google Cloud?
papanton
papanton3mo ago
correct - with the python library
nerdylive
nerdylive3mo ago
Ohh What region is it in Ur cloud file storage
papanton
papanton3mo ago
I am US multi region, but that wouldn't explain the slow speeds would it? It should perhaps have a bit more latency, but upload sped should not be affected
nerdylive
nerdylive3mo ago
Hmm yeah im kind of clueless about this maybe try another region first imo
haris
haris3mo ago
@papanton would you be able to DM me the endpoint ID so I can investigate this?