Model upload to huggingface is so slow it costs more than training
Model upload is always slow which makes runpod more expensive that it needs to be. But sometimes, it's super slow. Is there a specific datacenter which has a better internet connection, or any other way for me to avoid starting pods that have a very slow upload speed?
3 Replies
HuggingFace in particular is kind of weird - we can't really do much about their network speed and they like to throttle users by IP address. It should be better more times than other and if we can work out a way to improve this connection we can.
yeah and maybe your dc's location is faraway from hf's servers
from what i know their server is mostly in US
Maybe download the model locally and then push it to huggingface later
not an option if your model is hundreds of gigabytes