enfys
Extremely slow network storage
For a couple of weeks now, it's taken >15 minutes (sometimes up to 20) to load a 70B model from network storage in CA-MTL. This is, of course, paid time on the GPU while waiting for it (as well as being quite inconvenient) - it's actually quicker to download the model fresh from an internet server every time rather than waiting for it to load from network storage. Is the "network storage" actually on the same network as the GPU server, or is it on some cloud somewhere? Why is it so slow?
31 replies
A40 availability
There are a couple of >1 month old posts about this but it seems to be an issue again, A40s have become pretty much entirely unavailable other than at weird times (~7am GMT) and it's been like this for about a week now, what's going on? Availability seems unusually poor, I've never known it like this, I've got quite a lot of credit that I can't use.
43 replies