A40 availability
There are a couple of >1 month old posts about this but it seems to be an issue again, A40s have become pretty much entirely unavailable other than at weird times (~7am GMT) and it's been like this for about a week now, what's going on? Availability seems unusually poor, I've never known it like this, I've got quite a lot of credit that I can't use.
32 Replies
Yeah I’ve found the same!
7 am GMT is when I, personally, usually start On-Demand these cards, as it's worktime for us here. Our team finds these cards very cost-efficient. Depending on a scale of what you're doing, it might seem that we hoard them. But this stuff is FCFS, so if you really really really need these specific cards - you can use savings plans.
I used to start and stop them based on workload, I'm not running a 24/7 workload so savings plans don't really work for me, if I could I'd get my credit out of RunPod so I could put it into an alternative provider with slightly higher pricing but much better availability but I can't
It's not the specific cards, but everything vaguely suitable that has disappeared, only the smallest cards seem to be routinely available now
I assume that L40/L40s are not desirable? We sometimes have to resort to using those, though it's twice as expensive.
L40 is a good shout, but it's not routinely available in the two regions I use (EU-SE and CA-MTL), A40s have been a bit better supplied over the past 24 hours which is nice, and I've signed up with an alternative provider which offers a better price than RunPod (~$1.15/hr) on A100 80GB which also works for me, it's good to have a backup! I'm not overly price-sensitive but don't want to spend multiple $/hr on very high-end cards just for inference.
well just increase in demand i guess
But maybe runpod will add more supply if its constant on low supply
I wonder if they already have, as it seems to have improved a lot in the past few days especially in CA-MTL, either they've added more supply or a heavy user has stopped what they're doing
im not sure about that, but from what i know some other user may have like burst workflow, sometimes they hog the gpu's for a timespan then after done using them, the stock is back up
yep
I see we're back where we were with the A40s, I guess another week without them :/
Same for me, my serverless endpoints configured for A40 are stuck since yesterday
Try to select another gpu too
Makes it quite difficult to justify a top up when my current credit runs out tbh
And again, I'm paying (admittedly not much) to store stuff in two locations and can't use it
Sad, you should complain to runpod to increase the capacity 😁
They're getting rid of them I think, one I managed to deploy had a "this server is being removed" maintenance message 😦
Oh wow which location?
CA-MTL-1
A40?
Yeah
So your data?
Is fine?
My data is on network storage, so data is fine but it looks like they might be lowering the stock of A40s to make room for something else
Ohh yeah possibly
It was something like "this server is being removed to increase capacity" so sounds like they want to increase capacity of some other GPU
Ohhh yeah true
Great info
I guess it is h series lel
Maybe it's already happened in EU-SE-1 as A40s seem to have been low there for a while
If they can send me a couple of the A40s they take out that'd be great 😄
Hahah it'll be power ineffecient to use heavily used card
Maybe they're selling it if it is in a good condition try emailing if you want
I'm joking, I don't have 5k to blow 😄
Kek is that the new price or
that's about what they are on eBay!
Ic
This is not quite as bad as it was previously, but still going on, normally they all disappear evening European time (which is generally when I want to use it)
This is getting worse and worse again - there's a bit of availability in the morning European time, but by midday they're just gone, across all regions, every day, it seems to be when the US wakes up
Ooh unavailable now?