GPU type prioritization seems to have stopped working on 13th of March
I have an endpoint with 3 cheapest GPU types selected in the order of their price (i.e. 4090 is my 3rd option). I noticed that my expenses almost doubled after 13th of March, but the billed GPU time didn't change much. It turned out that before 13th of March the usage of 4090 was close to 0 (which is what I expect - it is my 3rd option). On 13th of March 4090 usage jumped to 50-70% skyrocketing my prices. This is unexpected and unfortunately leads to additional costs for me. For now I removed 4090 as my 3rd option, but this behavior seems abnormal, thus, reaching out to you here.
4 Replies
Now I think that GPU selection is just broken - I've removed my 3rd option (4090) completely hours ago and I still get jobs ran on 4090 as of right now and pay price of 4090. I suspect these might be old workers still cached on 4090, but it has been ~7h since I change the selection, so I expect these workers to be rescheduled or not used at all.
4090 still gets 25% of billing time
It doesn't update automatically, you have to scale the workers down to zero and back up again for the change in priority to take effect.
thank you, that explains it, unfortunately, i haven't seen this requirement anywhere
i think proper UI would be to disable gpu selection if there are > 0 workers and unlock it only when there is 0 workers
I believe RunPod are working on improving it.