R
RunPod4mo ago
sdev

Flashboot principles

Hello. Who can explain Flashboot principles? When worker is idle model stays in gpu memory or pc memory? How long the model stays in memory? Is some LRU eviction policy used?
1 Reply
nerdylive
nerdylive4mo ago
Gpu memory, no policies are written but it should kind of "cache" or keep warm your models For a period of time ( pretty long ) And I think it's dynamic based on few variables I suppose

Did you find this page helpful?