Flashboot principles
Hello. Who can explain Flashboot principles?
When worker is idle model stays in gpu memory or pc memory?
How long the model stays in memory? Is some LRU eviction policy used?
1 Reply
Gpu memory, no policies are written but it should kind of "cache" or keep warm your models
For a period of time ( pretty long )
And I think it's dynamic based on few variables I suppose