Serverless or Regular Pod? (How good is Flashboot?)
Hello, I’m a new user of RunPod, and I’m using it for image generation AI.
I’m planning to create an API based on the ComfyUI workflow I’ve developed, so people can enter prompts on my website and receive the generated images.
However, I’m not sure whether I should use Serverless or just keep a regular Pod running 24/7 and manually create the API there.
Given that there are not many users right now, Serverless seems more cost-effective.
But what worries me is the large size of AI models, which also makes Docker images quite large, and I’m concerned about the long time it might take to start an instance. If a user calls the API and has to wait several minutes each time, that would be the worst-case scenario.
In the case of Serverless, it seems like a new instance is started every time an API is called after idle time, which worries me.
However, I’ve also read that there's a feature called FlashBoot, which claims to boot almost instantly—in about 1 second. That sounds too good to be true, so I’m wondering if it’s reliable. If it’s true, it seems like most people should be using Serverless, but it seems like many are still using regular Pods.
Given my situation, which option should I choose?
1 Reply
Just use serverless I guess, serverless images are pre downloaded, so you'll dont need to worry about the long download times, once it's ready it'll be ready to run ( downloaded)
Yes flashboot is reliable if you do load the models right, (before the serverless.Start () call in the handler.py/handler file )