falk
RRunPod
•Created by falk on 1/11/2025 in #⚡｜serverless
Flashboot meaning?
Is there any documentation on what it does under the hood?
i am asking because of this:
"FlashBoot reduces majority cold-starts down to 2s, even for LLMs. Make sure to test output quality before enabling."
does that mean it reduces output quality of my models?
3 replies
RRunPod
•Created by falk on 6/27/2024 in #⚡｜serverless
Prevent Extra Workers from appearing
Many times extra workers are spawned for multiple hours even though there is no need for them as the load is easily kept up by the normal workers. How can i prevent these from appearing?
i already set max workers but it does not help. this costs so much money that i am thinking about switching provider.
12 replies