DannyB
RRunPod
•Created by DannyB on 4/26/2024 in #⚡｜serverless
A way to know if worker is persistent ("active") or not
It would be a great help if there was some way from inside the code (e.g. environment variable) to know if the model is running on a persistent worker or not.
Example use-case - If the worker is persistent, I can compile the model, it takes ~20 minutes but it is worth it so my users can get an almost 50% latency reduction. However, you can understand it is impossible to do for an ephemeral worker as it will take too long to initialize.
Is there any way to do this?
3 replies