wmute
RRunPod
•Created by wmute on 1/13/2024 in #⚡｜serverless
Creating serverless templates via GraphQL
I want to automatically create serverless templates from command line/python. I tried the example for creating a new pod here: https://docs.runpod.io/docs/create-pod-template which worked.
I want to create a serverless template though, but when I add
isServerless: true to the graphql request (as per the PodTemplate documentation) it errors out with:
Where is the saveTemplate mutation described in the documentation? I can't find it in https://graphql-spec.runpod.io/#group-Operations-Mutations2 replies
RRunPod
•Created by wmute on 1/7/2024 in #⚡｜serverless
Extremely slow Delay Time
We are using 2 serverless endpoints on runpod and the "Delay Time" (which I assume measures end to end time) varies drastically between the endpoints. They both use the same hardware (the A5000 option) and one of them has sub-second delay times and the other ~50 seconds up to 180s.
On the slow endpoint, the worst cold start time is reported as 13s, and the execution time is ~2s, which don't add up to the delay time. There are ~50 seconds unnacounted for.
The other endpoint using the same hardware does not observe such drastic delay time.
7 replies
RRunPod
•Created by wmute on 1/7/2024 in #⚡｜serverless
Custom template: update environment variables?
I have configured environment variables in my custom endpoint template. When I edit the template to change their contents, the workers still seem to be using the old ENV values.
What ultimately works is removing and recreating the whole endpoint, but I don't want to do that repeatedly. I've tried triggering a refresh using the "Create new release" functionality but it didn't seem to help.
What is the recommended way of making sure that the workers are using the latest environment variables from the template?
4 replies