RunPod•4mo ago

I want to increase/decrease workers by code or script, can you help? (GraphQL)

I have a serverless setup already. Generally we keep 1 active worker in the actual time when we expect the traffic throughout the day, and at night when no one is using the application we make active workers 0 to avoid any charges. And then the next day, we make active workers 1 manually from runpod dashboard. We are willing to do that automatically. I know there is a GraphQL but I am not able to find relevant code to do that. Can anyone please help? Thanks!!

9 Replies

Mark•4mo ago

I wanted to ask for the exact same thing!

jackson holeOP•4mo ago

Let's hope for the best!

Eren•4mo ago

it's actually possible by Graph QL API, you can find a sample Endpoint mutation call below

curl --location --globoff 'https://api.runpod.io/graphql?api_key={{RUNPOD_API_KEY}}' \
--header 'content-type: application/json' \
--data '{"query":"mutation saveEndpoint($input: EndpointInput!) {\n  saveEndpoint(input: $input) {\n    gpuIds\n    id\n    idleTimeout\n    locations\n    name\n    networkVolumeId\n    scalerType\n    scalerValue\n    templateId\n    userId\n    workersMax\n    workersMin\n    gpuCount\n    __typename\n  }\n}","variables":{"input":{"gpuIds":"ADA_24,AMPERE_24,-NVIDIA L4,-NVIDIA RTX A5000","gpuCount":1,"allowedCudaVersions":"","id":"{{ID}}","idleTimeout":1,"locations":null,"name":"faster_whisper -fb","networkVolumeId":null,"scalerType":"QUEUE_DELAY","scalerValue":4,"workersMax":3,"workersMin":0,"executionTimeoutMs":180000}}}'

curl --location --globoff 'https://api.runpod.io/graphql?api_key={{RUNPOD_API_KEY}}' \
--header 'content-type: application/json' \
--data '{"query":"mutation saveEndpoint($input: EndpointInput!) {\n  saveEndpoint(input: $input) {\n    gpuIds\n    id\n    idleTimeout\n    locations\n    name\n    networkVolumeId\n    scalerType\n    scalerValue\n    templateId\n    userId\n    workersMax\n    workersMin\n    gpuCount\n    __typename\n  }\n}","variables":{"input":{"gpuIds":"ADA_24,AMPERE_24,-NVIDIA L4,-NVIDIA RTX A5000","gpuCount":1,"allowedCudaVersions":"","id":"{{ID}}","idleTimeout":1,"locations":null,"name":"faster_whisper -fb","networkVolumeId":null,"scalerType":"QUEUE_DELAY","scalerValue":4,"workersMax":3,"workersMin":0,"executionTimeoutMs":180000}}}'

GraphQL API Docs: https://graphql-spec.runpod.io/ Without API Docs: you can go to your dashboard and open the Developer Tools by F12 go to Network tab then make an update on your endpoint and find the mutation API call then you can just copy that as CURL and work on it on postman remember to remove all Headers and add api_key query param

jackson holeOP•4mo ago

The network hack is a Gem 💎 Will surely try this!! Thanks!!

Eren•4mo ago

🚀 PS: To set Active Worker you need to set the workersMin variable in json

Jason•4mo ago

In the reference there is no such mutation for endpoints? Oh there is

jackson holeOP•2mo ago

Hello! Finally with the help of @Eren and of course you @nerdylive I was able to create a script so anyone can use it in place.

import requests
import json
import argparse
import logging
from datetime import datetime

# Logging setup
logging.basicConfig(
    filename="runpod_scheduler.log",
    level=logging.INFO,
    format="%(asctime)s - %(levelname)s - %(message)s"
)

API_KEY = "your_api_key_here"
URL = f"https://api.runpod.io/graphql?api_key={API_KEY}"


# Parse command-line arguments
parser = argparse.ArgumentParser(description="Update RunPod workersMin value")
parser.add_argument("--minWorkers", type=int, required=True, help="Value for workersMin")
parser.add_argument("--id", type=str, required=True, help="Endpoint ID")
parser.add_argument("--name", type=str, required=True, help="Endpoint Name")
args = parser.parse_args()


# GraphQL mutation for saving endpoint
mutation = """
mutation saveEndpoint($input: EndpointInput!) {
  saveEndpoint(input: $input) {
    gpuIds
    id
    name
    workersMin
  }
}
"""

# Payload data
variables = {
    "input": {
        "gpuIds": "AMPERE_24,-NVIDIA L4,-NVIDIA RTX A5000",
        "name": f"{args.name} -fb",
        "id": args.id,
        "workersMin": args.minWorkers,
    }
}



# Construct the request
payload = {
    "operationName": "saveEndpoint",
    "query": mutation,
    "variables": variables
}

try:
    logging.info(f"Mutating for the endpoint `{args.name=}` and `{args.id=}` ({args.minWorkers=})")
    # Send the request
    response = requests.post(URL, headers={"Content-Type": "application/json"}, data=json.dumps(payload))
        
    if response.status_code == 200:
        logging.info(f"Successfully updated workersMin to {args.minWorkers}\nResponse:\n{response.json()}")
    else:
        logging.error(f"Failed to update workersMin. Status: {response.status_code}, Response: {response.json()}")


except Exception as e:
    logging.error(f"Error: {str(e)}")

import requests
import json
import argparse
import logging
from datetime import datetime

# Logging setup
logging.basicConfig(
    filename="runpod_scheduler.log",
    level=logging.INFO,
    format="%(asctime)s - %(levelname)s - %(message)s"
)

API_KEY = "your_api_key_here"
URL = f"https://api.runpod.io/graphql?api_key={API_KEY}"


# Parse command-line arguments
parser = argparse.ArgumentParser(description="Update RunPod workersMin value")
parser.add_argument("--minWorkers", type=int, required=True, help="Value for workersMin")
parser.add_argument("--id", type=str, required=True, help="Endpoint ID")
parser.add_argument("--name", type=str, required=True, help="Endpoint Name")
args = parser.parse_args()


# GraphQL mutation for saving endpoint
mutation = """
mutation saveEndpoint($input: EndpointInput!) {
  saveEndpoint(input: $input) {
    gpuIds
    id
    name
    workersMin
  }
}
"""

# Payload data
variables = {
    "input": {
        "gpuIds": "AMPERE_24,-NVIDIA L4,-NVIDIA RTX A5000",
        "name": f"{args.name} -fb",
        "id": args.id,
        "workersMin": args.minWorkers,
    }
}



# Construct the request
payload = {
    "operationName": "saveEndpoint",
    "query": mutation,
    "variables": variables
}

try:
    logging.info(f"Mutating for the endpoint `{args.name=}` and `{args.id=}` ({args.minWorkers=})")
    # Send the request
    response = requests.post(URL, headers={"Content-Type": "application/json"}, data=json.dumps(payload))
        
    if response.status_code == 200:
        logging.info(f"Successfully updated workersMin to {args.minWorkers}\nResponse:\n{response.json()}")
    else:
        logging.error(f"Failed to update workersMin. Status: {response.status_code}, Response: {response.json()}")


except Exception as e:
    logging.error(f"Error: {str(e)}")

Jason•2mo ago

There's a rest api too now, if you wanna remake in another version 😄 😄 I think it might be easier Thanks for sharing too!

Eren•2mo ago

Thanks for sharing!

Gaming

Programming

I want to increase/decrease workers by code or script, can you help? (GraphQL)

Did you find this page helpful?