RunPod•3mo ago

I want to increase/decrease workers by code or script, can you help? (GraphQL)

I have a serverless setup already. Generally we keep 1 active worker in the actual time when we expect the traffic throughout the day, and at night when no one is using the application we make active workers 0 to avoid any charges. And then the next day, we make active workers 1 manually from runpod dashboard. We are willing to do that automatically. I know there is a GraphQL but I am not able to find relevant code to do that. Can anyone please help? Thanks!!

9 Replies

Mark•3mo ago

I wanted to ask for the exact same thing!

jackson holeOP•3mo ago

Let's hope for the best!

Eren•3mo ago

it's actually possible by Graph QL API, you can find a sample Endpoint mutation call below

curl --location --globoff 'https://api.runpod.io/graphql?api_key={{RUNPOD_API_KEY}}' \
--header 'content-type: application/json' \
--data '{"query":"mutation saveEndpoint($input: EndpointInput!) {\n  saveEndpoint(input: $input) {\n    gpuIds\n    id\n    idleTimeout\n    locations\n    name\n    networkVolumeId\n    scalerType\n    scalerValue\n    templateId\n    userId\n    workersMax\n    workersMin\n    gpuCount\n    __typename\n  }\n}","variables":{"input":{"gpuIds":"ADA_24,AMPERE_24,-NVIDIA L4,-NVIDIA RTX A5000","gpuCount":1,"allowedCudaVersions":"","id":"{{ID}}","idleTimeout":1,"locations":null,"name":"faster_whisper -fb","networkVolumeId":null,"scalerType":"QUEUE_DELAY","scalerValue":4,"workersMax":3,"workersMin":0,"executionTimeoutMs":180000}}}'

curl --location --globoff 'https://api.runpod.io/graphql?api_key={{RUNPOD_API_KEY}}' \
--header 'content-type: application/json' \
--data '{"query":"mutation saveEndpoint($input: EndpointInput!) {\n  saveEndpoint(input: $input) {\n    gpuIds\n    id\n    idleTimeout\n    locations\n    name\n    networkVolumeId\n    scalerType\n    scalerValue\n    templateId\n    userId\n    workersMax\n    workersMin\n    gpuCount\n    __typename\n  }\n}","variables":{"input":{"gpuIds":"ADA_24,AMPERE_24,-NVIDIA L4,-NVIDIA RTX A5000","gpuCount":1,"allowedCudaVersions":"","id":"{{ID}}","idleTimeout":1,"locations":null,"name":"faster_whisper -fb","networkVolumeId":null,"scalerType":"QUEUE_DELAY","scalerValue":4,"workersMax":3,"workersMin":0,"executionTimeoutMs":180000}}}'

GraphQL API Docs: https://graphql-spec.runpod.io/ Without API Docs: you can go to your dashboard and open the Developer Tools by F12 go to Network tab then make an update on your endpoint and find the mutation API call then you can just copy that as CURL and work on it on postman remember to remove all Headers and add api_key query param

jackson holeOP•3mo ago

The network hack is a Gem 💎 Will surely try this!! Thanks!!

Eren•3mo ago

🚀 PS: To set Active Worker you need to set the workersMin variable in json

Jason•3mo ago

In the reference there is no such mutation for endpoints? Oh there is

jackson holeOP•4w ago

Hello! Finally with the help of @Eren and of course you @nerdylive I was able to create a script so anyone can use it in place.

import requests
import json
import argparse
import logging
from datetime import datetime

# Logging setup
logging.basicConfig(
    filename="runpod_scheduler.log",
    level=logging.INFO,
    format="%(asctime)s - %(levelname)s - %(message)s"
)

API_KEY = "your_api_key_here"
URL = f"https://api.runpod.io/graphql?api_key={API_KEY}"


# Parse command-line arguments
parser = argparse.ArgumentParser(description="Update RunPod workersMin value")
parser.add_argument("--minWorkers", type=int, required=True, help="Value for workersMin")
parser.add_argument("--id", type=str, required=True, help="Endpoint ID")
parser.add_argument("--name", type=str, required=True, help="Endpoint Name")
args = parser.parse_args()


# GraphQL mutation for saving endpoint
mutation = """
mutation saveEndpoint($input: EndpointInput!) {
  saveEndpoint(input: $input) {
    gpuIds
    id
    name
    workersMin
  }
}
"""

# Payload data
variables = {
    "input": {
        "gpuIds": "AMPERE_24,-NVIDIA L4,-NVIDIA RTX A5000",
        "name": f"{args.name} -fb",
        "id": args.id,
        "workersMin": args.minWorkers,
    }
}



# Construct the request
payload = {
    "operationName": "saveEndpoint",
    "query": mutation,
    "variables": variables
}

try:
    logging.info(f"Mutating for the endpoint `{args.name=}` and `{args.id=}` ({args.minWorkers=})")
    # Send the request
    response = requests.post(URL, headers={"Content-Type": "application/json"}, data=json.dumps(payload))
        
    if response.status_code == 200:
        logging.info(f"Successfully updated workersMin to {args.minWorkers}\nResponse:\n{response.json()}")
    else:
        logging.error(f"Failed to update workersMin. Status: {response.status_code}, Response: {response.json()}")


except Exception as e:
    logging.error(f"Error: {str(e)}")

import requests
import json
import argparse
import logging
from datetime import datetime

# Logging setup
logging.basicConfig(
    filename="runpod_scheduler.log",
    level=logging.INFO,
    format="%(asctime)s - %(levelname)s - %(message)s"
)

API_KEY = "your_api_key_here"
URL = f"https://api.runpod.io/graphql?api_key={API_KEY}"


# Parse command-line arguments
parser = argparse.ArgumentParser(description="Update RunPod workersMin value")
parser.add_argument("--minWorkers", type=int, required=True, help="Value for workersMin")
parser.add_argument("--id", type=str, required=True, help="Endpoint ID")
parser.add_argument("--name", type=str, required=True, help="Endpoint Name")
args = parser.parse_args()


# GraphQL mutation for saving endpoint
mutation = """
mutation saveEndpoint($input: EndpointInput!) {
  saveEndpoint(input: $input) {
    gpuIds
    id
    name
    workersMin
  }
}
"""

# Payload data
variables = {
    "input": {
        "gpuIds": "AMPERE_24,-NVIDIA L4,-NVIDIA RTX A5000",
        "name": f"{args.name} -fb",
        "id": args.id,
        "workersMin": args.minWorkers,
    }
}



# Construct the request
payload = {
    "operationName": "saveEndpoint",
    "query": mutation,
    "variables": variables
}

try:
    logging.info(f"Mutating for the endpoint `{args.name=}` and `{args.id=}` ({args.minWorkers=})")
    # Send the request
    response = requests.post(URL, headers={"Content-Type": "application/json"}, data=json.dumps(payload))
        
    if response.status_code == 200:
        logging.info(f"Successfully updated workersMin to {args.minWorkers}\nResponse:\n{response.json()}")
    else:
        logging.error(f"Failed to update workersMin. Status: {response.status_code}, Response: {response.json()}")


except Exception as e:
    logging.error(f"Error: {str(e)}")

Jason•4w ago

There's a rest api too now, if you wanna remake in another version 😄 😄 I think it might be easier Thanks for sharing too!

Eren•4w ago

Thanks for sharing!

Gaming

Programming

I want to increase/decrease workers by code or script, can you help? (GraphQL)

Did you find this page helpful?