Eren Comments - Answer Overflow

Eren

•Created by dalton57918 on 3/31/2025 in #⚡｜serverless

Buil docker with environment variables

You can write them inside your Dockerfile for a quick fix, I remember Runpod had this on their roadmap like couple months ago

3 replies

RRunPod

•Created by Eren on 3/27/2025 in #⚡｜serverless

Meaning of -u1 -u2 at the end of request id?

yeah I just wanted to get more insight about what u1 and u2

4 replies

RRunPod

•Created by wrichert on 3/31/2025 in #⚡｜serverless

Fastest cloud storage access from serverless?

Use Cloudflare R2, it is free for ingress&egress

2 replies

RRunPod

•Created by xnerhu on 3/26/2025 in #⚡｜serverless

Ambiguity of handling runsync cancel from python handler side

I believe the worker boot does not get affected by :cancel, it still boots up and checks the queue, sees no job to acquire and stays idle. Thats my thought

5 replies

RRunPod

•Created by jackson hole on 1/13/2025 in #⚡｜serverless

I want to increase/decrease workers by code or script, can you help? (GraphQL)

Thanks for sharing!

13 replies

RRunPod

•Created by koop7450 on 3/9/2025 in #⚡｜serverless

Use SDK to create Network Storage Volumes for Serverless Endpoints

Many endpoints missing in docs site

8 replies

RRunPod

•Created by jim on 1/15/2025 in #⚡｜serverless

Serverless H200?

I'd utilize the H100 for that price if I realized this bug earlier...

11 replies

RRunPod

•Created by jim on 1/15/2025 in #⚡｜serverless

Serverless H200?

@flash-singh the Runpod Serverless Pricing page is not showing correct price for H100 It had to be $4.18/hr on flex but it shows $5.58

11 replies

RRunPod

•Created by Aleksei Naumov on 3/7/2025 in #⚡｜serverless

How to retrieve account spends using GraphQL

Returns this:

{
    "data": {
        "myself": {
            "billing": {
                "summary": [
                    {
                        "time": "2025-02-08T00:00:00.000Z",
                        "gpuCloudAmount": X,
                        "cpuCloudAmount": X,
                        "runpodEndpointAmount": X,
                        "serverlessAmount": XX.XXXXXXXXXXXXXXXX,
                        "storageAmount": XX.XXXXXXXXXXXXXXXX,
                        "__typename": "UserSummaryBilling"
},
...

{
    "data": {
        "myself": {
            "billing": {
                "summary": [
                    {
                        "time": "2025-02-08T00:00:00.000Z",
                        "gpuCloudAmount": X,
                        "cpuCloudAmount": X,
                        "runpodEndpointAmount": X,
                        "serverlessAmount": XX.XXXXXXXXXXXXXXXX,
                        "storageAmount": XX.XXXXXXXXXXXXXXXX,
                        "__typename": "UserSummaryBilling"
},
...

3 replies

RRunPod

•Created by Aleksei Naumov on 3/7/2025 in #⚡｜serverless

How to retrieve account spends using GraphQL

Hey Aleksei you can use this API call to return Daily Summary spends

curl --location 'https://api.runpod.io/graphql' \
 \
--header 'Content-Type: application/json' \
 \
--header 'Authorization: Bearer rpa_....' \
 \
--data '{"query":"query getUserBillingSummary($input: UserBillingInput!) {\n  myself {\n    billing(input: $input) {\n      summary {\n        time\n        gpuCloudAmount\n        cpuCloudAmount\n        runpodEndpointAmount\n        serverlessAmount\n        storageAmount\n        __typename\n      }\n      __typename\n    }\n    __typename\n  }\n}","variables":{"input":{"granularity":"DAILY"}}}' \

curl --location 'https://api.runpod.io/graphql' \
 \
--header 'Content-Type: application/json' \
 \
--header 'Authorization: Bearer rpa_....' \
 \
--data '{"query":"query getUserBillingSummary($input: UserBillingInput!) {\n  myself {\n    billing(input: $input) {\n      summary {\n        time\n        gpuCloudAmount\n        cpuCloudAmount\n        runpodEndpointAmount\n        serverlessAmount\n        storageAmount\n        __typename\n      }\n      __typename\n    }\n    __typename\n  }\n}","variables":{"input":{"granularity":"DAILY"}}}' \

3 replies

RRunPod

•Created by koop7450 on 3/9/2025 in #⚡｜serverless

Use SDK to create Network Storage Volumes for Serverless Endpoints

Yes you can use the GraphQL API call, sharing curl below, simply set your API key, size, data center, name and that is it. Returns the id of the volume on 200

curl --location curl --location 'https://api.runpod.io/graphql' \
 \
--header 'Content-Type: application/json' \
 \
--header 'Authorization: Bearer rpa_.....' \
 \
--data '{"query":"mutation createNetworkVolume($input: CreateNetworkVolumeInput!) {\n  createNetworkVolume(input: $input) {\n    id\n    name\n    size\n    dataCenterId\n    __typename\n  }\n}","variables":{"input":{"name":"volume-name","size":10,"dataCenterId":"EU-CZ-1"}}}' \

curl --location curl --location 'https://api.runpod.io/graphql' \
 \
--header 'Content-Type: application/json' \
 \
--header 'Authorization: Bearer rpa_.....' \
 \
--data '{"query":"mutation createNetworkVolume($input: CreateNetworkVolumeInput!) {\n  createNetworkVolume(input: $input) {\n    id\n    name\n    size\n    dataCenterId\n    __typename\n  }\n}","variables":{"input":{"name":"volume-name","size":10,"dataCenterId":"EU-CZ-1"}}}' \

Response:

{
    "data": {
        "createNetworkVolume": {
            "id": "0i.....",
            "name": "volume-name",
            "size": 10,
            "dataCenterId": "EU-CZ-1",
            "__typename": "NetworkVolume"
        }
    }
}

{
    "data": {
        "createNetworkVolume": {
            "id": "0i.....",
            "name": "volume-name",
            "size": 10,
            "dataCenterId": "EU-CZ-1",
            "__typename": "NetworkVolume"
        }
    }
}

8 replies

RRunPod

•Created by JohnDoe on 2/12/2025 in #⚡｜serverless

Pulling from the wrong cache when multiple Dockerfiles in same GitHub repo

Hey @flash-singh let me also add one of my similar issue: 1 Repo, 2 branch, 2 deployment Branches are mostly identical but each branch's Dockerfile has an ENV variable model name different - And on docker build, downloads the model with this ENV variable & uses that ENV variable on runtime to use that model On my concurrent deployment for these 2 branches After deployment, on deployment b - branch b: It tried to find correct model name (ENV on runtime was ok) but couldn't find it's model I doubt that it used the cache layer from branch-a's deployment of download_models.py, code is 100% same, file reads model_type from ENV Re-deploy branch-b fixed issue

24 replies

RRunPod

•Created by pkpio on 2/4/2025 in #⚡｜serverless

Setting up CD for serverless endpoint

it's the one with mutation saveTemplate($input: SaveTemplateInput) {

13 replies

RRunPod

•Created by pkpio on 2/4/2025 in #⚡｜serverless

Setting up CD for serverless endpoint

Yes it is not well documented, you can go ahead to web UI and view Network tab requests The request got triggered when clicking New Release -> Save the request you need, you can copy it via CURL, remove headers and put api key, then you will be changin imageName there when you want to release a new image This updates Template image that endpoint uses

13 replies

RRunPod

•Created by Eren on 1/28/2025 in #⚡｜serverless

delayTime representing negative value

Hey @yhlong00000 no I did not save the request id

4 replies

RRunPod

•Created by Eren on 1/28/2025 in #⚡｜serverless

delayTime representing negative value

4 replies

RRunPod

•Created by EMPZ on 12/16/2024 in #⚡｜serverless

GitHub integration: "exporting to oci image format" takes forever.

I'm following this thread since December and would be absolutely happy to have faster image builds As Arkadiy said in my case too there is only change on last uncached part on code level, however I still got >1 hr (sometimes 2-3 or even 4 hours) of "exporting to oci image format. This takes a little bit of time. Please be patient." It also randomly fails due to network errors after a while on build

25 replies

RRunPod

•Created by jackson hole on 1/13/2025 in #⚡｜serverless

I want to increase/decrease workers by code or script, can you help? (GraphQL)

🚀 PS: To set Active Worker you need to set the workersMin variable in json

13 replies

RRunPod

•Created by jackson hole on 1/13/2025 in #⚡｜serverless

I want to increase/decrease workers by code or script, can you help? (GraphQL)

it's actually possible by Graph QL API, you can find a sample Endpoint mutation call below

curl --location --globoff 'https://api.runpod.io/graphql?api_key={{RUNPOD_API_KEY}}' \
--header 'content-type: application/json' \
--data '{"query":"mutation saveEndpoint($input: EndpointInput!) {\n  saveEndpoint(input: $input) {\n    gpuIds\n    id\n    idleTimeout\n    locations\n    name\n    networkVolumeId\n    scalerType\n    scalerValue\n    templateId\n    userId\n    workersMax\n    workersMin\n    gpuCount\n    __typename\n  }\n}","variables":{"input":{"gpuIds":"ADA_24,AMPERE_24,-NVIDIA L4,-NVIDIA RTX A5000","gpuCount":1,"allowedCudaVersions":"","id":"{{ID}}","idleTimeout":1,"locations":null,"name":"faster_whisper -fb","networkVolumeId":null,"scalerType":"QUEUE_DELAY","scalerValue":4,"workersMax":3,"workersMin":0,"executionTimeoutMs":180000}}}'

curl --location --globoff 'https://api.runpod.io/graphql?api_key={{RUNPOD_API_KEY}}' \
--header 'content-type: application/json' \
--data '{"query":"mutation saveEndpoint($input: EndpointInput!) {\n  saveEndpoint(input: $input) {\n    gpuIds\n    id\n    idleTimeout\n    locations\n    name\n    networkVolumeId\n    scalerType\n    scalerValue\n    templateId\n    userId\n    workersMax\n    workersMin\n    gpuCount\n    __typename\n  }\n}","variables":{"input":{"gpuIds":"ADA_24,AMPERE_24,-NVIDIA L4,-NVIDIA RTX A5000","gpuCount":1,"allowedCudaVersions":"","id":"{{ID}}","idleTimeout":1,"locations":null,"name":"faster_whisper -fb","networkVolumeId":null,"scalerType":"QUEUE_DELAY","scalerValue":4,"workersMax":3,"workersMin":0,"executionTimeoutMs":180000}}}'

GraphQL API Docs: https://graphql-spec.runpod.io/ Without API Docs: you can go to your dashboard and open the Developer Tools by F12 go to Network tab then make an update on your endpoint and find the mutation API call then you can just copy that as CURL and work on it on postman remember to remove all Headers and add api_key query param

13 replies

Gaming

Programming