EMPZ Comments - Answer Overflow

EMPZ

Explore posts from servers

RRailway RRunPod

Posts Comments

RRunPod

•Created by EMPZ on 8/21/2024 in #⚡｜serverless

Serverless worker doesn't run asynchronously until I request its status in local development

Nope, I just do /runSync when testing locally.

14 replies

RRunPod

•Created by EMPZ on 9/12/2024 in #⚡｜serverless

Very slow upload speeds from serverless workers

It's not the EUR-IS. It's mostly the EU-RO-1, but if I uncheck it, there's no availability of the 16 GB GPU... Europe-to-US latencies should not cause this kind of slowdowns...

13 replies

RRunPod

•Created by EMPZ on 9/12/2024 in #⚡｜serverless

Very slow upload speeds from serverless workers

I'm sorry, but I keep getting this replies that tries to pin the cause of the issue to anything except RunPod's network. Of course there's nothing wrong with the Docker image, it's the default nvidia/cuda image. These extremely slow downs are not all the time, it happens some time, but when it happens it's crazy. Again, 2 minutes to transfer a single 8 MB file. I'm not usually this mad when these things happen, but here I'm being charged GPU time just to have a worker transferring files at 65 kB/second...

13 replies

RRunPod

•Created by EMPZ on 9/12/2024 in #⚡｜serverless

Very slow upload speeds from serverless workers

13 replies

RRunPod

•Created by EMPZ on 9/12/2024 in #⚡｜serverless

Very slow upload speeds from serverless workers

Same, using pre-signed URLs. I understand there might be networks slowdowns from time to time and that there's an extra latency between EU and US/CA, but 2 minutes for a 8 MB file is not an acceptable slowdown. Especially because I'm paying GPU time for this. Is there some kind of storage I can upload the files I need to process beforehand so that when the worker runs, can retrieve them quickly? Can volumes be used for this purpose?

13 replies

RRunPod

•Created by EMPZ on 9/12/2024 in #⚡｜serverless

Very slow upload speeds from serverless workers

I'm experiencing the same issue with downloading. Almost 2 minutes to download a file!!

13 replies

RRunPod

•Created by thushar on 8/20/2024 in #⚡｜serverless

Edit endpoint with new docker image

Is changing the image version in Edit Template the same as using the New Release dialog?

20 replies

RRunPod

•Created by EMPZ on 8/21/2024 in #⚡｜serverless

Do I need to base my serverless worker image from the official base image?

Thank you, I got it working.

4 replies

RRunPod

•Created by papanton on 2/11/2024 in #⚡｜serverless

webhooks custom updates

Can someone give me the TL;DR; for serverless webhooks? I've just started with RunPod and I'm planning to host some production inference jobs. Should I be using webhook or webhookV2?

18 replies

RRunPod

•Created by EMPZ on 8/21/2024 in #⚡｜serverless

Serverless worker doesn't run asynchronously until I request its status in local development

My Dockerfile ends with

CMD python3 -u ./handler.py

CMD python3 -u ./handler.py

But on development I'm launching it from a compose file where I override the command with

services:
  myservice:
    // Omitted
    command: python3 ./handler.py --rp_serve_api --rp_api_host 0.0.0.0 --rp_api_port 8000
    deploy:
      resources:
        reservations:
          devices:
            - driver: nvidia
              count: all
              capabilities: [gpu]

services:
  myservice:
    // Omitted
    command: python3 ./handler.py --rp_serve_api --rp_api_host 0.0.0.0 --rp_api_port 8000
    deploy:
      resources:
        reservations:
          devices:
            - driver: nvidia
              count: all
              capabilities: [gpu]

14 replies

RRunPod

•Created by EMPZ on 8/21/2024 in #⚡｜serverless

Serverless worker doesn't run asynchronously until I request its status in local development

If I do /runsync, they do start immediately.

14 replies

RRunPod

•Created by EMPZ on 8/21/2024 in #⚡｜serverless

Serverless worker doesn't run asynchronously until I request its status in local development

Yes, as I said, the webserver works fine, I can hit the endpoints from my local machine. The jobs won't simply start if I call /run unless I request the status afterwards.

14 replies

RRunPod

•Created by thushar on 8/20/2024 in #⚡｜serverless

Edit endpoint with new docker image

Yes, sometimes it works, but sometimes the workers just hang updating and I have to kill them. I've just started today with the platform and I'm doing some tests to see if it's worth to migrate from fly.io. Everything sounds great when reading the product pages and documentation, but I'm just wondering if it's a stable and mature service to migrate our production GPU workloads...

20 replies

RRunPod

•Created by thushar on 8/20/2024 in #⚡｜serverless

Edit endpoint with new docker image

20 replies

RRunPod

•Created by EMPZ on 8/21/2024 in #⚡｜serverless

Serverless worker doesn't run asynchronously until I request its status in local development

I have not, but anyway it should work well by running a local uvicorn server.

14 replies

RRunPod

•Created by EMPZ on 8/21/2024 in #⚡｜serverless

Pushing a new release to my existing endpoint takes too long

12 replies

RRunPod

•Created by EMPZ on 8/21/2024 in #⚡｜serverless

Pushing a new release to my existing endpoint takes too long

Well, it's not working at all. It's been 30 minutes since I created the new release and nothing happened. The workers are still on the previous version. I needed to terminate all workers to get update ones..

12 replies

RRunPod

•Created by thushar on 8/20/2024 in #⚡｜serverless

Edit endpoint with new docker image

Nevermind, I've just found out the "New Release" option in the endpoint.

20 replies

RRunPod

•Created by thushar on 8/20/2024 in #⚡｜serverless

Edit endpoint with new docker image

Sorry, I'm not following. When I created the endpoint I specified a specific tagname from my private DockerHub. Then, I pushed a new tag but I don't see a way to tell RunPod to pull this new tag. Unless I create a new endpoint, which is very inconvenient because that changes the URL and then I also need to update my application. What's the recommended approach to update the Endpoint Docker image?

20 replies

RRunPod

•Created by Encyrption on 8/6/2024 in #⚡｜serverless

Does webhook work when testing locally?

I'm also interested in this. A quick try when developing locally didn't call my webhook (yes, it's internet-facing and accessible).

14 replies

Gaming

Programming