nevermind Comments - Answer Overflow

Topics

nevermind

•Created by K-Woww on 11/1/2024 in #⛅｜pods

Is there something wrong in US-OR-1?

US-OR-1 is bad. We described the same problem directly to the RunPod team and also supplemented the report with speedtest. It is usually 3x slower than US-TX-3 with x6 ping

18 replies

•Created by nevermind on 9/26/2024 in #⛅｜pods

Urgent: {'message': 'Something went wrong. Please try again later or contact support.'}

We have a bunch of these errors in Grafana, so we can provide any additional information.

7 replies

•Created by nevermind on 9/26/2024 in #⛅｜pods

Urgent: {'message': 'Something went wrong. Please try again later or contact support.'}

First three occurred while podFindAndDeployOnDemand, Last one - query myPods

7 replies

•Created by nevermind on 9/26/2024 in #⛅｜pods

Urgent: {'message': 'Something went wrong. Please try again later or contact support.'}

We get it with python httpx, not UI

7 replies

•Created by nevermind on 9/26/2024 in #⛅｜pods

Urgent: {'message': 'Something went wrong. Please try again later or contact support.'}

Additionally, during this time we experience read timeouts (requests taking more than 90 seconds to process). I think these problems are interrelated.

7 replies

•Created by nevermind on 9/21/2024 in #⛅｜pods

SOS pod gpu errors

Alright, if we face the same issue again - I'll report to this thread and try my best to lock the pod

8 replies

•Created by nevermind on 9/21/2024 in #⛅｜pods

SOS pod gpu errors

We actually use multiple instances of this configuration and sometimes pods just "die" during its lifetime (it affects our flow control).

8 replies

•Created by nevermind on 9/21/2024 in #⛅｜pods

SOS pod gpu errors

Whats an "ERR!" than?

8 replies

•Created by bghira on 9/4/2024 in #⛅｜pods

GPU errored, machine dead

May be I should bring it into the feedback

11 replies

•Created by bghira on 9/4/2024 in #⛅｜pods

GPU errored, machine dead

Our practice is to run a short cuda test (like getting statistics or something). I think it will enhance DX if they do this on their side.

11 replies

•Created by bghira on 9/4/2024 in #⛅｜pods

GPU errored, machine dead

Why these pods are exposed to the users 🤯 It's such an easy task to detect broken gpu for RunPod, but they just ignore this issue for like 3 month

11 replies

•Created by nevermind on 9/3/2024 in #⛅｜pods

My pod had been stuck during initialization

Normally this image pulling for 1-2 mins, but these pods were pulling it for 5 min, until I've killed them

11 replies

•Created by nevermind on 9/3/2024 in #⛅｜pods

My pod had been stuck during initialization

Endless image fetching. Like there was no progress bar, just "still fetching XXX"

11 replies

•Created by nevermind on 9/3/2024 in #⛅｜pods

My pod had been stuck during initialization

This happened again right now - 9ff2sxw9irvb5s

11 replies

•Created by nevermind on 8/21/2024 in #⛅｜pods

How does runpod handle pod terminating

I appreciate your advice, I'll send it as feedback tho

26 replies

•Created by nevermind on 8/21/2024 in #⛅｜pods

How does runpod handle pod terminating

Because kubernetes does that way and it allows pod to handle graceful term, instead of instant annihillation (runpod does that way)

26 replies

•Created by nevermind on 8/21/2024 in #⛅｜pods

How does runpod handle pod terminating

pod receives termination -> sigterm -> 1min alive (graceful period) -> pod sends sigkill and dies

26 replies

•Created by nevermind on 8/21/2024 in #⛅｜pods

How does runpod handle pod terminating

for a graceful minute

26 replies

•Created by nevermind on 8/21/2024 in #⛅｜pods

How does runpod handle pod terminating

yeah you right

26 replies

•Created by nevermind on 8/21/2024 in #⛅｜pods

How does runpod handle pod terminating

on demand

26 replies