R
RunPod•7mo ago
rewbs

Jobs suddenly queuing up: only 1 worker active, 9 jobs queued

Endpoint: vieo12phdoc8kh Hi, are there any known issues at the moment with 4090s? Our processing has suddenly started backing up with no known changes on our side. Only 1 of our 8 workers seems to be active, with the others sitting "ready" according to the logs. This issue appears to have started about 90 minutes ago. We have failed over to another provider for now.
No description
4 Replies
rewbs
rewbs•7mo ago
Things seem to be running smoothly again now but it does look like there was an issue. It would be awesome to know if there's any way I could get more info on what happened here.
No description
flash-singh
flash-singh•7mo ago
are you using network volume? if so what region?
rewbs
rewbs•7mo ago
No network volume
flash-singh
flash-singh•7mo ago
we had a huge spike in 4090s usage, thats more stable now, we plan to add 100s more by next week
Want results from more Discord servers?
Add your server
More Posts
Issues with building the new `worker-vllm` Docker ImageI've been using the previous version of `worker-vllm` with the `awq` model in production, and it recImportError: version conflict: '/opt/micromamba/envs/comfyui/lib/python3.10/site-packages/psutil/_psI'm spinning up a new pod and copying from backblaze B2, it works just fine before the download but Jupyter runpod proxy extremely slowHello, since a few days im having massive issues with Jupyter running on runpod proxys. Its abysmallRunpod Running Slower Than Local MachineI conducted a benchmark test on stable diffusion image-to-image. My pipeline involves using ControlNHow to transfer outputs when GPU is not available?I ran into an issue with trying to transfer my outputs from a pod with 0 GPU's. I wasn't able to useCan I spin up a pod pre-loaded with my /workspace?I've been spinning up pods under a network volume but I'm not quite sure how to actually use it to mNew to RunPod but problemsI was excited to use the service mainly for the pre-set templates. And at first, the Stable DiffusioCannot turn on pod to backup dataHi guys, I cannot turn on my pod to backup the data before you wipe tomorrow.error with github workflowhi im using: https://github.com/runpod-workers/worker-template/tree/main as a template for the CI/Create new template from runpod sdxl github release 1.2.0I see that the github release has been updated here: https://github.com/runpod-workers/worker-sdxl a