Jack
RRunPod
•Created by Jack on 1/25/2024 in #⚡｜serverless
How to get around the 10/20 MB payload limit?
For use cases such as training LoRAs with Stable Diffusion, where a user could upload tens of photos, 10/20MB is quite small. This is especially true because you have to convert an image to base64 before sending it to the endpoint, which will increase the size of each photo.
My app requires the user to upload photos of themselves for training purposes. And if I can't find a way around the 10 MB payload limit, I just realized I can't use runpod's serverless GPUs.
Are there any clever ways of getting around this payload limit?
7 replies
RRunPod
•Created by Jack on 1/25/2024 in #⚡｜serverless
/runsync/ getting this error - {"Error":"bad request: body: exceeded max body size of 10MiB"}
In my app, I need the user to upload photos of themselves.
According to the docs here. The payload capacity for /runsync/ is 20MB
https://docs.runpod.io/docs/serverless-endpoint-urls
However,
/runsync/ is replying that the max body size is 10 MB, whereas in the docs, it's actually 20 MB.3 replies
RRunPod
•Created by Jack on 1/23/2024 in #⚡｜serverless
Tips on avoiding hitting this error whilst checking `/status/:job_id` using requests?
Full Error pasted at the bottom.
After sending out a request, I'm using
requests.get() to check the /status/:job_id of the endpoint every 3 seconds until the job either returns FAILED or COMPLETE.
Unfortunately one of the requests had a particularly long delay time (234 secs), on top of the 57 secs of execution time.
As I was checking /status/:job_id for this request, I got this error.
My question is - how do I make sure I don't hit this error when checking /status/:job_id?
requests.exceptions.SSLError: HTTPSConnectionPool(host='api.runpod.ai', port=443): Max retries exceeded with url: /v2/85i9atfpo9xuds/status/fbb4c8ff-93ff-41c5-954d-f799d0d5407a-e1 (Caused by SSLError(SSLEOFError(8, '[SSL: UNEXPECTED_EOF_WHILE_READING] EOF occurred in violation of protocol (_ssl.c:1006)')))3 replies
RRunPod
•Created by Jack on 1/14/2024 in #⚡｜serverless
Do you get charged whilst your request is waiting on throttled workers?
7 replies
RRunPod
•Created by Jack on 1/14/2024 in #⚡｜serverless
Is there a way to send an request to cancel a job if it takes too long?
I'm trying to find a way to cancel an API request if it takes too long. This is so that I have a way to deal with requests that are stuck "IN_QUEUE".
Preferably something like:
https://api.runpod.ai/v2/{endpoint_id}/cancel_job/{job_id}
Or even better. Enable a setting on the endpoint that automatically cancels any API requests that take longer than X amount of time (e.g. 10 mins).
4 replies
How to enable Jupyter Notebook and SSH support in a custom Docker container?
I built my own docker image to deploy on a pod. After creating the Custom Template with my docker image, there is no option to enable Jupyter Notebook or SSH for it.
I tried my best to imitate the official Runpod containers, by installing
jupyterlabs and openssh-server, but when setting up the pod, there is still no option to enable Jupyter Notebooks or SSH on the pod.
I am also not able to find any guides on how to incorporate Jupyter notebook support on a custom docker image.
How do I do it?18 replies
RRunPod
•Created by Jack on 1/12/2024 in #⚡｜serverless
Error whilst using Official A1111 Runpod Worker - CUDA error: an illegal instruction was encountered
https://github.com/runpod-workers/worker-a1111
I am using the official A1111 Runpod Worker. It's not actively maintained and I ran into 2 issues whilst building the Docker image but those were easily resolved.
After successfully building the docker image and loading it onto an API endpoint, I'm getting an error which I'm struggling to solve.
It's a RunTime error and here's the full error message:
CUDA error: an illegal instruction was encountered\nCUDA kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect.\nFor debugging consider passing CUDA_LAUNCH_BLOCKING=1.\nCompile with TORCH_USE_CUDA_DSA to enable device-side assertions.\n
I don't really know how to start debugging this issue. Any pointers?4 replies
RRunPod
•Created by Jack on 1/9/2024 in #⚡｜serverless
ailed to load library libonnxruntime_providers_cuda.so
Here is the full error:
[E:onnxruntime:Default, provider_bridge_ort.cc:1480 TryGetProviderInfo_CUDA] /onnxruntime_src/onnxruntime/core/session/provider_bridge_ort.cc:1193 onnxruntime::Provider& onnxruntime::ProviderLibrary::Get() [ONNXRuntimeError] : 1 : FAIL : Failed to load library libonnxruntime_providers_cuda.so with error: libcufft.so.10: cannot open shared object file: No such file or directory
I am running AUTOMATIC1111 on Serverless Endpoints using a Network Volume. I am using the faceswaplab extension. In this extension, there is the option to use GPU (by default, the extension only uses CPU). When I turn on the Use GPU option, I get the error.
It would seem the Serverless Endpoint does not have the
libonnxruntime_providers_cuda.so library. Can I install this particular library into the Servless Endpoint myself? Either onto the Network Volume or the Docker container?10 replies
RRunPod
•Created by Jack on 12/25/2023 in #⚡｜serverless
Queued serverless workers not running and getting charged for it?
13 replies
RRunPod
•Created by Jack on 12/22/2023 in #⚡｜serverless
Do I need to keep Pod open after using it to setup serverless APIs for stable diffusion?
Hi I'm following this tutorial on building serverless endpoints for running txt2img with ControlNet - https://www.youtube.com/watch?v=gv6F9Vnd6io
My question is - After deploying a Pod to set up the Network Volume to receive serverless endpoint requests, can I terminate the Pod and the disk attached to the Pod? Or do I have to keep the Pod running in order to receive serverless endpoint requests at any time?
4 replies
RRunPod
•Created by Jack on 12/21/2023 in #⚡｜serverless
Is it possible to build an API for an automatic1111 extension to be used through Runpod serverless?
I want to use the faceswaplab extension for automatic1111 as a serverless endpoint on Runpod.
I managed to find this github repository which features reActor (a similar faceswap extension) but not faceswaplab https://github.com/ashleykleynhans/runpod-worker-a1111
Is it possible build faceswaplab's features as an API to be used through Runpod's serverless endpoints? If yes, any pointers on how I can do so?
10 replies