letajmal Comments - Answer Overflow

Topics

letajmal

•Created by letajmal on 3/12/2024 in #⚡｜serverless

How can i make a follow up question to the endpoint

The question is not valid, I need to handle things like this in the model itself

7 replies

•Created by letajmal on 3/12/2024 in #⚡｜serverless

How can i make a follow up question to the endpoint

I have a serverless core42/jais-13b-chat endpoint

7 replies

•Created by letajmal on 3/12/2024 in #⚡｜serverless

How can i make a follow up question to the endpoint

like if i have already made an API request to my serverless endpoint, can i make another follow up request

7 replies

•Created by letajmal on 3/8/2024 in #⚡｜serverless

I am getting no response from serverless

Sorry, I forgot to mark it as solved

11 replies

•Created by letajmal on 3/8/2024 in #⚡｜serverless

I am getting no response from serverless

@justin [Not Staff] you were right, the issue was with the handler function. Thanks for your help guys

11 replies

•Created by letajmal on 3/8/2024 in #⚡｜serverless

I am getting no response from serverless

FROM runpod/base:0.4.0-cuda11.8.0 COPY builder/requirements.txt /requirements.txt RUN python3.11 -m pip install --upgrade pip && \ python3.11 -m pip install --upgrade -r /requirements.txt --no-cache-dir && \ rm /requirements.txt COPY builder/download.py /download.py RUN python3.11 /download.py ADD src . CMD python3.11 -u /handler.py

11 replies

•Created by letajmal on 3/7/2024 in #⚡｜serverless

Should i wait for the worker to pull my image

should i wait for all 3 workers to download the image?

7 replies

•Created by letajmal on 3/7/2024 in #⚡｜serverless

Should i wait for the worker to pull my image

No description

7 replies

•Created by letajmal on 3/7/2024 in #⚡｜serverless

Should i wait for the worker to pull my image

I've set a maximum of 3 workers. Can I initiate the job once any worker finishes downloading the image? I've noticed some image pull attempts failing and restarting. Should I terminate those workers, or will they recover automatically?

7 replies

•Created by letajmal on 3/1/2024 in #⚡｜serverless

What is the recommended System Req for Building Worker Base Image

24vCPU, 128 GB is not enough for CUDA 12.1

19 replies

•Created by letajmal on 3/1/2024 in #⚡｜serverless

What is the recommended System Req for Building Worker Base Image

for CUDA 12.1, this configuration is not enough

19 replies

•Created by letajmal on 3/1/2024 in #⚡｜serverless

What is the recommended System Req for Building Worker Base Image

I was able to build the image(CUDA 11.8) using a GCE VM 24vCPU, 128 GB. I am not sure if this is required, i was using the max cap in gcp free tier Also docker.io/docker image was not working because bash was not installed, so the ssh connection was failing If someone is trying to use runpod CPU/GPU for this purpose use some ubuntu dind image or github actions runner image by the community

19 replies

•Created by letajmal on 3/1/2024 in #⚡｜serverless

What is the recommended System Req for Building Worker Base Image

I have added a template for docker.io/docker but i am not able to start a web terminal session

19 replies

•Created by letajmal on 3/1/2024 in #⚡｜serverless

What is the recommended System Req for Building Worker Base Image

Let me try, thanks for the info

19 replies

•Created by letajmal on 3/1/2024 in #⚡｜serverless

What is the recommended System Req for Building Worker Base Image

since CPU pods are not that powerful, they are useless in this context

19 replies