Mohamed Nagy Posts - Answer Overflow

Mohamed Nagy

•Created by Mohamed Nagy on 1/26/2025 in #⚡｜serverless

How to respond to the requests at https://api.runpod.ai/v2/<YOUR ENDPOINT ID>/openai/v1

the openai input is in the job input, I extracted it and processes the request . when send the the response with yield or return it recived could you take a look at this [https://github.com/mohamednaji7/runpod-workers-scripts/blob/main/empty_test/test%20copy%203.py] [https://github.com/mohamednaji7/runpod-workers-scripts/blob/main/empty_test/handler.py] when I run made an request I got

   raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 500 Server Error: Internal Server Error for url: https://api.runpod.ai/v2/nqe3wqry3h7noa/openai/v1/chat/completions

   raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 500 Server Error: Internal Server Error for url: https://api.runpod.ai/v2/nqe3wqry3h7noa/openai/v1/chat/completions

18 replies

RRunPod

•Created by Mohamed Nagy on 1/20/2025 in #⚡｜serverless

I want to deploy a serverless endpoint with using Unsloth

Unsloth do bnb qunatization and it's better loading their model, I think. I did training using Unsloth on a pod; I want to deploy it on a serverless endpoint and get the OpenIA client API

2 replies

Gaming

Programming