R
RunPod•6d ago
sumansid

How can I run a pod as an API endpoint? (not serverless)

I want to run a model as an api endpoint
Solution:
Flask, FastAPI etc, but it can't scale so its not recommended to use an API on pods. its much better to use serverless.
Jump to solution
2 Replies
nerdylive
nerdylive•6d ago
well make the api search google Its too long to explain here, but anyways on python you got a few libraries to help with creating api's
Solution
digigoblin
digigoblin•6d ago
Flask, FastAPI etc, but it can't scale so its not recommended to use an API on pods. its much better to use serverless.
Want results from more Discord servers?
Add your server
More Posts
can't start pod using the cliI'm trying to start a pod using the cli by doing: ``` runpodctl start pod <id> ``` but I'm getting Can two serverless endpoint point to the same docker image with different tags?Hey all, I didn't realized deploy a new version will update the images for both my serverless endpoirunpod-python sdk to create serverless endpointHi I want to create and deploy a new serverless gpu endpoint by python sdk. But it seems https://giHow can I increase the execution wait time?I have a task running for about 30 minutes, I specify a parameter in the request: "policy": { Connection timeout to host https://api.runpod.ai/v2Whenever I'm sending a lot of async requests (approximately 100), my worker getting timeout exceptioComfyui not using GPU, how to fix this?We are running in instance of Comfyui and it's using CPU and not GPU, using a 4090. How can this be VLLM WORKER ERRROROn fp8 quantization:Just created a pod and it started with 99% gpu utilizationerror starting: Error response from daemon: Container aa58de3216b8515a3ee78aa46d9102331aaaf6c210a36cHey all, all my requests stopped working on p45dj1lfott9ob. Example: 2319a9c9-dccd-4930-b34f-0acceServerless Hardware equivalent of endpointHi guys I'm currently migrating from Faster Whisper endpoint to Serverless. What configuration would