ohno Posts - Answer Overflow

ohno

•Created by ohno on 8/31/2024 in #⚡｜serverless

FastAPI RunPod serverless request format

Hi everyone, Second post from me. I am (still) trying to deploy a FastAPI app in a Docker container hosted on a serverless RunPod. Here is a toy example:

import uvicorn

from fastapi import FastAPI
from pydantic import BaseModel


app = FastAPI()

class PingResponse(BaseModel):
    message: str

@app.post(path="/ping", response_model=PingResponse)
async def ping() -> dict[str, str]:

    response = {"message": "ok"}

    return response

if __name__ == "__main__":
    uvicorn.run(app=app, host="0.0.0.0", port=8080)

import uvicorn

from fastapi import FastAPI
from pydantic import BaseModel


app = FastAPI()

class PingResponse(BaseModel):
    message: str

@app.post(path="/ping", response_model=PingResponse)
async def ping() -> dict[str, str]:

    response = {"message": "ok"}

    return response

if __name__ == "__main__":
    uvicorn.run(app=app, host="0.0.0.0", port=8080)

The code is deployed as a Docker image with the port 8080 exposed. My problem is: I just can't figure out how to format my requests so that the payload hits the FastAPI path. This is the format I started with:

import requests

URL = "https://api.runpod.ai/v2/<POD_ID>/runsync"
HEADERS = {"Authorization" : "Bearer <TOKEN>"}

body = {
    "input": {
        "api": {
            "method": "POST",
            "endpoint": "/ping",
        },
        "payload": {}, # Empty for example, but holds parameters in real case
    },
}

response = requests.post(URL, json=body, headers=HEADERS)

import requests

URL = "https://api.runpod.ai/v2/<POD_ID>/runsync"
HEADERS = {"Authorization" : "Bearer <TOKEN>"}

body = {
    "input": {
        "api": {
            "method": "POST",
            "endpoint": "/ping",
        },
        "payload": {}, # Empty for example, but holds parameters in real case
    },
}

response = requests.post(URL, json=body, headers=HEADERS)

The request does get to the serverless worker, but then just hangs without being forwarded to the FastAPI path method. Am I missing something here? P.S.: last question from me I swear.

22 replies

RRunPod

•Created by ohno on 8/30/2024 in #⚡｜serverless

Mounting network volume into serverless Docker container

Hi everyone, I am trying to deploy a FastAPI app in a Docker container hosted on a serverless RunPod. My issue is: I need to pass a very large data folder to the Docker container. I could easily do this on a Pod by using the following Container Start Command:

docker run -it -p 8080:8080 -v "$(shell cd /workspace; pwd)":/opt/data my_image:latest

docker run -it -p 8080:8080 -v "$(shell cd /workspace; pwd)":/opt/data my_image:latest

However, I simply can't get it to work with a serverless deployment. I know the network volume is mounted at /runpod-volume rather than /workspace for serverless. But even by adjusting my Docker run command, I just can't get it to show up inside my container:

docker run -it -p 8080:8080 -v /runpod-volume:/opt/data my_image:latest

docker run -it -p 8080:8080 -v /runpod-volume:/opt/data my_image:latest

Also: serverless templates don't offer the Volume Mount Path option that is present for Pod templates, so that's a no-go as well. Has anyone any idea what I am doing wrong?

6 replies

Gaming

Programming