EMPZ Posts - Answer Overflow

EMPZ

Explore posts from servers

•Created by EMPZ on 10/8/2024 in #⚡｜serverless

Can we run Node.js on a Serverless Worker?

According to the Serverless Overview doc page (https://docs.runpod.io/serverless/workers/overview), we can write functions in the language you're most comfortable with. There's a Runpod SDK on NPM (https://www.npmjs.com/package/runpod-sdk), but that looks like it's meant to call existing endpoints, not to create handler functions. Is this possible? If so, are there any templates available to create the handler function in Node.js?

3 replies

RRunPod

•Created by EMPZ on 9/12/2024 in #⚡｜serverless

Very slow upload speeds from serverless workers

I'm uploading files to Supabase from within the serverless workers and I noticed the process is extremly slow. I understand there's some latency because most workers I'm getting are in Europe and my Supabase instance is on US east, but still, almost 20 seconds to upload a 8MB file is bad. I've check it's not a Supabase issue as I'm based in Europe and my upload speeds are just fine.

13 replies

RRunPod

•Created by EMPZ on 8/25/2024 in #⚡｜serverless

Question about delay and execution time billing

4 replies

RRunPod

•Created by EMPZ on 8/21/2024 in #⚡｜serverless

The Workers tab of an endpoint crashed with a frontend error

1 replies

RRunPod

•Created by EMPZ on 8/21/2024 in #⚡｜serverless

Pushing a new release to my existing endpoint takes too long

I've pushed a new release to my endpoint with a new Docker tag image. This tag only modifies some app code, so all the heavy docker layers should already be there. The system logs show "Pulling fs layer" for most of the layers except the first 5. Isn't RunPod caching the layers somewhere or does it have to pull ALL layers everytime I push a new release even though only the last layer has changed...?

12 replies

RRunPod

•Created by EMPZ on 8/21/2024 in #⚡｜serverless

Serverless worker doesn't run asynchronously until I request its status in local development

I'm following the docs and created a verey simple handler.py with the following:

import runpod

def handler(event):
    """
    This is a sample handler function that echoes the input
    and adds a greeting.
    """
    try:
        # Extract the prompt from the input
        prompt = event["input"]["prompt"]

        print (f"Prompt: {prompt}")

        result = f"Hello! You said: {prompt}"

        # Return the result
        return {"output": result}
    except Exception as e:
        # If there's an error, return it
        return {"error": str(e)}

# Start the serverless function
runpod.serverless.start({"handler": handler})

import runpod

def handler(event):
    """
    This is a sample handler function that echoes the input
    and adds a greeting.
    """
    try:
        # Extract the prompt from the input
        prompt = event["input"]["prompt"]

        print (f"Prompt: {prompt}")

        result = f"Hello! You said: {prompt}"

        # Return the result
        return {"output": result}
    except Exception as e:
        # If there's an error, return it
        return {"error": str(e)}

# Start the serverless function
runpod.serverless.start({"handler": handler})

` This is running inside a Docker container with a custom Dockerfile and executed with

python3 ./handler.py --rp_serve_api --rp_api_host 0.0.0.0 --rp_api_port 8000

python3 ./handler.py --rp_serve_api --rp_api_host 0.0.0.0 --rp_api_port 8000

The local server starts:

--- Starting Serverless Worker |  Version 1.6.2 ---
INFO   | Starting API server.
DEBUG  | Not deployed on RunPod serverless, pings will not be sent.      
INFO:     Started server process [1]                                     
INFO:     Waiting for application startup.
INFO:     Application startup complete.                                  
INFO:     Uvicorn running on http://0.0.0.0:8000 (Press CTRL+C to quit)

--- Starting Serverless Worker |  Version 1.6.2 ---
INFO   | Starting API server.
DEBUG  | Not deployed on RunPod serverless, pings will not be sent.      
INFO:     Started server process [1]                                     
INFO:     Waiting for application startup.
INFO:     Application startup complete.                                  
INFO:     Uvicorn running on http://0.0.0.0:8000 (Press CTRL+C to quit)

But when I POST localhost:8000/run with the expected prompt input, I get a id back saying the job is in progress, but nothing shows on the container logs. It doesn't run until I call /status/jobId. Only then it runs. Is this the expected behaviour? If so, why?

14 replies

RRunPod

•Created by EMPZ on 8/21/2024 in #⚡｜serverless

Do I need to base my serverless worker image from the official base image?

I have my own Dockerfile already optimized with only the things I need to perform the inference. Runpod docs says that we should start by forking the worker-template, but basing from it I end up with a HUGE image. Is there anything special in the runpod/base image or can I just use my own and simply make sure I'm installing runpod in Python and exposing a handler function with CMD at the end?

4 replies

Gaming

Programming