Encyrption Comments - Answer Overflow

Encyrption

•Created by Ammar Ahmed on 10/31/2024 in #⛅｜pods

Network Volume as Storage for images

IMHO you would be better storing your images in a S3 bucket somewhere. Network Volume are difficult to work with and are slow. For example, I store mine on Digital Oceans Spaces. There is rumors of RunPod offering them at some point.

3 replies

RRunPod

•Created by Untrack4d on 9/13/2024 in #⚡｜serverless

Assincronous Job

Here is an example return from one of my endpoints. You're custom output will be in output. This is whatever you returned from the handler.

{
  "delayTime": 491,
  "executionTime": 135484,
  "id": "your-unique-id-will-be-here",
  "output": {
    "output_video_url": "https://mybucket.nyc3.digitaloceanspaces.com/SadTalker/2024_06_16_16.20.48.mp4"
  },
  "status": "COMPLETED"
}

{
  "delayTime": 491,
  "executionTime": 135484,
  "id": "your-unique-id-will-be-here",
  "output": {
    "output_video_url": "https://mybucket.nyc3.digitaloceanspaces.com/SadTalker/2024_06_16_16.20.48.mp4"
  },
  "status": "COMPLETED"
}

7 replies

RRunPod

•Created by Untrack4d on 9/13/2024 in #⚡｜serverless

Assincronous Job

Yes, it is the same response you get when a job is completed. Whatever the output was from the request will be POST as JSON to your webhook URL.

7 replies

RRunPod

•Created by Untrack4d on 9/13/2024 in #⚡｜serverless

Assincronous Job

Yeah, you cannot test RUN endpoint locally only RUNSYNC. As the STATUS endpoint is not available locally. Also, webhook is not applied

7 replies

RRunPod

•Created by TheWitcher.... on 10/26/2024 in #⚡｜serverless

How to send an image as a prompt to vLLM?

If your image has been coded for it you can send an image in using a URL to overcome the size issue.

5 replies

RRunPod

•Created by DEOGEE on 10/26/2024 in #⚡｜serverless

Error downloading docker image for custom image mode(juggernaut)

The error seems to be due to two issues in your Dockerfile: Casing Warning: The Dockerfile has a casing inconsistency for FROM and AS. The as in FROM ... AS ... should match the casing of FROM, so it should be capitalized as AS in both instances. This is a warning, but aligning the casing will help maintain code consistency. Pipefail Option in Clone Script: The error in /clone.sh shows an illegal option -o pipefail. It seems set -o pipefail was used, which is only valid in bash, not in /bin/sh (the default shell for Docker RUN commands). To address this: Update the Dockerfile to use bash instead of sh for commands requiring pipefail. Modify the clone command to explicitly use bash to ensure compatibility. Here's a modified snippet of your Dockerfile:

# Update casing in FROM statements
FROM docker.io/alpine/git:2.36.2 AS builder
...
FROM docker.io/library/python:3.10.9-slim AS build_final_image

# Use bash for commands needing pipefail in RUN statements
RUN ["/bin/bash", "-c", ". /clone.sh taming-transformers https://github.com/CompVis/taming-transformers.git 24268930bf1dce879235a7fddd0b2355b84d7ea6 && rm -rf data assets **/*.ipynb"]

# Update casing in FROM statements
FROM docker.io/alpine/git:2.36.2 AS builder
...
FROM docker.io/library/python:3.10.9-slim AS build_final_image

# Use bash for commands needing pipefail in RUN statements
RUN ["/bin/bash", "-c", ". /clone.sh taming-transformers https://github.com/CompVis/taming-transformers.git 24268930bf1dce879235a7fddd0b2355b84d7ea6 && rm -rf data assets **/*.ipynb"]

3 replies

RRunPod

•Created by Jack on 10/18/2024 in #⚡｜serverless

What environment variables are available in a serverless worker?

You can get the POD_ID with ENV variable RUNPOD_POD_ID. Python example below:

POD_ID = os.getenv('RUNPOD_POD_ID')

POD_ID = os.getenv('RUNPOD_POD_ID')

6 replies

RRunPod

•Created by susmitha on 10/18/2024 in #⛅｜pods

I want to get a Public Url

There are 2 ways to do that. You can map a HTTP port. Map a HTTP port in your template and you can get a RunPod Proxy URL to it. In this example using port 8888:

POD_ID = os.getenv('RUNPOD_POD_ID')
WS_PORT = 8888

URL = f'https://{POD_ID}-{WS_PORT}.proxy.runpod.net'

print(f'You can connect to URL {URL}'

POD_ID = os.getenv('RUNPOD_POD_ID')
WS_PORT = 8888

URL = f'https://{POD_ID}-{WS_PORT}.proxy.runpod.net'

print(f'You can connect to URL {URL}'

or you can use a TCP port. Again, in this example using port 8888:

RUNPOD_PUBLIC_IP = os.getenv('RUNPOD_PUBLIC_IP')
RUNPOD_TCP_PORT = os.getenv('RUNPOD_TCP_PORT_8888')

URL = f'http://{RUNPOD_PUBLIC_IP}:{RUNPOD_TCP_PORT}'

print(f'You can connect to URL {URL}'

RUNPOD_PUBLIC_IP = os.getenv('RUNPOD_PUBLIC_IP')
RUNPOD_TCP_PORT = os.getenv('RUNPOD_TCP_PORT_8888')

URL = f'http://{RUNPOD_PUBLIC_IP}:{RUNPOD_TCP_PORT}'

print(f'You can connect to URL {URL}'

With TCP you can see I am using http instead of https. As, with TCP it is direct connection to your Pod and if you want to implement https your Pod has to be configured for SSL and have a SSL certificate installed. Both of these solutions are assuming your image has been coded to listen on the port you have mapped. These examples use Python but you can use any language you are familiar with.

3 replies

RRunPod

•Created by tzk on 8/15/2024 in #⛅｜pods

Running custom docker images (used in Serverless) to use in Pods

My example was not for using it after it is on RunPod it was showing you a way to get a shell from the image on your computer. For that you would pull the image to your computer and then run that command. You will need docker installed. But, that is just one way to get a shell. What @nerdylive has described should work after it is running on RP as a Pod.

9 replies

RRunPod

•Created by chonkie on 10/15/2024 in #⚡｜serverless

Initializing...

I don't think anything that takes that long will be successful. How big is your image?

4 replies

RRunPod

•Created by chonkie on 10/15/2024 in #⚡｜serverless

Initializing...

Can you show me your system and container logs?

4 replies

RRunPod

•Created by bert on 10/14/2024 in #⛅｜pods

How to Keep Installed Python Modules Persistent and How to Mount Multiple Volumes?

Yeah, in most cases you don't need a GPU to build your Docker image. It can be beneficial to test locally but not a requirement.

6 replies

RRunPod

•Created by bert on 10/14/2024 in #⛅｜pods

How to Keep Installed Python Modules Persistent and How to Mount Multiple Volumes?

Mounting multiple volumes is not an allowed. As for keeping Python modules persistent your best option would be to bake them into the image you are using. You might be able to do that with symbolic links into your network volume, but those would also have to be rebuilt each time you run your pod.

6 replies

RRunPod

•Created by joe on 10/14/2024 in #⚡｜serverless

Serverless and Azure.

2 replies

RRunPod

•Created by Ruslan Piloian on 10/8/2024 in #⚡｜serverless

Microsoft Florence-2 model in serverless container doesn't work

That's odd. Maybe it is a specific version of accelerator that it is looking for? If it was recently updated maybe roll back a version. If not, I'm not sure what else it could be.

13 replies

RRunPod

•Created by dxqbYD on 10/10/2024 in #⛅｜pods

Special characters in pod IDs

I just treat the id as a unique string, I don't think this would affect my code, as long as RunPod API responded accordingly to it.

15 replies

RRunPod

•Created by Xqua on 10/10/2024 in #⚡｜serverless

Application error on one of my serverless endpoints

Oh, never mind. That is the RunPod front end. Have you opened a ticket with RunPod?

6 replies

RRunPod

•Created by Xqua on 10/10/2024 in #⚡｜serverless

Application error on one of my serverless endpoints

6 replies

RRunPod

•Created by VidimusWolf on 10/9/2024 in #⚡｜serverless

Keeping Flashboot active?

It is pretty reliable but it was not designed so you can easily subvert it to get regular workers to act like active workers (which runs all the time). It was designed to help with scaling with endpoints that have traffic. The more traffic you have and the more max workers you have the more benefit you will get from Flashboot. If your endpoint has little or no traffic you would be better off adding an active server. I know that is not what you want to hear, as you likely want to pay less instead of more but if you do the research I think you will still find that nobody else will let you scale from 0 like RunPod does.

9 replies

Gaming

Programming