Encyrption Posts - Answer Overflow

Encyrption

Posts Comments

RRunPod

•Created by Encyrption on 9/16/2024 in #⚡｜serverless

All workers saying Retrying in 1 second.

1 replies

RRunPod

•Created by Encyrption on 9/11/2024 in #⚡｜serverless

Can we set public-read with rp_upload?

Using the boto module directly I can set the following when doing an s3 upload:

ExtraArgs={'ACL': 'public-read'}

ExtraArgs={'ACL': 'public-read'}

Is there a way I can apply that when using rp_upload? Below is an example of how I am using it:

image = rp_upload.upload_image(job_id, local_image_path)

image = rp_upload.upload_image(job_id, local_image_path)

This allows the uploaded bucket file to be accessible to the public for downloading.

2 replies

RRunPod

•Created by Encyrption on 8/27/2024 in #⚡｜serverless

v1 API definitions?

Is there any documentation for RunPod v1 endpoints? specifically looking for documentation for: https://hapi.runpod.net/v1/pod/{POD_ID}/logs This seems to be what RunPod uses to stream their logs from serverless workers to their website. I would like to implement similar functionality into my web app rather than streaming those logs over a web socket, with custom code as I do today. Is there an equivalent documented API call I can use to do this? If not, where can I find information on how to use the above API call in my code?

7 replies

RRunPod

•Created by Encyrption on 8/26/2024 in #⚡｜serverless

Monitor GPU VRAM - Which GPU to check?

I am trying to monitor the GPU VRAM usage in serverless worker. To do this with pynvml I need to provide the index of the GPU. Is there a way I can obtain the index of the GPU my worker is using? I did not see this info in the ENV variables. I do see RUNPOD_GPU_COUNT but not sure if that helps. Seems that RunPod is monitoring cpu, gpu stats as they present that information in their web interface. Does the RunPod python module expose those stats, without having to code our own? Below is a code snippet that reports VRAM usage in a %.

import pynvml
import time

# Initialize NVML
pynvml.nvmlInit()

handle = pynvml.nvmlDeviceGetHandleByIndex(0)  # Assuming you have only one GPU

while True:
    # Get the memory information for the GPU
    memory_info = pynvml.nvmlDeviceGetMemoryInfo(handle)

    used_vram = memory_info.used // (1024 ** 2)  # Convert bytes to MB
    total_vram = memory_info.total // (1024 ** 2)  # Convert bytes to MB
    vram_usage_percentage = round((used_vram / total_vram) * 100)

    print(f'vram usage: {vram_usage_percentage}%')

    time.sleep(5)

import pynvml
import time

# Initialize NVML
pynvml.nvmlInit()

handle = pynvml.nvmlDeviceGetHandleByIndex(0)  # Assuming you have only one GPU

while True:
    # Get the memory information for the GPU
    memory_info = pynvml.nvmlDeviceGetMemoryInfo(handle)

    used_vram = memory_info.used // (1024 ** 2)  # Convert bytes to MB
    total_vram = memory_info.total // (1024 ** 2)  # Convert bytes to MB
    vram_usage_percentage = round((used_vram / total_vram) * 100)

    print(f'vram usage: {vram_usage_percentage}%')

    time.sleep(5)

Thanks! 🙂

30 replies

RRunPod

•Created by Encyrption on 8/12/2024 in #⚡｜serverless

Any limits on execution timeout?

If I uncheck Enable Execution Timeout on an endpoint can I run a serverless worker indefinitely? If not what is the actual limit? Thanks!

2 replies

RRunPod

•Created by Encyrption on 8/6/2024 in #⚡｜serverless

Does webhook work when testing locally?

I am trying to test serverless worker locally and everything runs fine except it doesn't call the webhook I provided in the test_input.json file. Here is an example of the JSON I am sending is this correct for calling a webhook? { "input": { "sample": "testvalue" }, "webhook": "https://myhost.com/log" } Thanks! 🙂

14 replies

RRunPod

•Created by Encyrption on 8/5/2024 in #⚡｜serverless

Are the 64 / 128 Core CPU workers gone for good?

I noticed when selecting CPU workers for serverless endpoints that we are no longer given the option of the 64 or 128 vCPUs anymore. I know the 64/128 vCPUs were having issues running jobs. I am wondering if they are going to come back or are they gone for good? Thanks! 🙂

5 replies

RRunPod

•Created by Encyrption on 7/26/2024 in #⚡｜serverless

Is privileged mode possible?

I have an application that requires a kernel module be loaded. For an image to add a kernel modules requires privileged mode from the host. Is there anyway to get privileged mode enabled on my images so that I can add a kernel module to it?

2 replies

RRunPod

•Created by Encyrption on 6/12/2024 in #⚡｜serverless

Can serverless endpoints make outbound TCP connections?

I know endpoints can make http/https requests but is there any limit on outbound connections? Is there a FW or are all ports open? What about bandwidth limitations, etc.? Thanks!

15 replies

RRunPod

•Created by Encyrption on 6/10/2024 in #⚡｜serverless

Anyone have example template for OpenVoice V2 serverless?

I would like to deploy https://github.com/myshell-ai/OpenVoice on serverless. It has a huggingface module can it be implement in the Serverless vLLM quick deploy? If so are there any instructions for doing so? If not what are my options for getting it installed?

11 replies

RRunPod

•Created by Encyrption on 6/8/2024 in #⚡｜serverless

CPU Instances on 64 / 128 vCPUs FAIL

I can deploy my app on all instances except for 64 & 128 vCPU. Both of these run on AMD EPYC 9754 128-Core Processor. When it tries to run it gets stuck in QUEUE with the error (pasted below). When this happens it then just loops between "start container" and "failed to create shim task: the file python was not found: unknown". Any ideas what is causing this and how to resolve? There is similar issue reported in pods section here but I am using serverless and getting same problem. ERROR from instance: error creating container: container: create: Post "http://%2Fvar%2Frun%2Fdocker.sock/v1.43/containers/03f5da1a67e9f72498f779b9923cb7927a703cc84d173fa038041e72a7caac9b/start": context deadline exceeded

19 replies

Gaming

Programming