fanbing
RRunPod
•Created by fanbing on 3/26/2024 in #⚡｜serverless
Why all GPU Unavailable when "runpodctl project dev", a40 is available in runpod deploy page?
4 replies
RRunPod
•Created by fanbing on 3/20/2024 in #⚡｜serverless
Serverless prod cannot import name "ControlNetModel"
6 replies
RRunPod
•Created by fanbing on 3/13/2024 in #⚡｜serverless
base_image in dockerless serverless
when "runpodctl project dev", use runpod/pytorch:2.0.1-py3.10-cuda11.8.0-devel-ubuntu22.04 as base_image, log "apt not installed". When I test helloworld project using default runpod/base:0.6.1-cuda11.8.0, is ok .
9 replies
RRunPod
•Created by fanbing on 3/11/2024 in #⚡｜serverless
venv isolation in network volume
For venv isolation in my project, which consists of multiple services, with each service corresponding to a serverless worker, and each worker being built using a different subproject, should the venv be deployed within the subproject to prevent dependency conflicts between different subprojects?
3 replies
RRunPod
•Created by fanbing on 3/10/2024 in #⚡｜serverless
serverless endpoint, Jobs always 1 in queued, even 3 workers running
26 replies
Are there very few GPUs that support CUDA 11.8?
When I create a GPU Pod on Secure Cloud, if I select the CUDA 11.8 version, there are very few GPUs available. However, when I choose 'any', there are many more GPUs available for deployment. My project currently requires the use of CUDA 11.8.
4 replies
RRunPod
•Created by fanbing on 3/8/2024 in #⚡｜serverless
connection closed by remote host
Connection to 69.30.85.26 closed by remote host.
rsync: [sender] write error: Broken pipe (32)
rsync error: unexplained error (code 255) at io.c(848) [sender=3.2.7]
Activating Python virtual environment /7bb8882d/venv on Pod 9h5hmefddu8msr
Creating Project watcher...
4 replies
RRunPod
•Created by fanbing on 3/8/2024 in #⚡｜serverless
When using runpodctl project dev to upload a project, is there a speed limit?
From my tests, the speed seems to be around 1.2MB per second. When uploading for the first time, the various large models combined exceed 10GB, which takes a considerable amount of time to complete.
11 replies
RRunPod
•Created by fanbing on 3/8/2024 in #⚡｜serverless
Why serverless endpoint download sdxl1.0 from hugging Face hub so slow?
7 replies
RRunPod
•Created by fanbing on 3/7/2024 in #⚡｜serverless
Dockerless dev and deploy, async handler need to use async ?
handler.py in HelloWorld project, there is not 'async' before def handler(job): . But in serverless endpoint, there are Run and RunSync . So if I want to use async handler, Is it necessary to go like this ? Or am I misunderstanding something?
async def async_generator_handler(job):
for i in range(5):
output = f"Generated async token output {i}" yield output for a large language model await asyncio.sleep(1) runpod.serverless.start( { "handler": async_generator_handler,
"return_aggregate_stream": True, } )
output = f"Generated async token output {i}" yield output for a large language model await asyncio.sleep(1) runpod.serverless.start( { "handler": async_generator_handler,
"return_aggregate_stream": True, } )
5 replies
RRunPod
•Created by fanbing on 3/7/2024 in #⚡｜serverless
Should I use Data Centers or Network Volume when confige serverless endpoint ?
My project is an AI portrait app targeting global users. The advantage of using data centers is the ability to utilize GPUs from all data centers, while network volumes can speed up model loading times，However, GPU usage is limited to the data center where the network volume is located. How should I choose?
4 replies
RRunPod
•Created by fanbing on 3/3/2024 in #⚡｜serverless
Dockerless CLI can not sync local files to runpod server
when i try use dockerless following runpod blog feb 2 2024, i config and create project sucess on local ,but when i start a development session useing "runpodctl project dev", ERROR: Could not open requirements file: [Errno 2] No such file or directory: 'builder/requirements.txt' . cmd file is "C:\Users\Administrator\test-project>" .please tell why .
log info :
Waiting for Pod to come online... Project test-project Pod (npia2rx81eq4wp) created.
Checking remote project folder: /runpod-volume/68b7482a/dev/test-project on Pod npia2rx81eq4wp
Syncing files to Pod npia2rx81eq4wp
Activating Python virtual environment /68b7482a/venv on Pod npia2rx81eq4wp
[npia2rx81eq4wp] Retrieving existing venv from network volume...
[npia2rx81eq4wp] Requirement already satisfied: pip in /68b7482a/venv/lib/python3.10/site-packages (24.0)
[npia2rx81eq4wp] Using pip 24.0 from /68b7482a/venv/lib/python3.10/site-packages/pip (python 3.10)
[npia2rx81eq4wp] ERROR: Could not open requirements file: [Errno 2] No such file or directory: 'builder/requirements.txt'
Creating Project watcher...
Starting project endpoint...
[npia2rx81eq4wp] - Activated project environment.
[npia2rx81eq4wp] cat: /installreport.json: No such file or directory
[npia2rx81eq4wp] - Changed to project directory.
[npia2rx81eq4wp] cp: cannot stat '.runpodignore': No such file or directory
[npia2rx81eq4wp] - Started API server with PID: 178
[npia2rx81eq4wp] python: can't open file '/runpod-volume/68b7482a/dev/test-project/src/handler.py': [Errno 2] No such file or directory
[npia2rx81eq4wp]
[npia2rx81eq4wp] Connect to the API server at:
[npia2rx81eq4wp] > https://npia2rx81eq4wp-7270.proxy.runpod.net
[npia2rx81eq4wp]
[npia2rx81eq4wp] Synced venv to network volume
18 replies