Minozar
Minozar
PostsComments
RRunPod
Created by Minozar on 6/4/2024 in #⚡｜serverless
More than 5 workers
Question, how can I setup a billing limit / threshold?
15 replies
RRunPod
Created by Minozar on 6/4/2024 in #⚡｜serverless
More than 5 workers
Thanks!
15 replies
RRunPod
Created by Minozar on 6/4/2024 in #⚡｜serverless
More than 5 workers
All good!
15 replies
RRunPod
Created by Minozar on 6/4/2024 in #⚡｜serverless
More than 5 workers
Oh sorry, I thought I had already click on it
15 replies
RRunPod
Created by Minozar on 6/4/2024 in #⚡｜serverless
More than 5 workers
@Madiator2011
15 replies
RRunPod
Created by Minozar on 6/4/2024 in #⚡｜serverless
More than 5 workers
Is it possible to go beyond 20, up to 30 ?
15 replies
RRunPod
Created by Minozar on 6/4/2024 in #⚡｜serverless
More than 5 workers
I'm stuck with 20
15 replies
RRunPod
Created by Minozar on 6/4/2024 in #⚡｜serverless
More than 5 workers
What about worker concurrency ?
15 replies
RRunPod
Created by Minozar on 6/4/2024 in #⚡｜serverless
More than 5 workers
Perfect thanks !
15 replies
RRunPod
Created by Minozar on 5/17/2024 in #⚡｜serverless
Delay Time
I don't understand this part, that's the opposite of what shown on the repo
61 replies
RRunPod
Created by Minozar on 5/17/2024 in #⚡｜serverless
Delay Time
what's wrong with this dockerfile?
61 replies
RRunPod
Created by Minozar on 5/17/2024 in #⚡｜serverless
Delay Time
well I'm even more confused now... that's on the official repo
61 replies
RRunPod
Created by Minozar on 5/17/2024 in #⚡｜serverless
Delay Time
------------- I think this is what I was looking for 
# Cache Models
COPY builder/cache_models.py /cache_models.py
RUN python3.11 /cache_models.py && \
    rm /cache_models.py
# Cache Models
COPY builder/cache_models.py /cache_models.py
RUN python3.11 /cache_models.py && \
    rm /cache_models.py
61 replies
RRunPod
Created by Minozar on 5/17/2024 in #⚡｜serverless
Delay Time
# Base image
FROM runpod/base:0.4.2-cuda11.8.0

ENV HF_HUB_ENABLE_HF_TRANSFER=0

# Install Python dependencies (Worker Template)
COPY builder/requirements.txt /requirements.txt
RUN python3.11 -m pip install --upgrade pip && \
    python3.11 -m pip install --upgrade -r /requirements.txt --no-cache-dir && \
    rm /requirements.txt

# Cache Models
COPY builder/cache_models.py /cache_models.py
RUN python3.11 /cache_models.py && \
    rm /cache_models.py

# Add src files (Worker Template)
ADD src .

CMD python3.11 -u /rp_handler.py
# Base image
FROM runpod/base:0.4.2-cuda11.8.0

ENV HF_HUB_ENABLE_HF_TRANSFER=0

# Install Python dependencies (Worker Template)
COPY builder/requirements.txt /requirements.txt
RUN python3.11 -m pip install --upgrade pip && \
    python3.11 -m pip install --upgrade -r /requirements.txt --no-cache-dir && \
    rm /requirements.txt

# Cache Models
COPY builder/cache_models.py /cache_models.py
RUN python3.11 /cache_models.py && \
    rm /cache_models.py

# Add src files (Worker Template)
ADD src .

CMD python3.11 -u /rp_handler.py
61 replies
RRunPod
Created by Minozar on 5/17/2024 in #⚡｜serverless
Delay Time
ok update, I just checked the dockerfile of the project you shared
61 replies
RRunPod
Created by Minozar on 5/17/2024 in #⚡｜serverless
Delay Time
but this is only to load the model in VRAM, first I'd need to download it from somewhere after reading the documentation I understand it has to be in from the cache of the image, so it has to be linked with the dockerfile somehow
61 replies
RRunPod
Created by Minozar on 5/17/2024 in #⚡｜serverless
Delay Time
If I have no active worker, each time I'll spawn a new one, It'll dowload the model, that's not what I want, I'd like the model to be cached in the docker image so when I spawn a new worker, the model is already almost ready to use
61 replies
RRunPod
Created by Minozar on 5/17/2024 in #⚡｜serverless
Delay Time
Yes but the documentation says "be sure to cache them into your docker image." How to do that correctly? (the doc doesn't provide enough information I think)
61 replies
RRunPod
Created by Minozar on 5/17/2024 in #⚡｜serverless
Delay Time
So everything put outside of the handler function will be cached in the docker image??
61 replies
RRunPod
Created by Minozar on 5/17/2024 in #⚡｜serverless
Delay Time
import numpy as np

# Can I load the model here??

def handler(job):
    return True

runpod.serverless.start({"handler": handler})
import numpy as np

# Can I load the model here??

def handler(job):
    return True

runpod.serverless.start({"handler": handler})
Is that what you meant?
61 replies