Sandeep
RRunPod
•Created by Harish Natarajan on 4/30/2024 in #⚡｜serverless
Runpod doesn't work with GCP artifact registyr
@Harish Natarajan did you resolve the issue ?
7 replies
RRunPod
•Created by Untrack4d on 9/11/2024 in #⚡｜serverless
Training Flux Schnell on serverless
Use a more specific base image for efficiency
FROM runpod/base:0.6.2-cuda12.2.0
Set environment variables
ENV HF_HUB_ENABLE_HF_TRANSFER=0 \
PYTHONDONTWRITEBYTECODE=1 \
PYTHONUNBUFFERED=1 \
HF_HOME=/runpod-volume/huggingface-cache \
HUGGINGFACE_HUB_CACHE=/runpod-volume/huggingface-cache/hub \
WORKSPACE=/runpod-volume
RUN ls -a /
Create necessary directories
RUN mkdir -p ${WORKSPACE}/app ${HF_HOME}
Copy requirements first to leverage Docker cache for dependencies
COPY requirements.txt ${WORKSPACE}/
Install dependencies in a single RUN statement to reduce layers
RUN python3.11 -m pip install --no-cache-dir --upgrade pip && \
python3.11 -m pip install --no-cache-dir -r ${WORKSPACE}/requirements.txt && \
rm ${WORKSPACE}/requirements.txt
Copy source code to /runpod-volume/app
COPY test_input.json ${WORKSPACE}/app/
COPY src ${WORKSPACE}/app/src
Set the working directory
WORKDIR ${WORKSPACE}/app/src
Use the built-in handler script from the source
CMD ["python3.11", "-u", "runpod_handler.py"]
80 replies
RRunPod
•Created by Untrack4d on 9/11/2024 in #⚡｜serverless
Training Flux Schnell on serverless
80 replies
RRunPod
•Created by Untrack4d on 9/11/2024 in #⚡｜serverless
Training Flux Schnell on serverless
This system doesn't need datasets , it just use the models from huggingface , it will import models from huggingface and download the lora and will use that lora for inference
80 replies
RRunPod
•Created by Untrack4d on 9/11/2024 in #⚡｜serverless
Training Flux Schnell on serverless
What should be the container size
80 replies
RRunPod
•Created by Untrack4d on 9/11/2024 in #⚡｜serverless
Training Flux Schnell on serverless
Input is
Lora blob url
Modeltype
80 replies
RRunPod
•Created by Untrack4d on 9/11/2024 in #⚡｜serverless
Training Flux Schnell on serverless
I am using flux and sdxl models in this deployment,
When ever user sends flux lora request, I will generate of flux lora
Same applies to sdxl
80 replies
RRunPod
•Created by Untrack4d on 9/11/2024 in #⚡｜serverless
Training Flux Schnell on serverless
@navin_hariharan
80 replies
RRunPod
•Created by Untrack4d on 9/11/2024 in #⚡｜serverless
Training Flux Schnell on serverless
And may I know what is the inference time , it taking for an image to generate on A100 or any other gpus , for me its taking 15 seconds
,
80 replies
RRunPod
•Created by Untrack4d on 9/11/2024 in #⚡｜serverless
Training Flux Schnell on serverless
is this due the container size
80 replies