CoverGhoul
RRunPod
•Created by CoverGhoul on 2/2/2025 in #⚡｜serverless
openai/v1 and open-webui
https://meyer-laurent.com/deploying-deepseek-r1-on-runpod-serverless-and-use-it-in-pycharm#2-understanding-vllm
This is the answer, apparently it won't load the models without write permissions. Seems a bit silly, but hopefully this helps someone else
2 replies