PhysSci
50/50 success with running a standart vllm template
@yhlong00000 Hi, sorry for not answering. When I read you massge I already deleted the pod. I finally got back to developing my app so I started pod once again on 4xA40 which woked fine before, and now I can't do it with that do. I am sure that mu settings are the same cause I use python script to launch it with runpod api. Pod id j7tmr0531ylfd6. I can keep it running for a while so you can check. Here are the logs:
10 replies
50/50 success with running a standart vllm template
Why then I often can start it fine with this parameters? To use this in commercial application I need consistancy, either it works and I can use it or it not and I can troubleshoot this. I use runpod in my job and I almost always stick to it for prototyping, but I can not suggest it to my clients for production because such issues do occur occasionally. That's a pity cause I like runpod.
10 replies