Can worker-vllm work with https://huggingface.co/TheBloke/Wizard-Vicuna-30B-Uncensored-GPTQ
I have customers that want to use this and I think worker-vllm is the way to go with it. Hope it can work with worker-vllm! Link is https://huggingface.co/TheBloke/Wizard-Vicuna-30B-Uncensored-GPTQ and I guess it works with text-generation-webui so why not worker-vllm
0 Replies