wiki Comments - Answer Overflow

wiki

•Created by Casper. on 1/25/2025 in #⚡｜serverless

worker-vllm not working with beam search

Sure. @Casper.

4 replies

•Created by Misterion on 12/11/2024 in #⚡｜serverless

vllm worker OpenAI stream timeout

Or it was as it is MODEL_NAME?

20 replies

•Created by Misterion on 12/11/2024 in #⚡｜serverless

vllm worker OpenAI stream timeout

Did you set model name ?

20 replies

•Created by artbred on 9/15/2024 in #⚡｜serverless

GGUF vllm

I will add a support for this natively

16 replies

•Created by marticztn on 9/18/2024 in #⚡｜serverless

Serverless vLLM deployment stuck at "Initializing" with no logs

What GPU did you use? It could be the case, where the selected GPU is not available.

5 replies

•Created by JHenriP on 2/26/2024 in #⚡｜serverless

Severless 404

Hey, It works fine. I tested.

21 replies

•Created by JHenriP on 2/26/2024 in #⚡｜serverless

Severless 404

Let me check & try to reproduce!

21 replies

•Created by JHenriP on 2/26/2024 in #⚡｜serverless

Severless 404

Can you share error? Setup? It should be helpful.

21 replies

•Created by JHenriP on 2/26/2024 in #⚡｜serverless

Severless 404

@JHenriP Are you still facing the issue with the worker ?

21 replies