topz
RRunPod
•Created by topz on 11/11/2024 in #⚡｜serverless
qwen2.5 vllm openwebui
I have deployed qwen2.5-7b-instruct using the vLLM quick deploy template (0.6.2). But when using openwebui connected by the OpenAI API the runpod workers log these errors:
"code": 400,
"message": "1 validation error for ChatCompletionRequest\nmax_completion_tokens\n Extra inputs are not permitted [type=extra_forbidden, input_value=50, input_type=int]\n For further information visit https://errors.pydantic.dev/2.9/v/extra_forbidden",
"object": "error",
"param": null,
"type": "BadRequestError"
It reads like its related to the chat template but i sadly failed hard to get it to work. I'd appreciate help a lot!
It reads like its related to the chat template but i sadly failed hard to get it to work. I'd appreciate help a lot!
1 replies