Runpod•9mo ago

Serverless vLLM changing engine arguments

Hi, I got vLLM Serverless worker up and running, but want to change one engine argument (which is not overridable through environment variables), specifically

--limit-mm-per-prompt

--limit-mm-per-prompt

, how could I do that with your custom image

runpod/worker-v1-vllm:v2.3.0stable-cuda12.1.0

runpod/worker-v1-vllm:v2.3.0stable-cuda12.1.0

that endpoints use? Thanks

Jason•4/16/25, 1:19 AM

use t he configure button to see if its there

Jason•4/16/25, 1:19 AM

or use docs here https://github.com/runpod-workers/worker-vllm

GitHub

GitHub - runpod-workers/worker-vllm: The RunPod worker template for...

The RunPod worker template for serving our large language model endpoints. Powered by vLLM. - runpod-workers/worker-vllm

AleOP•4/16/25, 7:20 AM

thanks I'll look into it and report back

AleOP•4/21/25, 1:58 PM

@Jason https://github.com/runpod-workers/worker-vllm/issues/155 looks like support for it was implemented, and then someone accidentally removed it lol

Would it be possible to add it back again?

GitHub

setting limit_mm_per_prompt · Issue #155 · runpod-workers/worker-...

Is it possible to set limit_mm_per_prompt? there as a PR about it but this feature is not in main.