Coderik
Coderik
PostsComments
RRunPod
Created by Coderik on 9/12/2024 in #⚡｜serverless
TTL for vLLM endpoint
Is there a way to specify TTL value when calling a vLLM endpoint via OpenAI-compatible API?
13 replies