Visionscaper
RRunPod
•Created by Visionscaper on 12/11/2024 in #⚡｜serverless
Is VLLM Automatic Prefix Caching enabled by default?
Hello!
I setup a Serverless quick deployment for text generation and I was wondering if VLLM Automatic Prefix Caching is enabled by default? Also see:
https://docs.vllm.ai/en/latest/automatic_prefix_caching/apc.html
Kind regards,
-- Freddy
5 replies