Is VLLM Automatic Prefix Caching enabled by default?
Hello!
I setup a Serverless quick deployment for text generation and I was wondering if VLLM Automatic Prefix Caching is enabled by default? Also see:
https://docs.vllm.ai/en/latest/automatic_prefix_caching/apc.html
Kind regards,
-- Freddy
4 Replies
under this.
enable this
i don't think its enabled by default since the default here is unchecked
Thanks @nerdylive, will look in to it!