How to add python or API bindings for an vLLM?
I want my vLLM to be able to execute some Python code or call an API. How do I get it to do this?
4 Replies
hi
do you want like your LLM to be able to execute the python code it outputs?
No, something like "
"What kind of events do I have in my calendar?" ==> execute some python API to fetch the events
But I want to write to execute my own python code, not just some public API
Thats a vllm request, not for RunPod. Log a Github issue requesting it - https://github.com/vllm-project/vllm
GitHub
GitHub - vllm-project/vllm: A high-throughput and memory-efficient ...
A high-throughput and memory-efficient inference and serving engine for LLMs - vllm-project/vllm
Well yes this or just code your own thing for the llms to generate code and exec it
There is no built in vllm feature for that yet I think