ericmsilver
RRunPod
•Created by ericmsilver on 3/28/2025 in #⚡｜serverless
Rag on serverless LLM
More progress. Adding connection to webui to serverless worked. It now is able to use the serverless server to query the model.
6 replies
RRunPod
•Created by ericmsilver on 3/28/2025 in #⚡｜serverless
Rag on serverless LLM
So here is where I am at. I am looking to run webui in the cloud. I want to do rag via web UI. This works now by using an existing template in run pod. Knowledge works great so rag works as well. Ultimately I want to run webui on a dedicated machine but the end goal is to use a server less endpoint and connect to it from a webui server and make the call to the server less end point. I can connect to the endpoint in Python and works good. Does this concept make sense?
6 replies
RRunPod
•Created by ericmsilver on 3/28/2025 in #⚡｜serverless
Rag on serverless LLM
Is it possible to do?
6 replies