ACiDGRiM
RRunPod
•Created by ACiDGRiM on 3/7/2024 in #⚡｜serverless
secure connections
I want to ensure all traffic between my app and the server less backend is encrypted. Does the endpoint decrypt the traffic from the internet and transmit in plaintext to the server less container?
Specifically, is the data in my prompt In clear text even in memory before it reaches the container?
20 replies
RRunPod
•Created by ACiDGRiM on 3/7/2024 in #⚡｜serverless
server less capability check
I want to add runpod into a tier of load balanced llm models behind an app like openrouter.ai, but the decision will occur in our infrastructure. When i invoke a server less instance with my app and a task is completed, how am I billed for idle time if the container unloads the model from gpu memory?
In other words I want to reduce costs and increase performance by only needing to load the model after an idle timeout, paying only for the small app footprint in storage/memory
7 replies
RRunPod
•Created by ACiDGRiM on 1/12/2024 in #⚡｜serverless
Log retention and privacy
I'm weighing the cost benefit of cloud GPU for AI inference tasks and self-hosted vs privacy implications.
I want to be able to submit personally private information which I'm comfortable with submitting an API call to initialize a worker as long as it is ephemeral in RAM. However are the API calls logged to any medium that is persistent longer than the specific request is being processed?
5 replies