justin
justin
PostsComments
RRunPod
Created by blistick on 1/5/2024 in #⚡｜serverless
What does "throttled" mean?
(To my understanding - as a community helper)
16 replies
RRunPod
Created by blistick on 1/5/2024 in #⚡｜serverless
What does "throttled" mean?
Yes workers are bound to a specific region. Essentially workers are as if u spun up a pod in a data center, so adding more workers allows runpod to put more workers on standby meeting ur region criteria / gpu criteria, and gets pegged to ur app unless some sort of rotation rule comes in to rotate them out
16 replies
RRunPod
Created by TheWitcher.... on 10/26/2024 in #⚡｜serverless
How to send an image as a prompt to vLLM?
So before sending it in json encode as base64, then decode on the other end. as long the image is not too big
5 replies
RRunPod
Created by TheWitcher.... on 10/26/2024 in #⚡｜serverless
How to send an image as a prompt to vLLM?
Need to base64 encode + decode it
5 replies
RRunPod
Created by lunarlander on 9/7/2024 in #⛅｜pods
Trying to install Ollama: 'could not resolve host'
maybe madiator's template would work?
5 replies
RRunPod
Created by lunarlander on 9/7/2024 in #⛅｜pods
Trying to install Ollama: 'could not resolve host'
Have u tried this? just curious
5 replies
RRunPod
Created by lunarlander on 9/7/2024 in #⛅｜pods
Trying to install Ollama: 'could not resolve host'
https://discord.com/channels/912829806415085598/1261031850651029524
5 replies
RRunPod
Created by Gustavo Monti on 8/28/2024 in #⛅｜pods
How to override ollama/ollama image to run a model at startup
I personally couldn't ever figure out how to bake an ollama model into my template
6 replies
RRunPod
Created by Gustavo Monti on 8/28/2024 in #⛅｜pods
How to override ollama/ollama image to run a model at startup
2) Uh maybe a bash script where it starts the server in the background on startup and starts the download of the model, or change the path of where it checks to ur network volume.
6 replies
RRunPod
Created by Gustavo Monti on 8/28/2024 in #⛅｜pods
How to override ollama/ollama image to run a model at startup
Just b/c one, using Ollama seems to require some configurations to get it workign with GPU properly
6 replies
RRunPod
Created by Gustavo Monti on 8/28/2024 in #⛅｜pods
How to override ollama/ollama image to run a model at startup
I recommend to look here
6 replies
RRunPod
Created by Gustavo Monti on 8/28/2024 in #⛅｜pods
How to override ollama/ollama image to run a model at startup
https://discord.com/channels/912829806415085598/1261031850651029524
6 replies
RRunPod
Created by Shaplow on 8/28/2024 in #⛅｜pods
How to send request to a pod ?
No description
29 replies
RRunPod
Created by Shaplow on 8/28/2024 in #⛅｜pods
How to send request to a pod ?
No description
29 replies
RRunPod
Created by Shaplow on 8/28/2024 in #⛅｜pods
How to send request to a pod ?
**make sure that the runpod by edit template, has the port exposed on port 5000
29 replies
RRunPod
Created by Shaplow on 8/28/2024 in #⛅｜pods
How to send request to a pod ?
from flask import Flask

app = Flask(__name__)

@app.route('/hello', methods=['GET'])
def hello_world():
    return {"text": "hello"}

if __name__ == '__main__':
    app.run(debug=True, host='0.0.0.0', port=5000)
from flask import Flask

app = Flask(__name__)

@app.route('/hello', methods=['GET'])
def hello_world():
    return {"text": "hello"}

if __name__ == '__main__':
    app.run(debug=True, host='0.0.0.0', port=5000)
29 replies
RRunPod
Created by Shaplow on 8/28/2024 in #⛅｜pods
How to send request to a pod ?
Okay step by step manually at least: I create a runpod template on: runpod/pytorch:2.1.0-py3.10-cuda11.8.0-devel-ubuntu22.04 Create a virtualEnv: python3 -m venv venv source venv/bin/activate pip install flask Run the flask app: python main.py
29 replies
RRunPod
Created by Shaplow on 8/28/2024 in #⛅｜pods
How to send request to a pod ?
maybe someone else can help on the exposing ports programatically, i saw ur thing here
29 replies
RRunPod
Created by Shaplow on 8/28/2024 in #⛅｜pods
How to send request to a pod ?
also not too sure about dockerfile port exposing, i tend to do it through the console. so maybe this is where we are diverging
29 replies
RRunPod
Created by Shaplow on 8/28/2024 in #⛅｜pods
How to send request to a pod ?
basically proxy is for http requests (what u wanna do) if u wanna do more, can do the public ip method
29 replies