Jason Comments - Answer Overflow

Topics

Jason

•Created by papy on 4/1/2025 in #⛅｜pods

Training AI with a RunPod GPU

I'd suggest learning concepts that will be used here before the basics before you do the real training or else it'll lead to unexpected problems

7 replies

•Created by papy on 4/1/2025 in #⛅｜pods

Training AI with a RunPod GPU

Usually they control the cloud GPU server ( like in runpod) from your computer then run the training app / process there

7 replies

•Created by papy on 4/1/2025 in #⛅｜pods

Training AI with a RunPod GPU

You cant import cloud GPU (technically possible to connect but not for this usage) into your computer

7 replies

•Created by papy on 4/1/2025 in #⛅｜pods

Training AI with a RunPod GPU

Then import your dataset into runpod pod's storage

7 replies

•Created by papy on 4/1/2025 in #⛅｜pods

Training AI with a RunPod GPU

You basically need a. Training code for that model

7 replies

•Created by Yebs on 3/17/2025 in #⚡｜serverless

Can you now run gemma 3 in the vllm container?

No description

25 replies

•Created by Yebs on 3/17/2025 in #⚡｜serverless

Can you now run gemma 3 in the vllm container?

No description

25 replies

•Created by Yebs on 3/17/2025 in #⚡｜serverless

Can you now run gemma 3 in the vllm container?

you need access from hf + hf token to access it

25 replies

•Created by Yebs on 3/17/2025 in #⚡｜serverless

Can you now run gemma 3 in the vllm container?

just now i've tried for you!

25 replies

•Created by Yebs on 3/17/2025 in #⚡｜serverless

Can you now run gemma 3 in the vllm container?

Yes it works

25 replies

•Created by Eren on 3/27/2025 in #⚡｜serverless

Meaning of -u1 -u2 at the end of request id?

have you got an answer to this maybe from a support ticket?

3 replies

•Created by Realised Prophets on 3/15/2025 in #⚡｜serverless

Faster-Whisper output "None" — log 400 "Bed request"

No description

11 replies

•Created by wrichert on 3/30/2025 in #⚡｜serverless

Hi, I'm new to runpod and try to debug this error

if you try, using the nomal image that is attached in the readme.md (the one they've buildd) is it working?

6 replies

•Created by wrichert on 3/30/2025 in #⚡｜serverless

Hi, I'm new to runpod and try to debug this error

or there is some problems with the whisper repo's / its deps

6 replies

•Created by wrichert on 3/30/2025 in #⚡｜serverless

Hi, I'm new to runpod and try to debug this error

maybe, you could try it

6 replies

•Created by Yebs on 3/17/2025 in #⚡｜serverless

Can you now run gemma 3 in the vllm container?

already

25 replies

•Created by Yebs on 3/17/2025 in #⚡｜serverless

Can you now run gemma 3 in the vllm container?

yes i think vllm is updated

25 replies

•Created by ErezL on 3/30/2025 in #⚡｜serverless

Length of output of serverless meta-llama/Llama-3.1-8B-Instruct

https://github.com/runpod-workers/worker-vllm/?tab=readme-ov-file#sampling-parameters need to expand these 2 for reference

6 replies

•Created by ErezL on 3/30/2025 in #⚡｜serverless

Length of output of serverless meta-llama/Llama-3.1-8B-Instruct

{
  "input": {
    "messages": [
      {
        "role": "system",
        "content": "Your are an ai assistant."
      },
      {
        "role": "user",
        "content": "Explain llm models"
      }
    ],
"sampling_params": {
    "max_tokens": 3000,
    "temperature": 0.7,
    "top_p": 0.95,
    "n": 1,
    "stream": false,
    "stop": [],
    "presence_penalty": 0,
    "frequency_penalty": 0,
    "logit_bias": {},
    "best_of": 1
}
  }
}

{
  "input": {
    "messages": [
      {
        "role": "system",
        "content": "Your are an ai assistant."
      },
      {
        "role": "user",
        "content": "Explain llm models"
      }
    ],
"sampling_params": {
    "max_tokens": 3000,
    "temperature": 0.7,
    "top_p": 0.95,
    "n": 1,
    "stream": false,
    "stop": [],
    "presence_penalty": 0,
    "frequency_penalty": 0,
    "logit_bias": {},
    "best_of": 1
}
  }
}

6 replies

•Created by ErezL on 3/30/2025 in #⚡｜serverless

Length of output of serverless meta-llama/Llama-3.1-8B-Instruct

if you don't want, change your request to this

6 replies