Henky!! Comments - Answer Overflow

Topics

Henky!!

•Created by Netrve on 7/24/2024 in #⚡｜serverless

How does the vLLM template provide an OAI route?

No matter if its a exl2 webserver, kcpp webserver or ollama webserver it would work best that way

8 replies

•Created by Netrve on 7/24/2024 in #⚡｜serverless

How does the vLLM template provide an OAI route?

All we need is a 1:1 mapping of the webserver

8 replies

•Created by Netrve on 7/24/2024 in #⚡｜serverless

How does the vLLM template provide an OAI route?

We all need the same thing

8 replies

•Created by Netrve on 7/24/2024 in #⚡｜serverless

How does the vLLM template provide an OAI route?

Alright because the actual worker part i need would be insanely simplistic

8 replies

•Created by Netrve on 7/24/2024 in #⚡｜serverless

How does the vLLM template provide an OAI route?

+1 making a template for KCPP is pointless without this

8 replies

•Created by BelieveDiffusion on 4/10/2024 in #⛅｜pods

How do I select a different template to the default in the new RunPod UI?

Thats why its hard for me to know how to help others with it being blank

31 replies

•Created by BelieveDiffusion on 4/10/2024 in #⛅｜pods

How do I select a different template to the default in the new RunPod UI?

If I do it on my end I can pick the GPU and the Koboldcpp template will be prefilled with the correct values

31 replies

•Created by BelieveDiffusion on 4/10/2024 in #⛅｜pods

How do I select a different template to the default in the new RunPod UI?

Or the persistent disk I mean

31 replies

•Created by BelieveDiffusion on 4/10/2024 in #⛅｜pods

How do I select a different template to the default in the new RunPod UI?

Koboldcpp has no volume disk so thats normal

31 replies

•Created by BelieveDiffusion on 4/10/2024 in #⛅｜pods

How do I select a different template to the default in the new RunPod UI?

It happens before my template is even running

31 replies

•Created by BelieveDiffusion on 4/10/2024 in #⛅｜pods

How do I select a different template to the default in the new RunPod UI?

But its a runpod bug, I don't know whats causing it

31 replies

•Created by BelieveDiffusion on 4/10/2024 in #⛅｜pods

How do I select a different template to the default in the new RunPod UI?

I have heard of this yes

31 replies

•Created by Fenix on 3/21/2024 in #⛅｜pods

load a new network volumen into a pod?

The easiest way to do it with Koboldcpp is https://koboldai.org/runpodcpp it will allow you to specify the models in the runpod variables directly without the need for persistent or network storage.

3 replies

•Created by jibay on 1/8/2024 in #⛅｜pods

my pod start very slow

Feel free to join our own discord to hang out with other Kobold users and for faster support 😄

57 replies

•Created by jibay on 1/8/2024 in #⛅｜pods

my pod start very slow

On that screen find --remote and remove it. I just looked and cloudflare undid the update. So may happen again soon when they publish again.

57 replies

•Created by jibay on 1/8/2024 in #⛅｜pods

my pod start very slow

Nah, the easiest method is waiting two hours but when you start your runpod you have the ability to customize the template.

57 replies

•Created by jibay on 1/8/2024 in #⛅｜pods

my pod start very slow

If this specific cloudflare issue happens again the user can remove --remote from the args to stop the CF attempt. They luckily dont release binaries all that often but when they do the issue will be back for a few hours

57 replies

•Created by jibay on 1/8/2024 in #⛅｜pods

my pod start very slow

Also for your info the runpod template is supplied by our official docker so I do maintain it instead of runpod

57 replies

•Created by jibay on 1/8/2024 in #⛅｜pods

my pod start very slow

@justin We do still actively maintain it so if people have issues feel free to forward to https://koboldai.org/discord where more people can help in a support channel. Otherwise you have to hope I see the notification in time.

57 replies

•Created by jibay on 1/8/2024 in #⛅｜pods

my pod start very slow

It was stuck on trying to start a cloudflare tunnel because yesterday cloudflare released a new version and their CI always takes a few hours to release the new binary in the place we expect. So should have solved itself by now.

57 replies