How do I select a different template to the default in the new RunPod UI?

I might be missing something obvious, but: in the new RunPod Pods > Deploy UI, after selecting a GPU config, how do I pick a template other than the default? RunPod Pytorch 2.2.10 runpod/pytorch:2.2.0-py3.10-cuda12.1.1-devel-ubuntu22.04 There used to be a way to search for other provided templates, but I can't figure out how to do that in the new UI. Thanks!
26 Replies
BelieveDiffusion
BelieveDiffusionOP8mo ago
Huh… so, I'd selected H100 80GB SXM5 as the GPU config when I posted the message above. I just spotted that if I select (say) L40 or A40 instead, I see a "change template" option that isn't visible with the H100 config. Are alternate templates not available for H100 GPUs now?
digigoblin
digigoblin8mo ago
I believe its because torch is compiled specifically for H100.
BelieveDiffusion
BelieveDiffusionOP8mo ago
Ah, I see. Thank you!
Alseki
Alseki8mo ago
Not that I'm an expert, but I think its an error. I used to be able to run other templates on the H100. And then for a chunk of today no non-torch templates were loading at all for any GPU's. Now it looks like they have fixed it for most, just not the H100 yet [unless this is now a deliberate change]
kaj
kaj8mo ago
you should be able to change templates on h100s now
Alseki
Alseki8mo ago
Ideal. Thanks 🙂
Nubby
Nubby8mo ago
Hi, it seems that i have a problem with the templates too. Whenever i pick a template, let's say Koboldccp official, all arguments inside are completely blank, which defeats the purpose of a template. Is there anything i could to to fix it? New ui seems to have broken it.
digigoblin
digigoblin8mo ago
Ask @Henky!! if he knows about this.
Henky!!
Henky!!8mo ago
I have heard of this yes
Nubby
Nubby8mo ago
I've already talked with him
Henky!!
Henky!!8mo ago
But its a runpod bug, I don't know whats causing it
Nubby
Nubby8mo ago
i will just go around and run pytorch instead and apply the notebook
Henky!!
Henky!!8mo ago
It happens before my template is even running
digigoblin
digigoblin8mo ago
Probably similar issue to volume disk setting getting lost
Henky!!
Henky!!8mo ago
Koboldcpp has no volume disk so thats normal Or the persistent disk I mean
Nubby
Nubby8mo ago
New ui requires us to pick a GPU and THEN pick the template, so the link wtih official kobold ccp is non-functional, but there's an additional bug that makes all values in the config blank
digigoblin
digigoblin8mo ago
yeah but other templates had issues with it being lost, so I think the issue may be related
Henky!!
Henky!!8mo ago
If I do it on my end I can pick the GPU and the Koboldcpp template will be prefilled with the correct values Thats why its hard for me to know how to help others with it being blank
kaj
kaj8mo ago
a fix for this issue was deployed earlier, are you still seeing it now?
digigoblin
digigoblin8mo ago
Looks like its working for me: * last used template is remembered * volume disk not lost * environment variables not lost
Nubby
Nubby8mo ago
if i could stream it it would be easy for you to see, but the moment i pick a template, and attempt to change the values, everything is blank
kaj
kaj8mo ago
there was a hotfix for the former issue as well when was the last time you attempted this?
Nubby
Nubby8mo ago
5 minutes ago, and now i feel like a moron because it just got fixed you guys are the best
kaj
kaj8mo ago
no worries, the feedback is helpful
Nubby
Nubby8mo ago
Praise the Omnissaiah. 🙌
BelieveDiffusion
BelieveDiffusionOP8mo ago
Thanks for the updates and fixes, @zeke!
Want results from more Discord servers?
Add your server