RRunPod•Created by 🆁🅰🅻🅻🅴 on 7/9/2024 in #⛅｜pods TheBloke/goliath-120b-GPTQ with RunPod Kobold AI United
Hi! I got goliath-120b-GPTQ running with 3 A40. But the text generation speed is extremely slow. What is the best option for GPU config and settings to run this model?
Thank you in advance!