How to avoid Cloudflare timeouts on pods?
I saw a previous post mentionning using the public IP but it doesn't seem to work for me?
I'm using runpod to host a vLLM server (the serverless endpoint doesn't work for me).
I'm running batch workloads and those timeout (cloudflare)
How to avoid that please?
5 Replies
Use tcp or ssh tunnel
Understood for ssh tunnel, but I don't understand for tcp, would it still accept https connection?
i don't think so but you can try it
serverless doesn't work? why? maybe if you can post the problem i can try to help
I tried with the NousHermes model llama 3.1 8b and it doesn't work out of the box
the requests hang for ever
no it doesn't hang forever, i loaded 70b model in like 5mins ish
then the next ones are faster
maybe check logs