HF_TOKEN question
Hello everyone, i am trying to launch TGI, so I get the error of 'Cannot access gated repo for url https://huggingface.co/google/MYMODEL'
But i have my token as a variable, do i need to add it somewhere else? token is valid, i have doublechecked
18 Replies
Looks fine, have you actually been granted access to that model?
yes, this is the model we have trained in my company)
somehow it launches just from VS code, but not here)
token is same
I suggest logging a Github issue since its a huggingface image and not a RunPod one:
https://github.com/huggingface/text-generation-inference
GitHub
GitHub - huggingface/text-generation-inference: Large Language Mode...
Large Language Model Text Generation Inference. Contribute to huggingface/text-generation-inference development by creating an account on GitHub.
check if the model pulling has access to your token ( in code somewhere in that docker image )
It has access to it, people run it with docker run -e HF_TOKEN=.... which is the same as @annasuhstuff was doing by adding it to the pod.
oh then usually its because your token hasn't been granted access to the model
He said it has, hence why I said to log a Github issue, not sure why you must always add messages to threads that add no value when someone has already answered the person.
Nah, why do you keep assuming my messages has no value when someone has answered? it works for me btw the same image, same setup ( different model )
there's no need to say that my answer doesn't contribute, marcus.. im just trying to help thats all
Well you're telling them to check their token when they already said the same token works fine for other things etc, which is not useful.
And you literally repeated my message.
Well check again then, mine works for gated models using the same docker tag, so they doesn't need to report a bug that doesn't exist
I thought I told you here that I've tried that and it works well
Well it doesn't work for them, does not matter that it works for you
Oh cmon.. Its using the same code, same docker image, same env setup and gated model
Your "cmon" attitude is highly unacceptable
Don't know how someone like you becomes a community leader with such a bad attitude
right
anna, try check if the token has the access to the model repo in hf
' anna, try check if the token has the access to the model repo in hf' --
well, as I said in the begining, if I run it without runpod, but logging with token '1234', and it WORKS, it means my token '1234' is valid
but when I put '1234' in Runpod, it somehow breaks down and ends with an error...
anyways, I have fixed it just by refreshing the page and inputing this token again)
But indeed, my friend @nerdylive , it was kinda meaningless to check the token the third of time fourth time as you suggested)
But I really don't think that you @nerdylive are a bad community helper, because statistics says that people just do the most common mistakes and forget to check the basics)
Oh that's weird behavior of hf, have you found the cause of this ?
I'm trying on the same env, image as yours and it worked on gated repo that I've been granted access to
All ENV variables are strings. Is your code expecting a string or number? If it is expecting a number you may have to update the code to convert it into a number before using it.
i think its hf's image its standard, so i've tried using the same config as descripted above inputted token without ""