"IN QUEUE" and nothing happeneds
Hello everyone,
I'm currently running a TGI container (ghcr.io/huggingface/text-generation-inference:2.2.0) within a serverless environment, alongside my model from Hugging Face.
Issue: Although the status indicates "connected," there seems to be no further activity. The logs display various INFO and WARNING messages but do not show any errors. This has left me puzzled as to the root cause of the problem.
I have attached the relevant logs and screenshots for reference.
Any insights or suggestions to troubleshoot this issue would be greatly appreciated.
5 Replies
No that's not how serverless works
Look at the docs, serverless needs some kind of layer or python function that you need to call and that function has an argument of your callback which will handle each of the job
@nerdylive sorry, but even i do curl -X POST https://www.runpod.io/console/serverless/user/endpoint/jhfkefjljllckt -H 'Content-Type: application/json' -H 'Authorization: Bearer xxxxx' -d '{"input": {"prompt": "cancer is:"}}' , it is just in queue and nothing happends.
what exactly should i find in the documentation? can you explain please
Can you provide source code of your handler? You can identify the name of your handler by looking for a line like this in your Python code:
In this example handler in the name of handler so in your code look for code similar to this:
#Handler function that will be used to process jobs.
If you can provide this code it may shed light on your issue. Also, this is a good place to add print commands, to print out debug info. This debug info will appear in your logs.
@Encyrption dear friend, you have just explained everything I need. Indeed I have forgotten handler, silly me. now everything works perfectly! Thank you!
Great news! 🙂