How do I estimate completion time (ETA) of a job request?
I want to show users when their images are going to be ready (e.g. "Your images will be ready in about 20 seconds. You are number 5 in the queue.") and it means I need to know the queue position of a job and average duration of jobs recently completed.
15 Replies
This is not possible
Hmm is there a plan for runpod to support this use case?
How will free users know when their job will be completed then?
You can use webhooks to know when the job is completed and notify the user, otherwise there is no other way to know this.
You can have something on your own backend to keep track of the the number in the queue.
I think the status api should give me that number because runpod knows about the queue more than external services
Especially when you'll implement the priority tag, it will get confusing quickly when outsiders count the position and have to increase the position for free users once some paid users enter the queue.
Many people use this in production and don't have a need for this. I don't know of a messaging queue type system that tracks the number in the queue, nobody else has requested this feature, they all manage to track it themselves.
There is something on the roadmap for queue priority though for free vs paid users.
The priority isn't available yet though
fal.ai reports queue position https://www.fal.ai/docs/model-endpoints/queue#request-status so there is a way to do it.
You might think that I should go and use fal.ai but I am already using Runpod and don't want to migrate.
I think that if users can manage to track the queue position externally, it should be possible to track internally.
Also how does others track it themselves? I'm new to web dev so maybe not aware of the design pattern.
Queue - fal docs
Build serverless AI apps.
Well too bad for you because as I keep telling you and you don't listen, RunPod DOES NOT OFFER THIS
@Chanchana you can use the /health endpoint for this https://docs.runpod.io/docs/serverless-endpoint-urls#health
RunPod
Endpoint URLs
These endpoints provide standard functionality for submitting jobs and retrieving the output from job requests. To use these endpoints, you will need to have your endpoint ID. The constructed URL will start with https://api.runpod.ai/v2/{endpoint_id} followed by one of the following:/run Used to sub...
I already know that it doesn't offer from what you are saying. I'm asking whether runpod will implement this feature and you response is like"no because we don't know how to do it and there's probably no way to do it" so i show you that there's a someone already doing it. So you guys do not want to implement it?
I don't work for RunPod, its just pointless, nobody wants this besides you because you are too lazy or stupid it implement it yourself.
The health endpoint seems to return numbers of all jobs, but not queue position of a particular job id, right?
If you are so smart why don't you suggest a way to implement it instead of being arrogant?
Also the way you respond is like you own the company or something.
And you are speaking for other customers without asking them, this is bad take.
Track it in your code, as @River Snow said, you can use the /health endpoint to get number of requests in queue after sending a new request, to get the position in the queue, but doesn't seem like you even bothered to try it, you are just approaching the problem with negativity and assuming it cannot be done. When you have a negative attitude towards a problem, you will never find a solution.
I'm not assuming it cannot be done this is why I'm asking for a way to do it in the first place. I simply know that if it can be done external ly it can be done internally
This is A COMMUNITY, ANYONE is free to answer things here NOT JUST RUNPOD staff, if you dont want the community to get involved, then go use the live chat or something and remove yourself from the community.
You are like an annoying cockroach, just try the suggested solution and stop being an asshole.
Ok so you think that community members can assume that no one wants a certain feature? Community members can lash on someone's request?
If you open a post requesting a feature, and someone tell you that no one wants it, you are problematic, you would like it?
I'll have to count the number of jobs in front of job X to know where it is in the queue. It requires tracking position in my queue. I'll check if there's a way to check for queue position in the background task service that I use.
But yes if it's possible it should be possible for runpod to track it too because they are both queue.
And the reason this should be tracked in runpod is because the priority queue feature will work with this queue position. If paid members request jobs, they will increase queue position for free members.
OK I think I know rough solution to tackle it.
One way I'll solve this for now is by freezing the number of seconds e.g. when users submit the job, I check /health endpoint, and see there are 5 jobs then I assume the new job is at queue position 6 and estimate time of completion around 6X where X is the duration of execution. For example, if it takes 5 seconds per job, then the estimated time will be 30 seconds.
And from that I do not hit /health endpoint again, I just count down from 30 to 0 on the frontend. It will be a frozen timer that isn't updated but it's good enough.
I was thinking of how to constantly update the timer by hitting /health frequently but it's not needed