Performance Difference between machine u3q0zswsna6v88 and cizgr1kbbfrp04
Hey all, what are the difference between these two machines? For the exact same code, u3q0zswsna6v88 takes 60s and cizgr1kbbfrp04 takes 8s. I repeat the same request multiple times and none of these requests hit cold start. Happened around 5:15pm today.
Thanks!
9 Replies
You probably have flash boot enabled and the one that took 8s was already warmed up while the one that took 60s still had to load the model etc from a cold start.
I checked the logs. None of them hit cold start and all the models are loaded.
The same input for all these requests, but the execution time is very different...
Here are more tests:
request 09ab4d26 hit cold starts and it takes 30s
request daf19ba4 no need cold start, finish in 8.61s
request 2ad5f3b6 cold start again
request 5675044f cold start on d1nlfjprou0nsj, and it is so much slower
request 8a189d57 no need cold start, finish in 8.61s
request dec2ad75 no need cold start, finish in 44.73s
im very confused about these two workers: d1nlfjprou0nsj and gnlbiyyjbgaca7
they are all the same input and code...
@Justin can you point me to how I can debug this? Thank you so much!
What type of work is being performed? Is it CPU intensive? Did you also confirm that the works are both using the same GPU type?
it's SDXL inference with custom comfy workflow.
I only selected 80GB in the list and it does print out the same A100 type
are these workers only performing my handlers? I mean are the CPUs also shared with others?
CPU is dedicated correct, however the type of CPU between each worker might be slightly diffrent.