Sassy Pantsy Comments - Answer Overflow

Topics

Sassy Pantsy

•Created by Charixfox on 5/21/2024 in #⚡｜serverless

Speed up cold start on large models

oh that might actually be pretty good then

17 replies

•Created by Charixfox on 5/21/2024 in #⚡｜serverless

Speed up cold start on large models

Also, assuming I bake the image with my entire comfyui + nodes + models installation, I'd still need to copy it to a network storage during installation, no?

17 replies

•Created by Charixfox on 5/21/2024 in #⚡｜serverless

Speed up cold start on large models

How will baking the models into the docker image help reduce cold start time? The engine itself (in my case ComfyUI) loads its models from the same path anyway, which is on the network storage (because the comfyUI engine is on the network storage), so it will have to do a copy operation and load operation anyway. I could have the entire engine and models all be a part of the docker image, but won't that make the cold start time be longer due to the download time? Currently I'm having crazy weird numbers, sometimes it take 0.1 delay and then 160 seconds of execution time, sometimes it takes 40 delay and 200 exec, and as long as the pod not idle, it takes about 1s delay and 20-40s of exec time. I'm trying to better understand why

17 replies

•Created by Sassy Pantsy on 6/10/2024 in #⚡｜serverless

How can I connect my network volume to a serverless endpoint?

It works. Thanks for the help man, you have no idea how much time you've saved me

51 replies

•Created by Sassy Pantsy on 6/10/2024 in #⚡｜serverless

How can I connect my network volume to a serverless endpoint?

thanks man, I'll test it out and get back to you

51 replies

•Created by Sassy Pantsy on 6/10/2024 in #⚡｜serverless

How can I connect my network volume to a serverless endpoint?

I will! at least i learned some docker 🙂

51 replies

•Created by Sassy Pantsy on 6/10/2024 in #⚡｜serverless

How can I connect my network volume to a serverless endpoint?

thank you so much

51 replies

•Created by Sassy Pantsy on 6/10/2024 in #⚡｜serverless

How can I connect my network volume to a serverless endpoint?

oh my god i've looked so much online

51 replies

•Created by Sassy Pantsy on 6/10/2024 in #⚡｜serverless

How can I connect my network volume to a serverless endpoint?

hahaha no way dude I never saw this one

51 replies

•Created by Sassy Pantsy on 6/10/2024 in #⚡｜serverless

How can I connect my network volume to a serverless endpoint?

wait what

51 replies

•Created by Sassy Pantsy on 6/10/2024 in #⚡｜serverless

How can I connect my network volume to a serverless endpoint?

yeah

51 replies

•Created by Sassy Pantsy on 6/10/2024 in #⚡｜serverless

How can I connect my network volume to a serverless endpoint?

whoops sorry i seem to have uploaded the wrong start.sh script, updated it now

51 replies

•Created by Sassy Pantsy on 6/10/2024 in #⚡｜serverless

How can I connect my network volume to a serverless endpoint?

that's the repo with the dockerfile, start.sh and main.py

51 replies

•Created by Sassy Pantsy on 6/10/2024 in #⚡｜serverless

How can I connect my network volume to a serverless endpoint?

https://github.com/AoutoCooper/ComfyUI-serverless-image

51 replies

•Created by Sassy Pantsy on 6/10/2024 in #⚡｜serverless

How can I connect my network volume to a serverless endpoint?

yeah, worker's restarting as well

51 replies

•Created by Sassy Pantsy on 6/10/2024 in #⚡｜serverless

How can I connect my network volume to a serverless endpoint?

sure

51 replies

•Created by Sassy Pantsy on 6/10/2024 in #⚡｜serverless

How can I connect my network volume to a serverless endpoint?

Hi just want to update that I've remapped the paths according to what you said (took some trial and error but at the end we got it) and now my trouble isn't with files and direcories not being found, is that the start.sh script (which is the container's entry point) is being called over and over again, practically on set intervals. When I look at the logs I see that comfy ui thankfully starts printing out logs, but is getting stuck on a prestartup time for a custom_node (the comfyui manager if it matters). Do you have any idea why it runs in this recursive manner?

51 replies

•Created by Sassy Pantsy on 6/10/2024 in #⚡｜serverless

How can I connect my network volume to a serverless endpoint?

Ok thanks man!!! I'll rebuild my container and let you know how it goes

51 replies

•Created by Sassy Pantsy on 6/10/2024 in #⚡｜serverless

How can I connect my network volume to a serverless endpoint?

ok yeah I see it now

51 replies

•Created by Sassy Pantsy on 6/10/2024 in #⚡｜serverless

How can I connect my network volume to a serverless endpoint?

so my commands need to be to first start the venv?

51 replies