cauldron_cake💚
RRunPod
•Created by cauldron_cake💚 on 12/29/2023 in #⛅｜pods
Services don't start
So weird. Thanks for checking! I'll keep testing
18 replies
RRunPod
•Created by cauldron_cake💚 on 12/29/2023 in #⛅｜pods
Services don't start
Hey Ashley! Thanks for the quick reply on github. So far I've tried running it on an RTX 6000 Ada and an RTX 4090, and the server has been set to any, but I can try a specific server to isolate.
I did notice that runpod is no longer giving me the choice between secure cloud (which I normally use) and gpu cloud. Currently my only option is GPU cloud. Wasn't sure if something changed on my end or if it was just a change in runpod's UI, but seemed worth mentioning.
18 replies
RRunPod
•Created by cauldron_cake💚 on 12/29/2023 in #⛅｜pods
Services don't start
Update: Assuming it's something with the template. The Kohya_ss standalone template worked fine
18 replies
RRunPod
•Created by cauldron_cake💚 on 12/29/2023 in #⛅｜pods
Services don't start
I thnk everything looks normal except maybe the symlinks for dreambooth and text inversion
18 replies
RRunPod
•Created by cauldron_cake💚 on 12/29/2023 in #⛅｜pods
Services don't start
2023-12-29T16:05:53.743971926Z Syncing ComfyUI to workspace, please wait...
2023-12-29T16:05:58.184933424Z Syncing Application Manager to workspace, please wait...
2023-12-29T16:05:58.654091435Z Fixing Stable Diffusion Web UI venv...
2023-12-29T16:05:58.657608222Z Fixing Kohya_ss venv...
2023-12-29T16:05:58.661171919Z Fixing ComfyUI venv...
2023-12-29T16:05:58.665633451Z Configuring accelerate...
2023-12-29T16:05:58.688464516Z Starting Stable Diffusion Web UI
2023-12-29T16:05:58.689192512Z Stable Diffusion Web UI started
2023-12-29T16:05:58.689210491Z Log file: /workspace/logs/webui.log
2023-12-29T16:05:58.691251259Z Starting Kohya_ss Web UI
2023-12-29T16:05:58.692034254Z Kohya_ss started
2023-12-29T16:05:58.692047304Z Log file: /workspace/logs/kohya_ss.log
2023-12-29T16:05:58.696341637Z Starting ComfyUI
2023-12-29T16:05:58.706286843Z ComfyUI started
2023-12-29T16:05:58.706305543Z Log file: /workspace/logs/comfyui.log
2023-12-29T16:05:58.709811431Z Starting Tensorboard
2023-12-29T16:05:58.717965869Z ln: failed to create symbolic link '/workspace/logs/dreambooth/dreambooth': File exists
2023-12-29T16:05:58.720013886Z ln: failed to create symbolic link '/workspace/logs/ti/textual_inversion': File exists
2023-12-29T16:05:58.722665349Z Tensorboard Started
2023-12-29T16:05:58.723045417Z All services have been started
2023-12-29T16:05:58.723575894Z Pod Started
2023-12-29T16:05:58.723615143Z Starting Jupyter Lab...
2023-12-29T16:05:58.723913951Z Jupyter Lab started
2023-12-29T16:05:58.723957801Z Exporting environment variables...
2023-12-29T16:05:58.731740011Z Container is READY!
18 replies
RRunPod
•Created by cauldron_cake💚 on 12/29/2023 in #⛅｜pods
Services don't start
2023-12-29T16:04:35.791740132Z Starting Nginx service...
2023-12-29T16:04:35.811540626Z * Starting nginx nginx
2023-12-29T16:04:35.824502734Z ...done.
2023-12-29T16:04:35.824513804Z Running pre-start script...
2023-12-29T16:04:35.824517553Z Container is running
2023-12-29T16:04:35.824521583Z Syncing venv to workspace, please wait...
2023-12-29T16:05:17.502348603Z Syncing Stable Diffusion Web UI to workspace, please wait...
2023-12-29T16:05:19.924204304Z Syncing Kohya_ss to workspace, please wait...
18 replies
RRunPod
•Created by cauldron_cake💚 on 12/29/2023 in #⛅｜pods
Services don't start
I did notice this template didn't populate in a search or by default like it usually does, I had to launch it using the link from the GitHub page, not sure if that might be relevant
18 replies
RRunPod
•Created by cauldron_cake💚 on 12/29/2023 in #⛅｜pods
Services don't start
same issue on the new pod, ID: yzbsztfwcaf1ga
Here are the container logs:
(as a file since I don't have Nitro)
18 replies
RRunPod
•Created by cauldron_cake💚 on 12/29/2023 in #⛅｜pods
Services don't start
Hmm. I just started it on another GPU config and it's going through a longer cycle building the container, so maybe didn't build properly on the other config?
I was using one of the RTX 4090 configs which I had used several times previously, and used last yesterday
18 replies
RRunPod
•Created by cauldron_cake💚 on 12/29/2023 in #⛅｜pods
Services don't start
I can restart the pod to grab the exact logs, but it looked like everything was standard in the logs all the way through "Container is READY!", system and container both started up without issue in 1-2 minutes
18 replies