umami stoped working
Hi all!
Sorry, I'm a quite new user of railway, not quite sure about everything.
I'm happily using a umami service under railway, which have no problem at all, but a few days ago everything stopped working, the service cannot be reachable. I did not changed any variable all.
Tried to redeploy everything, but it seems umami unable to reach the DB.
Error message:
"$ npm-run-all check-db update-tracker start-server
$ node scripts/check-db.js
β DATABASE_URL is defined.
β Unable to connect to the database.
error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.
ERROR: "check-db" exited with 1.
error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.
yarn run v1.22.19"
And the other problem is that these reconnections attempts used all of my available credits. π₯
Project ID: 0dfb3481-24bb-4b76-b325-d31f1c4cf72d
11 Replies
Project ID:
0dfb3481-24bb-4b76-b325-d31f1c4cf72dis umami deployed from a github repo or a docker image?
Github repo
is there any errors in the postgres logs?
have you tried restarting postgres?
Yes, there is a lot of message, but i'm not sure that these are errors. Regarding some research these messages could be normal:
"2024-01-01 21:17:23.143 UTC [7] LOG: checkpointer process (PID 20614) was terminated by signal 6: Aborted
2024-01-01 21:17:23.143 UTC [7] LOG: terminating any other active server processes
2024-01-01 21:17:23.146 UTC [7] LOG: all server processes terminated; reinitializing
2024-01-01 21:17:23.161 UTC [20616] LOG: database system shutdown was interrupted; last known up at 2024-01-01 21:17:23 UTC
2024-01-01 21:17:23.213 UTC [20616] LOG: database system was not properly shut down; automatic recovery in progress
2024-01-01 21:17:23.215 UTC [20616] LOG: redo starts at 1/728C74F8
2024-01-01 21:17:23.215 UTC [20616] LOG: invalid record length at 1/728C79F0: wanted 24, got 0
2024-01-01 21:17:23.215 UTC [20616] LOG: redo done at 1/728C79C8 system usage: CPU: user: 0.00 s, system: 0.00 s, elapsed: 0.00 s
2024-01-01 21:17:23.218 UTC [20617] LOG: checkpoint starting: end-of-recovery immediate wait
2024-01-01 21:17:23.218 UTC [20617] PANIC: could not write to file "pg_logical/replorigin_checkpoint.tmp": No space left on device"
Yes, I tried to restart both services a bunch of times.
Solution
show me a screenshot of your postgres volumes graph please
Ohh god, that was it. You're awwsome @Brody ! π
For the record:
Postgres service, Settings, Go to Volume. Normally, the default size of a volume is limited at 500 MB, which can be resized to 5 GB with a ridiculously long command, which is quite funny. π
After doing these I just need to restart all of the services and everything is up and running again!
Again, thanks for the quick help, great community! 
the default size of a volume is limited at 500 MBthats only true for trial users, once you upgrade to hobby or above you can grow the volume. any volumes or services with volumes created after upgrading to hobby will already be 5gb
long commanddo you mean long confirmation?
Yes, confirmation, sorry. I started this service while in trial, after I upgraded to hobby it doesn't changed. I think because of the service interruption. It would be nice if Railway send any notification about volume size increases.
hey thats a very good idea
made a feedback post with your feedback here: https://discord.com/channels/713503345364697088/1191507681282895903