Is there a cron job to re-deploy services?
Hey guys! So, I have some 3rd party services that are using too much RAM (no memory spikes, it slowly increases and then keeps consistent).
As I can't edit the code, it came to my mind that I can restart them every 24 hours. I was planning to use the Railway's API to achieve this, but then I thought there might be a cron job for this :)
PS: Just out of curiosity, does the V2 runtime improve the RAM consumption or is it indifferent?
33 Replies
Project ID:
N/Ahell yeah, let me know if you have any questions
man I don't know how much Railway is paying you but it's not enough
it works really well
thank you so much
no problem!
hey brody, is this project open source by any chance? I'd like to implement a retry system, since I'm getting these errors, which look more like a Railway's API issue
some services got redeployed, but most of them actually not
the services do exist, and I'm checking the SCHEDULE_N with the Cron Config bookmark and it's exactly the same, that's why I think it may be a Railway's issue (also because of the 503 status code lol)
it is, follow the source of the template service
do you know go?
looks like you where the first ever to report this issue - https://discord.com/channels/713503345364697088/1270868312091394088/1271422392203218954
but the code should do retries for a minute or however many minutes you are okay with the cron running late.
I do
perfect, what retry package are you thinking of?
at my job we use https://github.com/cenkalti/backoff
but happy to give a try to this one https://github.com/avast/retry-go which looks simpler
wdyt?
if you are more familiar with backoff then you can use that, just make sure not to hammer the api and set a max time before "giving up"
sure
what's the best way to test this?
something that returns 503?
https://utilities.up.railway.app/status-code/503
I added the backoff retry, but I can't create PRs in your repo
oh why not?
nvm, created a PR from my fork
cool, I'll look over it tonight
will be using the fork for now to see how it goes
will keep you posted
have you tested your retries with this that returns 503
hey man! sorry I was off during the weekend
I updated the template in my project, but still having some issues:
- The 503 issue is still there, not sure what could be causing it. I think I will try by changing the backoff params, but if it keeps returning 503 errors, the retry system is kind of useless.
- I think the os.Exit() after a request fails could be removed. Maybe it failed to redeploy a service, but I want it to keep trying with all the other services in the list instead of restarting the container. What do you think?
Here's the project ID, just in case: b27fab0d-135f-4876-afbe-34e83cad72ac
The 503 issue is still theredifferent error handling logic is not going to fix issues with the backboard.
I think the os.Exit() after a request fails could be removed.agree, changed, image built.
Yeah, still getting the 503 issue. I'll try changing the schedule of the cron jobs so they're not performed all at the same time
they're all being executed concurrently? could I add a delay of 30 seconds between each execution to see if this solves the issue?
how many schedules are you running?
around 50
why??
because I have multiple projects using Chatwoot and I want to restart them at 6am UTC
why do you need to restart Chatwoot?
the RAM usage is too high in some cases
and when I redeploy them, it decreases
I just don't want to manually redeploy them
yeah, then try adjusting the schedules to stagger the restarts
it worked after staggering the redeploys :)
thank you
then I think you where getting rate limited