Recommended approach for django-crontab on railway
I'm using the django-crontab package to schedule a daily job that works fine when run locally, but I haven't been able to figure out how to get it working on Railway.
I've tried updating my Procfile as shown below, but haven't had any luck. What's the recommended approach to scheduling a job on Railway to interact with a Django app?
web: python manage.py crontab add && python manage.py migrate && python manage.py collectstatic --noinput --clear && gunicorn django_project.wsgi
Project ID for reference: 3eeb1796-c531-4b55-9016-db32a2502d6175 Replies
Project ID:
3eeb1796-c531-4b55-9016-db32a2502d61so
python manage.py crontab add is the command you run to start the cron tasks?Correct. Once added, it then looks inside the Django settings to find the cron schedule and task to run.
and is that a one off thing, or does the command stay alive and run on the set schedule
It runs according to the cron schedule defined in the Django settings.
yes i did send before i was done typing 😅
so you will want two railway services, one service will run just the crontab, and the other will run just the django site.
the identical services will be deploying into the same project, they will have the same repo, same branch, and same service variables, the only difference would be the service names, and the start commands.
name one service something like
Crontab and the other mySite <-- choose a better name
once you have two identical services, in the Crontab service's settings, change the start command to python manage.py crontab add, in the mySite service's settings, change the start command to python manage.py migrate && python manage.py collectstatic --noinput --clear && gunicorn django_project.wsgiNo worries...
python manage.py crontab add adds the active job to run, and then runs as part of the Django app itself. Example docs here -> https://pypi.org/project/django-crontab/
Because it's part of the app, I wasn't sure if things would still work on Railway if I were to split them out separately...okay then hold on, i might have misjudged something
it looks like
python manage.py crontab add adds the jobs to the systems cron scheduler
but deployments dont come with a cron scheduler, do you get any errors in the deploy logs?No errors in the deploy logs.
do you think you could put together the most basic django app with a cron job that prints a message every minute?
with that i could dig into this issue
Yea, I can spin up a very basic version that does that. How do the logs need to be emitted to show up in Railway? Does a simple python 'print' statement suffice?
yes print should do just fine, thanks!!
Should I just share the github repo to you for the basic app, or actually deploy it on railway as well?
share it with me, just make sure it works locally, ill do my best to get it running on railway, then whatever i do, we apply to your actual app
so yeah django app that just has a hello world page and prints hello world every minute, no need for assets or anything like that
This should be all set to deploy with no additional changes needed -> https://github.com/gymanji/railway-django-crontab
GitHub
GitHub - gymanji/railway-django-crontab
Contribute to gymanji/railway-django-crontab development by creating an account on GitHub.
cool, what command did you run locally to have it start printing hello world every minute?
Assuming you're running it locally, you'd run:
First time:
1. pip3 install -r requirements.txt
2. python3 manage.py migrate
After that:
1. python3 manage.py crontab add
2. python3 manage.py runserver
thats for the database stuff, what about starting the cron jobs?
hit enter too fast.. see updated commands above
haha you did the same thing as me
The 'print' statement from the cron.py file shows up in /var/mail/{user}
what do you mean by that? shouldnt i see it in the railway deploy logs if its working correctly?
Any time you change anything related to the cron.py file or the cron schedule (set at the end of "django_project/settings.py"), you'll need to re-add the crontab.
I didn't test deploying the simple app on railway, I'm just sharing what I saw in the local behavior.
gotcha
will update if i found a solution or if i have any questions!
so i was correct when i said you would need two services.
delete your Procfile
add this nixpacks.toml file to your project:
deploy two nearly identical services, same variables, same repo, etc (the cron service doesnt need a domain)
on the service that is going to do the cron tasks, set this as the start command
python manage.py crontab add && cron -f
on the service that will be django, set this as the start command python manage.py migrate && python manage.py collectstatic --noinput --clear && gunicorn django_project.wsgi
do ask if you need extra clarification on anythingI followed the above steps, but am seeing the following error in the Deploy logs for the cron version of the service
/usr/bin/crontab: not found.seems like you didn't add the nixpacks.toml file
It's added to the root of the git repo. Does it need to be in a different directory?
what's the root directory of the service set to
Also the root of the git repo. Where did you place the nixpacks.toml file when you tested separately with the simple app?
in the root of the project
can you send the build logs of your latest deployment please
also a screenshot of your railway project
Hold on, I just made one minor change and it looks like that error is gone now. I'll get back in a bit to confirm whether or not it's working once the cron job has a chance to run.
may I still see the project screenshot please
This?
perfect, I assume the django app is working as intended?
Yes, it seems to be.
alright, let me know if that cron job runs, but from the logs I got during my testing, my logs looked exactly like yours and it worked for me, so very good sign
Fingers crossed then! I appreciate the assistance as well!
happy to help
Ok, so I've tried a few different tweaks now and still don't see the cron process executing. There are multiple print statements inside the function called, but none of them show up in the logs area. The logs aside, the function is also configured to send an email once it completes that isn't arriving.
The deploy logs show this right after running, which seems to look normal (but there's no output otherwise).
Here are the most recent set of build logs as well:
``
there isnt going to be logs, its a background process
Ok, good to know.
In the start command, does it need to have "... && cron -f" appended at the end?
yes, use the start command as i wrote it
Yes, it's using "python manage.py crontab add && cron -f" right now.
email auth is likely failing then
whos your email provider
Gmail. The config works locally though, so the only difference would be in how the app accesses the variables inside Railway. However, the app has no problems accessing the database variables set in Railway 🤷♂️
gmail is super well known to block auth from non residential ip addresses
id recommend using your own domain with zoho instead, ive had 100% success rate with a zoho email on railway
Is there anyway to view process info to confirm that it's an IP block issue first?
since its a background task it cant log to the parent stdout/stderr, therefore you can't see it in the deploy logs
Is there a 'tmp' local directory or something similar that a log message could be written to?
you dont have access to the filesystem so theres no point
Ok, worth a shot 😆
maybe time to move to the python scheduler module?
with that you have no need to install cron via nix, and you can see all prints in the deploy logs (two services still needed)
your crontab module is from 2016 after all, time for an update
Might be the best route.
i strongly think so
from your log, you only have one job, i cant imagine it would be too hard to switch over to the scheduler module
No, likely not a major switch.
Why would using the schedule module still require two services?
the event loop is likely a blocking task so it needs it own service, plus its just generally better practice
but im not too sure on what is the best schedular module, i think
sched is built in?
or this one https://schedule.readthedocs.io/en/stable/ it looks very user friendly, this is what i think i would chooseCircling back on this in case it helps others, but ultimately I ended up doing the following:
1. Moved the function into a custom django admin command
2. Create a duplicated version of the django service in Railway
3. Set the "Cron Schedule" inside the service settings with a 'Custom Start Command' that executes the custom django admin command
@darth thanks for the update, I'm trying to deploy a weekly cron email for my django app and have been thinking about these same problems. To clarify, step 2 is necesary because you don't want to take up resources on your primary django service?
it's necessary because a single railway service can't be a cron job and a regular service at the same time, also it's good practice incase the email schedule crashes, it won't take down the django site
Gotcha, thanks @Brody
If I have multiple cron jobs (let's say, for different emails to users that run at different frequencies), it's normal/a best practice to spin up a new service for each one? Seems like it might be expensive but the pro plan billing page indicates "Unlimited concurrent builds", so... fine I guess?
builds == services in this case?the pro plan does not have unlimited concurrent builds anymore, that needs to be removed. but yeah a service for the different frequencies seems good to me, after all, if you use railways cron scheduler properly you would only be charged for the resources used while the job is running, the rest of the time it would be off completely
oh and no, builds does not equal services, builds are the time when you deploy your code and it's being built
(it's now 10 concurrent builds)
Ah awesome, thanks.
the rest of the time it would be off completely - do I need to turn on App Sleeping for that?nope, if you use railways cron scheduler properly the process goes like so:
schedule time reached -> railway starts your service up -> your service does what it does -> your service exits as soon as it's done.
and if your service exits when it's done, then it's not using resources until the next scheduled time
(I'm using Railway's cron scheduler)
now I keep saying properly because everything I said is only correct if your service exits as soon as it's done doing what it needs to do
Cool. I assume nothing happens after I run my custom django management command, but I guess I'll need to research django a little
```
"deploy": {
"numReplicas": 1,
"startCommand": "python manage.py migrate && python manage.py collectstatic --no-input && python manage.py new_custom_management_command && gunicorn -b [::]:$PORT faves_project.wsgi",
"restartPolicyType": "ON_FAILURE",
"restartPolicyMaxRetries": 10
}
is that the command you have the cron service running?
Yeah, I guess I could not run gunicorn after the custom command 🤔
because that will start django in the cron service
and you will be charged for resource usage 24/7
since gunicorn running django is a long running process
you can set the appropriate start command in the service settings
Hm so just removing gunicorn should do the trick I hope
it's possible, but you'd need to check locally, you also don't need to do migrations or collect static
My emails may rely on the static content or migrations so I'm leaving them in for now
oh okay that makes sense
but definitely check if your added custom command exits when it's done
It's just a python function that loops through all users and possibly sends an email to each one, so I assume so
you'd want to be sure