Empty Postgres reference env var

I referred the guide on https://blog.railway.app/p/automated-postgresql-backups to create a node cron job in our monorepo. It runs fine, but the backup archives are empty. But when I try running the
pg_dump ${env.BACKUP_DATABASE_URL} -F t | gzip > ${path}
pg_dump ${env.BACKUP_DATABASE_URL} -F t | gzip > ${path}
command locally, I am getting a full dump of ~20MB
80 Replies
Percy
Percy17mo ago
Project ID: N/A
sebastiantf
sebastiantf17mo ago
N/A
sebastiantf
sebastiantf17mo ago
sebastiantf
sebastiantf17mo ago
sebastiantf
sebastiantf17mo ago
sebastiantf
sebastiantf17mo ago
Running the node cron job locally with the right env vars also work fine. Only failing on Railway Already tried redeploying / restarting the cron service. No luck Seems like the issue is BACKUP_DATABASE_URL not being correct. pg_dump would still return and an empty *.tar.gz will be created. Thats whats being uploaded to S3
sebastiantf
sebastiantf17mo ago
sebastiantf
sebastiantf17mo ago
Can someone from the team please look into this?
sebastiantf
sebastiantf17mo ago
Also noticed this in one of the network request responses:
sebastiantf
sebastiantf17mo ago
@Brody Sorry about the ping. But seems like this post is not getting any attention 😓
Brody
Brody17mo ago
bruh it's 4:30 am
sebastiantf
sebastiantf17mo ago
damn! sorry about that 🙏
Brody
Brody17mo ago
if you require priority support you may want to look into the teams plan
Percy
Percy17mo ago
Flagging this thread. A team member will be with you shortly.
Unknown User
Unknown User17mo ago
Message Not Public
Sign In & Join Server To View
sebastiantf
sebastiantf17mo ago
Bump!
Ray
Ray17mo ago
What's your project ID, and where are you setting ${{Postgres.DATABASE_URL}} from?
sebastiantf
sebastiantf17mo ago
Service Id: e041e6bc-e740-4fec-a6dc-41e022086166 Setting the env var using the helper in the UI to set reference vars Sorry for the late response, I wasn't notified of anything here
Brody
Brody17mo ago
total shot in the dark here but can you create another postgres database, attach a variable reference to the cron service, then run railway variables again
sebastiantf
sebastiantf17mo ago
I'll try that out. actually, can i create another postgres database alongside the existing one? what would be the reference name for that then? the current db is being used by staging backend api and i dont really want that to go down
Brody
Brody17mo ago
doesn't really matter, this is purely just a test to see if the reference gets rendered oh and yes you absolutely can create more databases of the same type, it won't effect anything
sebastiantf
sebastiantf17mo ago
ah right i see the -1 appended
Brody
Brody17mo ago
yep!
sebastiantf
sebastiantf17mo ago
Brody
Brody17mo ago
no no you where supposed to add another variable, with a different name
sebastiantf
sebastiantf17mo ago
oh mb
Brody
Brody17mo ago
try it anyway railway variables
sebastiantf
sebastiantf17mo ago
okay. waiting for redeploy to finish
Brody
Brody17mo ago
also not needed
sebastiantf
sebastiantf17mo ago
oh okay
Brody
Brody17mo ago
assuming you have the cli correctly linked to the cron service, just run the command
sebastiantf
sebastiantf17mo ago
still empty. uploading screenshot..
sebastiantf
sebastiantf17mo ago
sebastiantf
sebastiantf17mo ago
Also noticing this (discrepancy?) when i shell into it
sebastiantf
sebastiantf17mo ago
sebastiantf
sebastiantf17mo ago
I can see DATABASE_URL in the shell env, but not in the variables command abovc
Brody
Brody17mo ago
little correction here, that's a local shell
sebastiantf
sebastiantf17mo ago
wdym
Brody
Brody17mo ago
railway shell is a local shell
sebastiantf
sebastiantf17mo ago
yes, but with service's env in it, right?
Brody
Brody17mo ago
correct
sebastiantf
sebastiantf17mo ago
so the railway variables output should also be same as the service's env?
Brody
Brody17mo ago
as the service variables in the UI, yes either way, where is that DATABASE_URL variable from?
sebastiantf
sebastiantf17mo ago
yeah thats the discrepancy i'm noticing
Brody
Brody17mo ago
is that something you've set in your own shell?
sebastiantf
sebastiantf17mo ago
no my shell prints empty
Brody
Brody17mo ago
now that's beyond bizarre
sebastiantf
sebastiantf17mo ago
I do have that DATABASE_URL env in another service in the same project tho - the backend api service
sebastiantf
sebastiantf17mo ago
sebastiantf
sebastiantf17mo ago
but still, separate service
Brody
Brody17mo ago
would the DATABASE_URL variable that shows up in the railway shell happen to be the same variable that's being referenced in that screenshot?
sebastiantf
sebastiantf17mo ago
yeah, that's correctly referenced to the correct expected database
Brody
Brody17mo ago
not what I meant
sebastiantf
sebastiantf17mo ago
sorry 😓
Brody
Brody17mo ago
is the variable that's hidden under the red block the same as the variable for this reference
sebastiantf
sebastiantf17mo ago
yes. thats what i meant
Brody
Brody17mo ago
I wish I could see the things Ray can see somethings
sebastiantf
sebastiantf17mo ago
anything else you want me to try?
Brody
Brody17mo ago
yeah but there's just so much back and forth not your fault, just comes with text based support
sebastiantf
sebastiantf17mo ago
I also tried changing BACKUP_DATABASE_URL to DATABASE_URL both in code & env But still seeing the same thing
sebastiantf
sebastiantf17mo ago
I now tried directly entering the database connection url in plaintext Backups are still empty Seeing something weird on CLI
sebastiantf
sebastiantf17mo ago
^ Tried similarly with prod db url which is on heroku, still empty, and similar output as above screenshots above bump! bump!
Brody
Brody17mo ago
just a friendly reminder, if you require priority support you may want to look into the teams plan
Greg Schier
Greg Schier17mo ago
By "empty" do you mean there is an archive in your S3 bucket but it's 0 bytes?
sebastiantf
sebastiantf17mo ago
yes
Seems like the issue is BACKUP_DATABASE_URL not being correct. pg_dump would still return and an empty *.tar.gz will be created. Thats whats being uploaded to S3
Ray
Ray17mo ago
do you see any connection errors in that template's deployment logs?
sebastiantf
sebastiantf17mo ago
no
sebastiantf
sebastiantf17mo ago
sebastiantf
sebastiantf17mo ago
bump
Ray
Ray17mo ago
I looked at your service again and noticed a few things: - You only have the variables set for the backup service in staging. Is this intended? - The BACKUP_DATABASE_URL is pointing to your staging instance Postgres. Does it have any data? - When you tried railway shell / railway variables, which Railway environment is that running in?
sebastiantf
sebastiantf17mo ago
1. We have set other variables for other services. But the BACKUP_DATABASE_URL variable is only set for the backup service. That is intended. 2. Yes there is data in the staging Postgres instance 3. The screenshots also include prev commands for railway link, railway service that chooses the backup service / backend service in the staging env
Ray
Ray17mo ago
It's still empty? I can confirm your container sees the database URL correctly. This might be a template issue
sebastiantf
sebastiantf17mo ago
yup they're still empty.
Ray
Ray17mo ago
@fp any ideas?
Faraz
Faraz17mo ago
Looks like you've forked the repo into a private repo? If you can replicate the issue with the actual template repo, I can try and take a look. Because I know a bunch of people using the template and no one's reported any issues.
sebastiantf
sebastiantf17mo ago
Yeah we have a private monorepo. So we've forked the template as a subrepo in it. Also it works very fine when i run it in local from within the monorepo
Faraz
Faraz17mo ago
It could be an issue with how you've configured your monorepo?
sebastiantf
sebastiantf17mo ago
i dont think so, because the script does get executed fine on railway and the expected file is being uploaded to s3 monorepo config error should really prevent successful execution, right?
Adam
Adam17mo ago
It definitely can If the template is working fine, something about your custom configuration is wrong
sebastiantf
sebastiantf17mo ago
Template repo within our monorepo is executing fine in local and railway. Only the pgdump is empty in railway
Want results from more Discord servers?
Add your server