R
Railway9mo ago
ryan

Deploy logs stopped appearing after a manual redeploy

Hi, I have a project running several services. Deploy logs for one of the services randomly stopped appearing within the dashboard. The logs still appear fine in our “Development” environment but have stopped working in the “Production” environment. I noticed the deploy logs stopped appearing after manually triggering a “Redeploy” from the service menu. I’ve also tried restarting, redeploying, and pushing a code change to trigger a full rebuild.
I have experienced this with another service, however, the logs started randomly appearing after some time. Project ID: d80407b4-a520-4fc9-acec-d8bf622dc449 Service ID: eb5da072-e189-4fab-b4d1-0918eb0425d6 Service name: scheduler
6 Replies
Percy
Percy9mo ago
Project ID: d80407b4-a520-4fc9-acec-d8bf622dc449,eb5da072-e189-4fab-b4d1-0918eb0425d6
Brody
Brody9mo ago
is the app in the production environment using a different logger?
ryan
ryanOP9mo ago
same configuration + logger
Duchess
Duchess9mo ago
New reply sent from Help Station thread:
Since you are business class, I will go ahead and mark this thread as private so that the team can look into this matter. locking the discord thread in favor of the help thread.
You're seeing this because this thread has been automatically linked to the Help Station thread.
Brody
Brody9mo ago
me ^
Duchess
Duchess9mo ago
New reply sent from Help Station thread:
can you provide some extra information before I go ahead and do that, like for example, what logger you are using?
You're seeing this because this thread has been automatically linked to the Help Station thread. New reply sent from Help Station thread:
Yeah, it's a Django project using structlog. There is a processor that turns these logs into json format. Three of the services (platform-api, platform-worker, scheduler) use this logger with the same configuration. Prior to the that logger even being initialized, we run some echo statements in the start up script.
#!/bin/bash

set -e

# Railway recommends sleeping for 3s when using private networking due to an initial delay
# in dns resolution
echo "Sleeping..."
sleep 3

echo "Starting scheduler..."
celery -A config.celery beat --loglevel=INFO
#!/bin/bash

set -e

# Railway recommends sleeping for 3s when using private networking due to an initial delay
# in dns resolution
echo "Sleeping..."
sleep 3

echo "Starting scheduler..."
celery -A config.celery beat --loglevel=INFO
These logs have stopped appearing entirely.
You're seeing this because this thread has been automatically linked to the Help Station thread. New reply sent from Help Station thread:
What's also interesting is that I can see a single log entry in my deploy logs now in the production environment. But it appears to be missing the entire start up logs which includes the shell script and the structlog/json format logs of the app itself.
You're seeing this because this thread has been automatically linked to the Help Station thread. New reply sent from Help Station thread:
is the app running though?
You're seeing this because this thread has been automatically linked to the Help Station thread. New reply sent from Help Station thread:
Since you are business class, I will go ahead and mark this thread as private so that the team can look into this matter. locking the discord thread in favor of the help thread. I'm not sure that we're actually business class -- so that's why I started in Discord. We were put on the business class plan as a trial but we never ended up converting
You're seeing this because this thread has been automatically linked to the Help Station thread. New reply sent from Help Station thread:
is the app running though? Yes
You're seeing this because this thread has been automatically linked to the Help Station thread. New reply sent from Help Station thread:
I am trying to attach a screenshot but cant figure out how. But to the best of my knowledge, it's running? The scheduler service is a celery beat process which is used to schedule async tasks. The tasks are being scheduled and ran. The single log line that I referenced a few messages back indicates it's queuing up the tasks.
You're seeing this because this thread has been automatically linked to the Help Station thread. New reply sent from Help Station thread:
I'm not sure that we're actually business class -- so that's why I started in Discord. We were put on the business class plan as a trial but we never ended up converting maybe you'll consider converting if the team can swiftly resolve this issue :) I am trying to attach a screenshot but cant figure out how. at the moment these forums don't support uploading attachments. I think there's enough information here now, going to make this private right now! (I will lose access to this thread)
You're seeing this because this thread has been automatically linked to the Help Station thread. New reply sent from Help Station thread:
(hidden)
You're seeing this because this thread has been automatically linked to the Help Station thread. New reply sent from Help Station thread:
(hidden)
You're seeing this because this thread has been automatically linked to the Help Station thread.
Want results from more Discord servers?
Add your server