Tim
Tim
RRailway
Created by Tim on 2/5/2024 in #✋|help
Django with Gunicorn gives frequent [CRITICAL] WORKER TIMEOUTs on simple requests
@MIGHTY_MIDHUN I solved it eventually by moving my static files to a Google Cloud Storage bucket.
25 replies
RRailway
Created by Tim on 2/5/2024 in #✋|help
Django with Gunicorn gives frequent [CRITICAL] WORKER TIMEOUTs on simple requests
I did not, based on my logging it seems to be hanging on loading static files, which I am serving through whitenoise (6.6.0)
INSTALLED_APPS = [
...
'whitenoise.runserver_nostatic',
"django.contrib.staticfiles",
...
]

MIDDLEWARE = [
"django.middleware.security.SecurityMiddleware",
"whitenoise.middleware.WhiteNoiseMiddleware",
...
]

STORAGES = {
"default": {
"BACKEND": "storages.backends.gcloud.GoogleCloudStorage",
},
"staticfiles": {
"BACKEND": "whitenoise.storage.CompressedManifestStaticFilesStorage",
},
}
INSTALLED_APPS = [
...
'whitenoise.runserver_nostatic',
"django.contrib.staticfiles",
...
]

MIDDLEWARE = [
"django.middleware.security.SecurityMiddleware",
"whitenoise.middleware.WhiteNoiseMiddleware",
...
]

STORAGES = {
"default": {
"BACKEND": "storages.backends.gcloud.GoogleCloudStorage",
},
"staticfiles": {
"BACKEND": "whitenoise.storage.CompressedManifestStaticFilesStorage",
},
}
25 replies
RRailway
Created by Tim on 2/5/2024 in #✋|help
Django with Gunicorn gives frequent [CRITICAL] WORKER TIMEOUTs on simple requests
I already put the gunicorn timeout on 60 to test if it eventually would finish (which it does not).
25 replies
RRailway
Created by Tim on 2/5/2024 in #✋|help
Django with Gunicorn gives frequent [CRITICAL] WORKER TIMEOUTs on simple requests
It happens on arbitrary requests that do not take any significant amount of time (should not be even close to a second), and it does not happen locally nor on my PythonAnywhere hosted (test)server (that does not deploy with Docker) 🤔 .
25 replies
RRailway
Created by Tim on 2/5/2024 in #✋|help
Django with Gunicorn gives frequent [CRITICAL] WORKER TIMEOUTs on simple requests
I added debug logging, but I don't see anything in my logs when this happens.
[2024-02-08 18:58:05 +0000] [16] [CRITICAL] WORKER TIMEOUT (pid:17)
[2024-02-08 19:58:05 +0100] [17] [INFO] Worker exiting (pid: 17)
[2024-02-08 18:58:06 +0000] [16] [ERROR] Worker (pid:17) exited with code 1
[2024-02-08 18:58:06 +0000] [16] [ERROR] Worker (pid:17) exited with code 1.
[2024-02-08 18:58:06 +0000] [19] [INFO] Booting worker with pid: 19
[2024-02-08 18:59:09 +0000] [16] [CRITICAL] WORKER TIMEOUT (pid:19)
[2024-02-08 19:59:09 +0100] [19] [INFO] Worker exiting (pid: 19)
[2024-02-08 18:59:09 +0000] [16] [ERROR] Worker (pid:19) exited with code 1
[2024-02-08 18:59:09 +0000] [16] [ERROR] Worker (pid:19) exited with code 1.
[2024-02-08 18:59:09 +0000] [21] [INFO] Booting worker with pid: 21
[2024-02-08 19:00:11 +0000] [16] [CRITICAL] WORKER TIMEOUT (pid:21)
[2024-02-08 20:00:11 +0100] [21] [INFO] Worker exiting (pid: 21)
[2024-02-08 19:00:11 +0000] [16] [ERROR] Worker (pid:21) exited with code 1
[2024-02-08 19:00:11 +0000] [16] [ERROR] Worker (pid:21) exited with code 1.
[2024-02-08 19:00:11 +0000] [23] [INFO] Booting worker with pid: 23
[2024-02-08 19:01:12 +0000] [16] [ERROR] Worker (pid:23) exited with code 1.
[2024-02-08 19:01:12 +0000] [25] [INFO] Booting worker with pid: 25
[2024-02-08 20:01:13 +0100] [25] [DEBUG] GET /super-admin/campaigns/campaign/
[2024-02-08 18:58:05 +0000] [16] [CRITICAL] WORKER TIMEOUT (pid:17)
[2024-02-08 19:58:05 +0100] [17] [INFO] Worker exiting (pid: 17)
[2024-02-08 18:58:06 +0000] [16] [ERROR] Worker (pid:17) exited with code 1
[2024-02-08 18:58:06 +0000] [16] [ERROR] Worker (pid:17) exited with code 1.
[2024-02-08 18:58:06 +0000] [19] [INFO] Booting worker with pid: 19
[2024-02-08 18:59:09 +0000] [16] [CRITICAL] WORKER TIMEOUT (pid:19)
[2024-02-08 19:59:09 +0100] [19] [INFO] Worker exiting (pid: 19)
[2024-02-08 18:59:09 +0000] [16] [ERROR] Worker (pid:19) exited with code 1
[2024-02-08 18:59:09 +0000] [16] [ERROR] Worker (pid:19) exited with code 1.
[2024-02-08 18:59:09 +0000] [21] [INFO] Booting worker with pid: 21
[2024-02-08 19:00:11 +0000] [16] [CRITICAL] WORKER TIMEOUT (pid:21)
[2024-02-08 20:00:11 +0100] [21] [INFO] Worker exiting (pid: 21)
[2024-02-08 19:00:11 +0000] [16] [ERROR] Worker (pid:21) exited with code 1
[2024-02-08 19:00:11 +0000] [16] [ERROR] Worker (pid:21) exited with code 1.
[2024-02-08 19:00:11 +0000] [23] [INFO] Booting worker with pid: 23
[2024-02-08 19:01:12 +0000] [16] [ERROR] Worker (pid:23) exited with code 1.
[2024-02-08 19:01:12 +0000] [25] [INFO] Booting worker with pid: 25
[2024-02-08 20:01:13 +0100] [25] [DEBUG] GET /super-admin/campaigns/campaign/
The get request only shows after it keeps hanging for a while
25 replies
RRailway
Created by Tim on 2/5/2024 in #✋|help
Django with Gunicorn gives frequent [CRITICAL] WORKER TIMEOUTs on simple requests
Ok, will do that. Do you mean in Gunicorn or in Django or in my postgres service (or all of them) 🤔 ?
25 replies
RRailway
Created by Tim on 2/5/2024 in #✋|help
Django with Gunicorn gives frequent [CRITICAL] WORKER TIMEOUTs on simple requests
I increased it to 5 to be on the safe side, but this won't resolve my timeouts 😅
25 replies
RRailway
Created by Tim on 2/5/2024 in #✋|help
Django with Gunicorn gives frequent [CRITICAL] WORKER TIMEOUTs on simple requests
I am using django-environ
DATABASES = {
"default": env.db("DEFAULT_DB_URL"),
}
DATABASES = {
"default": env.db("DEFAULT_DB_URL"),
}
migrating the database, loading the fixtures, etc. works (I have a sleep 2 on startup to ensure database connection is ready, as recommended in another post). And I can see the data in my admin panel if it does not time out.
25 replies
RRailway
Created by Tim on 2/5/2024 in #✋|help
Django with Gunicorn gives frequent [CRITICAL] WORKER TIMEOUTs on simple requests
I am using
DEFAULT_DB_URL=${{Postgres.DATABASE_PRIVATE_URL}}
DEFAULT_DB_URL=${{Postgres.DATABASE_PRIVATE_URL}}
, and this can't explain why it does work sometimes right?
25 replies
RRailway
Created by Tim on 2/5/2024 in #✋|help
Django with Gunicorn gives frequent [CRITICAL] WORKER TIMEOUTs on simple requests
Update: I now also have requests hanging for 5 minutes before a definitive timeout.
[2024-02-05 14:56:50 +0000] [16] [CRITICAL] WORKER TIMEOUT (pid:33)
[2024-02-05 14:56:50 +0000] [16] [ERROR] Worker (pid:33) exited with code 1
[2024-02-05 14:56:50 +0000] [16] [ERROR] Worker (pid:33) exited with code 1.
[2024-02-05 14:57:52 +0000] [16] [CRITICAL] WORKER TIMEOUT (pid:35)
[2024-02-05 14:57:52 +0000] [16] [ERROR] Worker (pid:35) exited with code 1
[2024-02-05 14:57:52 +0000] [16] [ERROR] Worker (pid:35) exited with code 1.
[2024-02-05 14:58:53 +0000] [16] [CRITICAL] WORKER TIMEOUT (pid:37)
[2024-02-05 14:58:53 +0000] [16] [ERROR] Worker (pid:37) exited with code 1
[2024-02-05 14:58:53 +0000] [16] [ERROR] Worker (pid:37) exited with code 1.
[2024-02-05 14:59:54 +0000] [16] [CRITICAL] WORKER TIMEOUT (pid:39)
[2024-02-05 14:59:55 +0000] [16] [ERROR] Worker (pid:39) exited with code 1
[2024-02-05 14:59:55 +0000] [16] [ERROR] Worker (pid:39) exited with code 1.
[2024-02-05 14:56:50 +0000] [16] [CRITICAL] WORKER TIMEOUT (pid:33)
[2024-02-05 14:56:50 +0000] [16] [ERROR] Worker (pid:33) exited with code 1
[2024-02-05 14:56:50 +0000] [16] [ERROR] Worker (pid:33) exited with code 1.
[2024-02-05 14:57:52 +0000] [16] [CRITICAL] WORKER TIMEOUT (pid:35)
[2024-02-05 14:57:52 +0000] [16] [ERROR] Worker (pid:35) exited with code 1
[2024-02-05 14:57:52 +0000] [16] [ERROR] Worker (pid:35) exited with code 1.
[2024-02-05 14:58:53 +0000] [16] [CRITICAL] WORKER TIMEOUT (pid:37)
[2024-02-05 14:58:53 +0000] [16] [ERROR] Worker (pid:37) exited with code 1
[2024-02-05 14:58:53 +0000] [16] [ERROR] Worker (pid:37) exited with code 1.
[2024-02-05 14:59:54 +0000] [16] [CRITICAL] WORKER TIMEOUT (pid:39)
[2024-02-05 14:59:55 +0000] [16] [ERROR] Worker (pid:39) exited with code 1
[2024-02-05 14:59:55 +0000] [16] [ERROR] Worker (pid:39) exited with code 1.
25 replies
RRailway
Created by Tim on 2/5/2024 in #✋|help
Django with Gunicorn gives frequent [CRITICAL] WORKER TIMEOUTs on simple requests
45fb0a1b-efd2-4699-81c8-16a885d8c33c
25 replies
RRailway
Created by Tim on 12/17/2023 in #✋|help
Logs filled with errors due to django allowed hosts settings
Sure, but it seems undesirable that I either need to disable the allowed hosts security setting or that I need to whitelist arbitrary GCP hosts that pop-up in my logs to prevent my logs being filled with this error.
12 replies
RRailway
Created by Tim on 12/17/2023 in #✋|help
Logs filled with errors due to django allowed hosts settings
As can by seen in my logs, and the linked thread, that is not the case. For example 35.204.95.109
12 replies
RRailway
Created by Tim on 12/17/2023 in #✋|help
Logs filled with errors due to django allowed hosts settings
Do you know if there is a complete list of ips that belong to railway that may be used?
12 replies
RRailway
Created by Tim on 12/17/2023 in #✋|help
Logs filled with errors due to django allowed hosts settings
Sounds like a bad idea (security-wise) to add ip addresses that are unknown to me to the allowed-hosts setting. Why is railway trying to host my app on other domains than the one that is specified?
12 replies
RRailway
Created by Tim on 12/17/2023 in #✋|help
Logs filled with errors due to django allowed hosts settings
a9861a99-1c51-4b34-8970-6e92d5cd7f47
12 replies
RRailway
Created by Tim on 11/25/2023 in #✋|help
NS_ERROR_REDIRECT_LOOP
Hmm it appears to be a consequence of SECURE_HSTS_PRELOAD=True
7 replies
RRailway
Created by Tim on 11/25/2023 in #✋|help
NS_ERROR_REDIRECT_LOOP
No
7 replies