Railway•6mo ago

Excessive vCPU and memory on Django boot

I have a django service that is using an excessive amount of resources on startup. You can see the start of that realization https://discord.com/channels/713503345364697088/1250987443918995466/1251185056949735535 Ignoring the memory for a second (unless it isn't possible to ignore) why is the vCPU so high? The service is idle. Nothing is hitting it. The running script is gunicorn BBWare.wsgi -b [::]:${PORT:-8000}. This is built with a dockerfile. So I guess a few things. A) what is normal expected computer consumption for an idle django app. Is maybe the issue simply that I am require more ram and that limit is causing some type of compute overload and really there isn't an issue at all, I just need to put money into this account? To that end, how would I best test this through local dev? IE, run the script locally and observe how much resources the idle image is consuming

41 Replies

Percy•6mo ago

Project ID: 40943c2e-d9a8-4f3c-b414-73ea73eccd8f

JoshieOP•6mo ago

40943c2e-d9a8-4f3c-b414-73ea73eccd8f

Brody•6mo ago

A) a lot less then that

JoshieOP•6mo ago

well,,, I went to go test by deploying the blank django template 😄

Brody•6mo ago

railway might not think you're verifiable, in the top right does it tell you that you are on a limited trial?

JoshieOP•6mo ago

Brody•6mo ago

hmmmm that's the regular trial plan indeed

JoshieOP•6mo ago

Are you only allowed one project on the trail plan?

Brody•6mo ago

nope, I don't remember the limit but it's more than one that's for sure, but I can promise you the django template does not use anywhere near the amount of CPU/memory your screenshot shows

JoshieOP•6mo ago

It fails to create that second project. But it shows up on the dash. However, it is blank. But it seems that the django-server is showing up in the danger panel? This is all kinds of messed up The project id for the screen shots: 8eca756c-48e3-4e8d-8bb1-77562165bf2b

JoshieOP•6mo ago

Yea, I would think. To be fair, the original screenshot is not a blank project like the template. But I can't even begin testing if this stays broken :shrug: I can't start slowly adding on little components like this

Brody•6mo ago

that's a separate issue, let's come back to that you said you had a gunicorn config, perhaps it's using it automatically? could you temporarily remove it?

JoshieOP•6mo ago

I will try that in a sec. But I noticed something about the metrics. There are no deployments but yet it is still showing excessie usage. Is this just how the metrics are calculated / displayed?

Brody•6mo ago

are they updating? they might simply be frozen because there are no new metrics points coming in

JoshieOP•6mo ago

If I had to guess, it is the lack of updating. But the metrics panel doesn't really show the desecrate points. So I have no clear way of telling

Brody•6mo ago

yeah it's the same for when an app sleeps, the graph just freezes in place instead of padding with zeros

JoshieOP•6mo ago

Well ,,, :shrug: it works now. These metrics are jumping up and down a lot. The memory is way down. The amount I would expect.

Brody•6mo ago

was this because you removed the gunicorn config?

JoshieOP•6mo ago

Yes. But that doesn't make perfect sense becuase I wan't calling it in my command. Unless gunicorn just grabs it without you specifying it :squint:. Might add the file back in and see if that is the case. Would be kinda strange kinda not though

Brody•6mo ago

id believe it, care to share the config file though?

JoshieOP•6mo ago

The entrypoint file

#!/bin/sh

set -ex \
    && python manage.py migrate --noinput \
    && python manage.py collectstatic --noinput

if [ "$DEBUG" = "False" ]
then
    gunicorn BBWare.wsgi -b [::]:${PORT:-8000}
    # gunicorn --config ./gunicorn.conf.py BBWare.wsgi -b [::]:${PORT:-8000}
else
    python manage.py runserver 0.0.0.0:${PORT:-8000}
fi

#!/bin/sh

set -ex \
    && python manage.py migrate --noinput \
    && python manage.py collectstatic --noinput

if [ "$DEBUG" = "False" ]
then
    gunicorn BBWare.wsgi -b [::]:${PORT:-8000}
    # gunicorn --config ./gunicorn.conf.py BBWare.wsgi -b [::]:${PORT:-8000}
else
    python manage.py runserver 0.0.0.0:${PORT:-8000}
fi

The config file

import multiprocessing

workers = multiprocessing.cpu_count() * 2 + 1

import multiprocessing

workers = multiprocessing.cpu_count() * 2 + 1

Super complex config file

Brody•6mo ago

yeah that's going to spawn 65 workers cpu_count reports 32 CPUs when on railway at least, bug with the platform

JoshieOP•6mo ago

With that one solved; onto the other issues I have. I guess the two would be the one I mentioned above with the django template not wanting to build And the other is related to not being able to access my project. I am using Caddy. I am sure I did something wrong for how to actually expose the endpoints. But I will make a new thread for the access stuff. Should we just continue on this one for the not being able to make a blank django template?

Brody•6mo ago

let's worry about getting your own project running before we worry about the template, since that's what matters more here so caddy? what's that being used for?

JoshieOP•6mo ago

Reverse proxy. Mostly following // basing it off of this one cool guys stuff: https://github.com/railwayapp-templates/caddy-reverse-proxy But I am confused about a few parts. First, what is the purpose of the Auth Proxy?

Brody•6mo ago

the auth proxy is something I threw together for a user who wanted to add basic auth to his service that didn't offer a way to secure itself

JoshieOP•6mo ago

Second, none of the services seem to be exposed? And yet the sites are accessible?

Brody•6mo ago

the caddy proxy communicates to them exclusively over the private network

JoshieOP•6mo ago

Sure, but where is it being expossed finally? That is what I am missing How does that result in someone being able to access the page

Brody•6mo ago

they are being exposed via the mysite - caddyproxy this setup turns something like - domain.com api.domain.com into - domain.com domain.com/api

JoshieOP•6mo ago

Unless the view just doesn't show me everything (becuase public but I don't have perms or something) I don't actually see where you are expossing anything through the caddyproxy?

Brody•6mo ago

though it's public domain it's not shown for some reason above the deployment, but you can see the domain on the service card itself

JoshieOP•6mo ago

Because for me, I am enableing the network for that caddy service. But then nothing ,,, going to that site doesn't actually work. Wait, let me enable debug mode on the caddy file and see if I can see anything in the logs

JoshieOP•6mo ago

Yea. I just wasn't sure where that was actually coming from. But ok, that does make sense

Brody•6mo ago

do you get application failed to respond on your domain?

JoshieOP•6mo ago

Yea

Brody•6mo ago

send me your caddyfile please

JoshieOP•6mo ago

Maybe I am doing port wrong :shrug: I just put debug on and I don't see any changes in the log :shrug:

# Proxy file for connecting services through railway.app
{
    debug
    admin off # theres no need for the admin api in railway's environment
    persist_config off # storage isn't persistent anyway
    auto_https off # railway handles https for us, this could in some cases cause issues if left enabled
    log {
        format json # set runtime log format to json mode
    }
    # server options
    servers {
        trusted_proxies static private_ranges # trust railway's proxy
    }
}

{$PORT} {
    log {
        format json # set access log format to json mode
    }

    encode zstd gzip

    handle_path /ingest* {
        rewrite * {path}
        reverse_proxy https://us.i.posthog.com:443 {
          header_up Host us.i.posthog.com
          header_down -Access-Control-Allow-Origin
        }
    }

    @django path /admin* /api* /accounts* /static/*
    handle @django {
        reverse_proxy {$BACKEND_HOST}
    }


    handle {
    reverse_proxy {$FRONTEND_HOST}
    }
}

# Proxy file for connecting services through railway.app
{
    debug
    admin off # theres no need for the admin api in railway's environment
    persist_config off # storage isn't persistent anyway
    auto_https off # railway handles https for us, this could in some cases cause issues if left enabled
    log {
        format json # set runtime log format to json mode
    }
    # server options
    servers {
        trusted_proxies static private_ranges # trust railway's proxy
    }
}

{$PORT} {
    log {
        format json # set access log format to json mode
    }

    encode zstd gzip

    handle_path /ingest* {
        rewrite * {path}
        reverse_proxy https://us.i.posthog.com:443 {
          header_up Host us.i.posthog.com
          header_down -Access-Control-Allow-Origin
        }
    }

    @django path /admin* /api* /accounts* /static/*
    handle @django {
        reverse_proxy {$BACKEND_HOST}
    }


    handle {
    reverse_proxy {$FRONTEND_HOST}
    }
}

Brody•6mo ago

you are missing the colon

JoshieOP•6mo ago

It worked! I love that so much. well it kinda worked. There are some other issues. The frontend seems to be working just fine. But not the backend. So I will need to investigate that after I have something to eat. I will be back with more questions Thank you so much with all the help so far

Brody•6mo ago

no problem!

Gaming

Programming

Excessive vCPU and memory on Django boot