Railway•4mo ago

rust deploy uses 10x more ram than local dev

hey guys, i have a rust api that is using 4-8gb of ram suddenly, now for rust that's unheard of considering it used to chill at 200mb +/- when running cargo run --release --package x locally i get out to a max of 1.5gb ram (this includes compiling) yet on railway it ticks out to 8gb at peak (this includes compiling) takes roughly 26 minutes to compile, build, deploy on railway There's a total of 950 +/- crate dependencies projectId = 7c6a1d9f-d3d2-4808-96b1-10c8df175503 No deploy that happened in that weird spike, nor am i able to reproduce the low memory usage anymore

132 Replies

Percy•4mo ago

Project ID: 7c6a1d9f-d3d2-4808-96b1-10c8df175503

Brody•4mo ago

dockerfile or nixpacks?

Floris•4mo ago

nixpacks

// nixpacks.toml
providers = ['rust']

[phases.build]
cmds = ['echo building xylex image', 'cargo build --package xylex']

[phases.test]
cmds = ["cargo test --package xylex"]
dependsOn = ["build"]

[start]
cmd = 'cargo run --package xylex'

[variables]
NIXPACKS_INSTALL_CACHE_DIRS = './nixpacks_cache'
NIXPACKS_BUILD_CACHE_DIRS = './nixpacks_cache'

// nixpacks.toml
providers = ['rust']

[phases.build]
cmds = ['echo building xylex image', 'cargo build --package xylex']

[phases.test]
cmds = ["cargo test --package xylex"]
dependsOn = ["build"]

[start]
cmd = 'cargo run --package xylex'

[variables]
NIXPACKS_INSTALL_CACHE_DIRS = './nixpacks_cache'
NIXPACKS_BUILD_CACHE_DIRS = './nixpacks_cache'

Brody•4mo ago

you should not do cargo run as the start command, you should be building a release and then running the resulting binary

Floris•4mo ago

i had a cargo run --release but i just changed it a few min ago

Brody•4mo ago

let's see your new nixpacks.toml file please additionally, @milo / salvage your input here would be greatly appreciated

Floris•4mo ago

// nixpacks.toml
providers = ['rust']

[phases.build]
cmds = ['echo building xylex image', 'cargo build --release --package xylex']

[phases.test]
cmds = ["cargo test --package xylex"]
dependsOn = ["build"]

[start]
cmd = './target/release/xylex'

[variables]
NIXPACKS_INSTALL_CACHE_DIRS = './nixpacks_cache'
NIXPACKS_BUILD_CACHE_DIRS = './nixpacks_cache'

// nixpacks.toml
providers = ['rust']

[phases.build]
cmds = ['echo building xylex image', 'cargo build --release --package xylex']

[phases.test]
cmds = ["cargo test --package xylex"]
dependsOn = ["build"]

[start]
cmd = './target/release/xylex'

[variables]
NIXPACKS_INSTALL_CACHE_DIRS = './nixpacks_cache'
NIXPACKS_BUILD_CACHE_DIRS = './nixpacks_cache'

Brody•4mo ago

yeah that looks appropriate now

Floris•4mo ago

yeah well i used to do it like this in the start but had tremendous amount of issues as you might remember it would just brick and never deploy unless done via the cli

Brody•4mo ago

I do remember, did you also have those variables set? I don't see any purpose to them and can see them causing issues

Floris•4mo ago

yes no i hadn't set them yet this is just me trying to get it working in a normal way

Brody•4mo ago

for good measure, remove them please

Floris•4mo ago

the nixpacks_cache ?

Brody•4mo ago

and then let me know if you run into further issues yes, both those variables you have there

Floris•4mo ago

okay, gotcha 2 min the target could be up to 20gb in some weird c folder hence i deferred it to the cache folder

Brody•4mo ago

I'm assuming the default cache stuff nixpacks does is sufficient

Floris•4mo ago

compiling rn same issue as before on railway and on nixpacks locally

Floris•4mo ago

local

Brody•4mo ago

just freezes there?

Floris•4mo ago

yes

milo•4mo ago

god

Floris•4mo ago

i love railway

milo•4mo ago

the rust provider in nixpacks honestly is shit

Floris•4mo ago

but rust has been very painful to work with on railway

milo•4mo ago

yeah i've moved to cargo-chef in a dockerfile

Floris•4mo ago

i was reading about that yesterday

Brody•4mo ago

haha milo beat me to it

milo•4mo ago

much better then nixpacks

Brody•4mo ago

agree with Milo here

milo•4mo ago

also this issue looks unrelated to nixpacks

Floris•4mo ago

right okay, so you advise me to do the same im not sure, it stalls local and on railway

milo•4mo ago

nixpacks or

Floris•4mo ago

yes

milo•4mo ago

okay nevermind this is a nixpacks issue 😂

Floris•4mo ago

milo•4mo ago

im not sure why it is making it stall that bad though

Brody•4mo ago

yes, I'm not a rust dev but have heard great things about cargo chef and when nixpacks is painful (almost always) dockerfiles are the way to go

Floris•4mo ago

we still chilling here

milo•4mo ago

from yours truly

Floris•4mo ago

yeah it was the exact same issue months ago somehow the cli fixed it

milo•4mo ago

just use cargo-chef, see if that helps

Floris•4mo ago

omw

Brody•4mo ago

yep cargo chef in a dockerfile for sure

milo•4mo ago

ping me if this happens again (or get brody to lol)

Floris•4mo ago

yeah as funnily enough we went to rust cus of low resource usage whahaha

milo•4mo ago

no you dont need to install it

milo•4mo ago

please read up how to use it:https://github.com/LukeMathWalker/cargo-chef

GitHub

GitHub - LukeMathWalker/cargo-chef: A cargo-subcommand to speed up ...

A cargo-subcommand to speed up Rust Docker builds using Docker layer caching. - LukeMathWalker/cargo-chef

Floris•4mo ago

you just make the recipe.json ? i am

milo•4mo ago

you do all that in the dockerfile

Floris•4mo ago

ah i see

milo•4mo ago

jsut look at the example

milo•4mo ago

this bit

Floris•4mo ago

should be able to copy most of that

milo•4mo ago

yup

Floris•4mo ago

milo•4mo ago

delete ur nixpacks.toml make sure the Dockerfile is named Dockerfile

Floris•4mo ago

yes wait im showing u something

Floris•4mo ago

milo•4mo ago

thats just nixpacks doing extra stuff dont worry about that please delete ur nixpacks,toml

Brody•4mo ago

you somehow needed up with two rust providers but yeah, Dockerfile time

Floris•4mo ago

well i think it's a bug in NixPacks when cargo workspaces are enabled as xylex_wss is not the main bin it starts building a random workspace and the core bin

milo•4mo ago

nixpacks and rust dont mix unfortunately

Floris•4mo ago

going to docker now ig

milo•4mo ago

yup

Floris•4mo ago

fucking hate docker time consuming

milo•4mo ago

not really this will be easy

Brody•4mo ago

yeah you won't need any kind of complex Dockerfile

Floris•4mo ago

okay so i changed these from app to xylex

Floris•4mo ago

anything besides that ? my cargo is called xylex

milo•4mo ago

what is the binary u wanna okay cool should be good

Floris•4mo ago

railway autodetects this yes

milo•4mo ago

yes

Floris•4mo ago

milo•4mo ago

let us know how it goes

Brody•4mo ago

and please cancel those old deployments

Floris•4mo ago

whahah yes i will

Floris•4mo ago

i am guessing one of these should've been app still

Brody•4mo ago

I assume that means you need to install openssl

Floris•4mo ago

bros too experienced

milo•4mo ago

RUN apt update -y && apt install openssl libssl-dev -y put that in the line before the ENTRYPOINT instruction

Brody•4mo ago

perfect

Floris•4mo ago

man idk where milo came from but they needa give him a raise bro's got shortcuts to paste solutions

Brody•4mo ago

Milo is the rust professional around here

Floris•4mo ago

what a king

Brody•4mo ago

milo•4mo ago

just from my personal rust app dockerfile haha

Floris•4mo ago

are you in the railway dev team milo

Brody•4mo ago

he's not, just a community mod

Floris•4mo ago

i see, so it is personal projects/companies that u used rust in

milo•4mo ago

yeah, some enterprise things as well rust is really good and scalable in my experience

Floris•4mo ago

yes we switched in january it is amazing

milo•4mo ago

haha

Floris•4mo ago

i am unsure if the ssl install was the right call certificate verify failed"

milo•4mo ago

:anguish: what dependencies are u using

Brody•4mo ago

might need ca-certificates too? just a guess

Floris•4mo ago

shouldn't matter it deploys fine on ubuntu server

milo•4mo ago

can you send ur cargo.toml

Floris•4mo ago

yes

Brody•4mo ago

I mean this is not the same at all though

milo•4mo ago

are you calling reqwest directly

Floris•4mo ago

alright brace for impact guys

// cargo.toml 
[package]
name = "xylex"
version = "0.1.0"
edition = "2021"


[dependencies]
polars = {version = "0.35.0", features = ["lazy", "rolling_window"] }
reqwest = { version = "0.11.23", features = ["json", "blocking"] }
tokio = { version = "1.35.1", features = ["full", "macros", "rt-multi-thread"] }
chrono = { version = "0.4", features = ["serde"] }
serde  = { version = "1.0.196", features = ["derive"] }
supabase_rs = "0.2.2"
thiserror = "1"
rocket = "0.5.0"
serde_json = "1.0.111"
dotenv = "0.15.0"
mail-send = "0.4.6"
mail-builder = "0.3.1"
rsi = "0.1.2"
csv = "1.1"
arrow = "*"
serenity = { version = "0.12", default-features = false, features = ["client", "gateway", "rustls_backend", "model", "collector"] }
clokwerk = "0.4.0"
image = "0.24.8"
rusttype = "0.9.3"
rand = "0.8.5"
qrcode = "0.13"
async-trait = "0.1.77"
futures = "0.3.3"
imageproc = "0.23.0"
tokio-tungstenite = { version = "0.21.0", features = ["native-tls"] }
serde_yaml = "0.9.31"
lazy_static = "1.4.0"
trading_sessions = "0.1.2"
md-5 = "0.10.6"
hex-literal = "0.4.1"
fibonacci_retracement = "0.1.0"
futures-util = "0.3.30"
futures-channel = "0.3.30"
url = "2.5.0"
sha256 = "1.5.0"
strum = "0.26"
strum_macros = "0.26"
sled = "0.34.7"

xylex_spread_tracker = { path = "./xylex_spread_tracker" }
xylex_card_generator = { path = "./xylex_card_generator" }
xylex_hash_router = { path = "./xylex_hash_router" }
xylex_scheduler = { path = "./xylex_scheduler" }
xylex_discord = { path = "./xylex_discord"}

[dev-dependencies]
anyhow = "1"

[profile.dev]
opt-level = 0
incremental = true


[profile.release]
opt-level = 3

[workspace]
members = [
    "xylex_wss",
    "xylex_card_generator",
    "xylex_spread_tracker",
    "xylex_scheduler",
    "xylex_hash_router",
    "xylex_discord"
]

// cargo.toml 
[package]
name = "xylex"
version = "0.1.0"
edition = "2021"


[dependencies]
polars = {version = "0.35.0", features = ["lazy", "rolling_window"] }
reqwest = { version = "0.11.23", features = ["json", "blocking"] }
tokio = { version = "1.35.1", features = ["full", "macros", "rt-multi-thread"] }
chrono = { version = "0.4", features = ["serde"] }
serde  = { version = "1.0.196", features = ["derive"] }
supabase_rs = "0.2.2"
thiserror = "1"
rocket = "0.5.0"
serde_json = "1.0.111"
dotenv = "0.15.0"
mail-send = "0.4.6"
mail-builder = "0.3.1"
rsi = "0.1.2"
csv = "1.1"
arrow = "*"
serenity = { version = "0.12", default-features = false, features = ["client", "gateway", "rustls_backend", "model", "collector"] }
clokwerk = "0.4.0"
image = "0.24.8"
rusttype = "0.9.3"
rand = "0.8.5"
qrcode = "0.13"
async-trait = "0.1.77"
futures = "0.3.3"
imageproc = "0.23.0"
tokio-tungstenite = { version = "0.21.0", features = ["native-tls"] }
serde_yaml = "0.9.31"
lazy_static = "1.4.0"
trading_sessions = "0.1.2"
md-5 = "0.10.6"
hex-literal = "0.4.1"
fibonacci_retracement = "0.1.0"
futures-util = "0.3.30"
futures-channel = "0.3.30"
url = "2.5.0"
sha256 = "1.5.0"
strum = "0.26"
strum_macros = "0.26"
sled = "0.34.7"

xylex_spread_tracker = { path = "./xylex_spread_tracker" }
xylex_card_generator = { path = "./xylex_card_generator" }
xylex_hash_router = { path = "./xylex_hash_router" }
xylex_scheduler = { path = "./xylex_scheduler" }
xylex_discord = { path = "./xylex_discord"}

[dev-dependencies]
anyhow = "1"

[profile.dev]
opt-level = 0
incremental = true


[profile.release]
opt-level = 3

[workspace]
members = [
    "xylex_wss",
    "xylex_card_generator",
    "xylex_spread_tracker",
    "xylex_scheduler",
    "xylex_hash_router",
    "xylex_discord"
]

[package]
name = "xylex_card_generator"
version = "0.1.0"
edition = "2021"

[dependencies]
image = "0.25.0"
imageproc = "0.24.0"
reqwest = "0.12.2"
rusttype = "0.9.3"
serde = "1.0.197"
serde_json = "1.0.114"
serde_yaml = "0.9.34"

[package]
name = "xylex_card_generator"
version = "0.1.0"
edition = "2021"

[dependencies]
image = "0.25.0"
imageproc = "0.24.0"
reqwest = "0.12.2"
rusttype = "0.9.3"
serde = "1.0.197"
serde_json = "1.0.114"
serde_yaml = "0.9.34"

[package]
name = "xylex_discord"
version = "0.1.0"
edition = "2021"

[dependencies]
serenity = { version = "0.12", default-features = false, features = ["client", "gateway", "rustls_backend", "model", "collector"] }
tokio = { version = "1.35.1", features = ["full", "macros", "rt-multi-thread"] }
chrono = { version = "0.4", features = ["serde"] }
serde  = { version = "1.0.196", features = ["derive"] }
thiserror = "1"
serde_json = "1.0.111"
dotenv = "0.15.0"
serde_yaml = "0.9.31"
reqwest = "0.12.2"

[dev-dependencies]
anyhow = "1"

[profile.dev]
opt-level = 0
incremental = true

[profile.release]
opt-level = 3

[package]
name = "xylex_discord"
version = "0.1.0"
edition = "2021"

[dependencies]
serenity = { version = "0.12", default-features = false, features = ["client", "gateway", "rustls_backend", "model", "collector"] }
tokio = { version = "1.35.1", features = ["full", "macros", "rt-multi-thread"] }
chrono = { version = "0.4", features = ["serde"] }
serde  = { version = "1.0.196", features = ["derive"] }
thiserror = "1"
serde_json = "1.0.111"
dotenv = "0.15.0"
serde_yaml = "0.9.31"
reqwest = "0.12.2"

[dev-dependencies]
anyhow = "1"

[profile.dev]
opt-level = 0
incremental = true

[profile.release]
opt-level = 3

[package]
name = "xylex_hash_router"
version = "0.1.0"
edition = "2021"

[dependencies]
dotenv = "0.15.0"
serde_json = "1.0.115"
supabase_rs = "0.2.2"

[package]
name = "xylex_hash_router"
version = "0.1.0"
edition = "2021"

[dependencies]
dotenv = "0.15.0"
serde_json = "1.0.115"
supabase_rs = "0.2.2"

[package]
name = "xylex_spread_tracker"
version = "0.1.0"
edition = "2021"

[dependencies]
regex = "1.10.4"
reqwest = "0.12.2"
serde = "1.0.197"
serde_derive = "1.0.197"
serde_json = "1.0.115"
serde_yaml = "0.9.34"
supabase_rs = "0.2.2"
tokio = "1.36.0"

[package]
name = "xylex_spread_tracker"
version = "0.1.0"
edition = "2021"

[dependencies]
regex = "1.10.4"
reqwest = "0.12.2"
serde = "1.0.197"
serde_derive = "1.0.197"
serde_json = "1.0.115"
serde_yaml = "0.9.34"
supabase_rs = "0.2.2"
tokio = "1.36.0"

[package]
name = "xylex_wss"
version = "0.1.0"
edition = "2021"

[dependencies]
amqprs = "1.5.3"
serde = "1.0.197"
serde_json = "1.0.114"
serde_yaml = "0.9.34"
tokio =  "1.36.0"

[package]
name = "xylex_wss"
version = "0.1.0"
edition = "2021"

[dependencies]
amqprs = "1.5.3"
serde = "1.0.197"
serde_json = "1.0.114"
serde_yaml = "0.9.34"
tokio =  "1.36.0"

Brody•4mo ago

you can attach files lmao

Floris•4mo ago

oh well this works too

milo•4mo ago

add rustls-tls as a feature of reqwest also, half these depenendices are obsolete :anguish: and can be replaced by something better, not needed at all, or stdlib

Floris•4mo ago

i mean bro im sure but i'm not sure if refactoring 155k lines of rust is something i wanna do tonight

milo•4mo ago

i mean, you don't need strum-macros if you have strum, lazy-static is just stupid these days, dotenv should use dotenvy and thats just to name a few anyways do this please

Floris•4mo ago

yes its deploying

milo•4mo ago

okay cool

Brody•4mo ago

damn Milo knows his stuff

Floris•4mo ago

yeah but what's the difference between dotenv and dotenvy

milo•4mo ago

dotenvy is maintained and isn't.. bad

milo•4mo ago

https://rustsec.org/advisories/RUSTSEC-2021-0141.html

RUSTSEC-2021-0141: dotenv: dotenv is Unmaintained › RustSec Advisor...

Security advisory database for Rust crates published through https://crates.io

Floris•4mo ago

i see ill make a linear ticket ig

milo•4mo ago

same api, literally just changing crate name anyways!

Floris•4mo ago

same issue lmfao

milo•4mo ago

screenshot of error

Floris•4mo ago

"certificate verify failed", file

milo•4mo ago

well then i dont know why show me the line of code that is causing this erorr

Floris•4mo ago

every single one that does an api request

Brody•4mo ago

maybe?

milo•4mo ago

show me please

Floris•4mo ago

pub async fn discord_logs(
    log_message: &str
) -> Result<(), Box<dyn Error>> {
    let client: Client = Client::new();
    let webhook_url: String = env::var("DISCORD_WEBHOOK_URL").unwrap();
    let current_time: String = Utc::now().to_string();

    client.post(&webhook_url)
        .body(json!({
            "content": format!(
                "**xylex-daemon @ {} | **{}",
                current_time, log_message
            )

        }).to_string())
        .header(
            "Content-Type",
             "application/json"
        )
        .send()
        .await?;

    Ok(())
}

pub async fn discord_logs(
    log_message: &str
) -> Result<(), Box<dyn Error>> {
    let client: Client = Client::new();
    let webhook_url: String = env::var("DISCORD_WEBHOOK_URL").unwrap();
    let current_time: String = Utc::now().to_string();

    client.post(&webhook_url)
        .body(json!({
            "content": format!(
                "**xylex-daemon @ {} | **{}",
                current_time, log_message
            )

        }).to_string())
        .header(
            "Content-Type",
             "application/json"
        )
        .send()
        .await?;

    Ok(())
}

all this shouldn't matter as it works fine on any other deploy it's just railway can't build it unless it's 4-8gb ram

milo•4mo ago

well thats norrmal about high ram amounts yeah no clue, you aren't directly telling it to use openssl and the features make it use rustls so im lost sorry

Floris•4mo ago

ig so but only 10-20k lines r being deployed on railway and idle usage used to be 200mb or so

milo•4mo ago

sorry idk

Floris•4mo ago

thanks for the help though guys, i appreciate your time have a good night

Gaming

Programming

rust deploy uses 10x more ram than local dev