Hmmm interesting. I'm using Avada website builder on Wordpress and nothing else just yet. Maybe it's building out themes and whatnot that are spiking memory usage. I'll keep trying.
Ok so if anyone ever comes across this thread, the headache I was having was confusing a 'nixpacks.toml' with the 'railway.toml'. Put the build version in railway and keep your aptPkgs in nixpacks.