ERROR: could not resize shared memory segment "/PostgreSQL.1432368484" to 12615680 bytes:
Hi,
I am using Postgres database in Railway and I am getting this error. Can someone please help me figure out why this is happening? All of the solutions I see online tells me to change
shm_size of the database. But I don't think there is a way to change shm_size in Railway.
This is only happening for some of the complex or large queries that returns a lot of rows. How to fix this? What can I do to solve this?
Project ID: ef4b4933-5e06-4f9f-93d1-1153cb313e98
Some of the errors I am seeing are these:
ERROR: could not resize shared memory segment "/PostgreSQL.1432368484" to 12615680 bytes: No space left on device
FATAL: terminating background worker "parallel worker" due to administrator command
ERROR: could not resize shared memory segment "/PostgreSQL.1662690888" to 12615680 bytes: No space left on device
background worker "parallel worker" (PID 35856) exited with exit code 1
My usage dashboard says Your plan includes up to 8 GB of RAM, 8 vCPU, and 100 GB of Shared Disk. Manage your Plan.
and I am no where near those limits.Solution:Jump to solution
the default shm size on railway is 64mb and there is no way to increase this
65 Replies
Project ID:
ef4b4933-5e06-4f9f-93d1-1153cb313e98Solution
the default shm size on railway is 64mb and there is no way to increase this
@Brody Yes, I know. Hence this question. Will wait for someone from team to suggest a solution to this.
ive talked to the team about this myself, they don’t seem too interested in this issue, you may need to host the database elsewhere
Wow! Didn't expect that. Ok, then I guess I need to move my database off of Railway!
yeah for your workload you would need to, as this is definitely a blocker they don’t seem too interested in solving unfortunately
Got it. Thanks Brody!
no problem, sorry for the letdown here
Are these any team members who checks the conversations in discord? or is it all community managed?
This is a community forum, you would need to be on Pro in order to get support from the team
but having Pro wouldn't help here as other Pro users have been left in the dark on this issue as well
Not really sure if this answers is satisfying, but I will ask the team directly as well on Slack. Should be as simple as passing a variable to the docker run command.
Yes in theory it's that simple, but they very likely don't have a mechanism in place that would allow them to tack on an extra flag to a specific users container run command
the team has directly told me and another pro user they can't offer custom shm sizes, so unless you are actually enterprise then you may get the same answer I did: you would need to tune postgres to not use as much shared memory
hey all! Brody is correct, you can't pass a flag to your start command, and we hadn't been able to prioritize a fix for this until recently, but I'm happy to report that we do have an environment variable now that enables control over
shm_size. the variable name is RAILWAY_SHM_SIZE_BYTES which takes a value of bytes (as the name implies). let us know if you have any problemsI shall test that with my database!
edit, works perfectly!
@Bhanu Teja P, flagging Melissa's note above which should unblock you by allowing you to control
shm_size using the RAILWAY_SHM_SIZE_BYTES variable.@Christian where do I add this env variable? In my repo which connects to the database? Or should I add this in the env variables of the database?
@Brody where did you add this? And how much did you set it to?
please do not tag the team, you add it to your service variables
Hmm… ok. Can you please share where that is? Service variables of the database?
correct
You can follow these steps:
1. Click on your database on the canvas
2. In the service settings pane, click Variables from the top menu
3. Click
New Variable
4. Enter VARIABLE_NAME = shm_size // VALUE = your custom shm size
5. Click AddThanks Christian. Is there a way to directly reach out to your team when needing customer support instead of posting it on a community managed forum. Previously, it used to be so simple to contact the team via discord. Now it looks like my ticket automatically got marked as resolved without the team even seeing it
^
Yeah, but you said don't take pro as they would say the same thing. If I would have listened to that, I would have migrated to another service
I can only provide the information I am aware of, I did not know they were going to add this variable
the statement is true regardless
Hmm... ok, thanks.
I set the value to
512000000 (assuming this means 512 MB)
Hoping this will solve the problemplease don't set it anywhere near that high, as a hobby user that would likely be far above the limit for shm sizes on the hobby plan when they implement the limits
set it to 128MB, or 134217728 in bytes
I tried 128 MB, but it's still not enough for my usecase
the error above states you only went 12mb out of memory
I have no idea. I tried 128 MB, but it still throws the same error. I set it to 512 MB, and no error
let me set it back to 128 MB again and retry it
we may set the default to 160mb so try that
160 MB didn't work
Trying 256 MB next
please provide the errors
Same error
yeah same error, but the number would change
It didn't change
My guess is there are multiple of these queries running in parallel, and together combined is more than 160 MB
otherwise, it would not have worked for 512 MB too
this is starting to sound not like a hobby usecase if I'm being honest
what's a hobby usecase?
Setting it to 256 MB worked. I am going to leave it at this.
that's good to hear
Thanks for all the help @Brody
no problem
Now, I don't have to write a migration script anymore 😅
I'll preemptively say this, when / if you use your 5gb volume up on the database, you will need to upgrade to pro
Where do I check this?
click on the volume attached to postgres
not even close to where you should be looking haha
This?
yep, looks like you are soon going to hit the limit soon, once that happens your database becomes inaccessible
In Pro, do I get a higher size? How much higher?
pro users get 50gb
Nice! Will upgrade to Pro
once on pro don't feel bad if you need to set your shm size higher
As soon as I upgraded to Pro, I see this
have you moved the project over to the pro workspace?
yeah
what do I do?
please confirm that the project is now located in the pro workspace
It is
show me what's in the drop-down for the workspace selector on the left of this screenshot
interesting, can you quickly check your transactions for the card you used to upgrade, where you charged $20?
Checking
There was an unpaid invoice. Paid it. Now, it's back to normal.
Any idea why this would still show 5 GB?
you would need to grow it, it's not done automatically as growing the volume means downtime, so it's left to the users discretion, you can grow it when you want to in it's settings menu
Understood. Again, thanks for all the help!
no problem