Railway•2y ago

BullMQ (with Redis) on Railway keeps getting ECONNREFUSED errors when running locally

I used the BullMQ Template that is offered by Railway, which spun up a node codebase for BullMQ and a redis server. I created a local version of the environment as well. When I had a lot of jobs queued (maybe 100s), if I start the bullmq service locally it processes a few and then starts to get the following error message over and over - I also can't load the BullMQ UI. [[ IP ADDRESS REMOVED BELOW]]

Error: connect ECONNREFUSED ***{{IP ADDRESS}}***
    at TCPConnectWrap.afterConnect [as oncomplete] (node:net:1487:16) {
  errno: -61,
  code: 'ECONNREFUSED',
  syscall: 'connect',
  address: {{IP ADDRESS}},
  port: 6541
}

Error: connect ECONNREFUSED ***{{IP ADDRESS}}***
    at TCPConnectWrap.afterConnect [as oncomplete] (node:net:1487:16) {
  errno: -61,
  code: 'ECONNREFUSED',
  syscall: 'connect',
  address: {{IP ADDRESS}},
  port: 6541
}

Any idea what's going on here? I would guess that Railway is stopping my connections from the local service, or redis is actually going down for some reason?

26 Replies

Percy•2y ago

Project ID: aa1a93fc-4414-42ac-9398-928082181803

Percy•2y ago

You might find these helpful: - I cannot connect to Redis - read ECONNRESET on my database - Can't connect to Postgres DB anymore

⚠️ experimental feature

niconico6710OP•2y ago

aa1a93fc-4414-42ac-9398-928082181803 What powers this "You might find these helpful" service? Hey! I wanted to follow up here to see if anyone had any recommendations on how I might debug this? I'm going to deploy to production and see if I hit similar issues, but wanted to see if this is a known Railway limitation before I explore changing implementation approaches

niconico6710OP•2y ago

Hey team, just to follow up here so that the thread doesn't get left dangling. What I did so far in trying to confirm the source of the ECONNREFUSED errors is to deploy to production and see if the running Railway service based on my Github repo has issue connecting to the production Redis instance running on Railway. At first, it didn't seem to have any issue connecting, but then as it continued to run, and I did another deploy to make my DB connection a shared singleton, I started seeing issues:

angelo•2y ago

all good, it appears that Discord running into a Slash command issue means that we couldn't raise the sev of the issue I am tagging in @neb :) who can assist here

niconico6710OP•2y ago

Thanks Angelo

angelo•2y ago

The ERRCON reset is from the Redis?

niconico6710OP•2y ago

Presumably

angelo•2y ago

I am looking in: aa1a93fc-4414-42ac-9398-928082181803 Interesting

niconico6710OP•2y ago

I don’t know what else it could be from. I do connect to a Postgres db on AWS but I don’t think that’s it I was also connecting to a local db when trying it loacally It works for a while then starts to econnrefused as I load jobs into bullmq

angelo•2y ago

niconico6710OP•2y ago

I am using the template service y’all had for it

angelo•2y ago

I got it Yea

niconico6710OP•2y ago

I’m afk, will be back at my laptop in like 10 min, so might not be able to help on some issues I’m also happy to hop on a call to debug together and share screen if helpful Ah it looks like the next available time is a few days away

angelo•2y ago

whoops

niconico6710OP•2y ago

I need to deploy this service today, or else move off railway

angelo•2y ago

throw any time on [email protected] today will clear the board for ya

niconico6710OP•2y ago

I’ll be back in 15 minutes at mine - would that work?

angelo•2y ago

that works!

niconico6710OP•2y ago

Will ping you shortly, and thanks! 👋 hey @Angelo I'm free when you are - let me know what the best way to hop on would be

angelo•2y ago

cool, spinning up a meet

niconico6710OP•2y ago

hey none sec, not sure what's happening with Google Meet It's stuck in Getting Ready... Can I send you a Zoom?

angelo•2y ago

sure thing! Updating: seems to be a internal netoworking issue with our VMs and TCP. Working on a long term fix.

rgmvisser•2y ago

@Angelo I am also running into this issue, do you mind updating me as well when this is/will be fixed? 🙂

angelo•2y ago

#🛂｜readme - the Team isn’t online on the weekends and please don’t tag the team I can look into it first thing my Monday

rgmvisser•2y ago

Great thanks! FWIW my issue is fixed, I think it was something unrelated.

Gaming

Programming

BullMQ (with Redis) on Railway keeps getting ECONNREFUSED errors when running locally