Railway•13mo ago

getaddrinfo EAI_AGAIN

Hello, recently sometimes I have started to see 1-2 errors / day about this error: getaddrinfo EAI_AGAIN to a whole different set of addresses which makes me think it's not external issue. Examples: ERROR: The DNS server returned an error, perhaps the server is offline getaddrinfo EAI_AGAIN eu-central.storage.cloudconvert.com ERROR: The DNS server returned an error, perhaps the server is offline getaddrinfo EAI_AGAIN www.googleapis.com I think the proxy/dns settings are managed directly by railway so I don't know how could debug this? As said everyday I receive around 10k webhook, and I send 10k http requests to external APIs where 1-5 errors / day happen with this same error. Ideas? Thanks.

19 Replies

Percy•13mo ago

Project ID: 40e44f3c-0508-45c6-b255-a4558921d6f5

ENT3I <3OP•13mo ago

40e44f3c-0508-45c6-b255-a4558921d6f5

Brody•13mo ago

is there way to log the dns server in use during each of these 10k requests? perhaps passing in a custom lookup config that logs the dns server that was used for that request

ENT3I <3OP•13mo ago

No idea, my dns is hosted by godaddy which is where I have added the cname that Railway gave me

Brody•13mo ago

haha not what I meant at all

ENT3I <3OP•13mo ago

you think I could see that in Godaddy?

Brody•13mo ago

dns resolver

ENT3I <3OP•13mo ago

ah but since the connections (ip, dns, etc) are managed by Railway? How can I see that?

Brody•13mo ago

they aren't really managed by railway, they do add two nameservers to your container though. - 8.8.8.8 for public dns lookups - fd12::10 for private dns lookups, will use 8.8.8.8 under the hood as an upstream resolver I'm wondering what dns resolver your code uses for a lookup when these dns requests are failing with the EAI_AGAIN error

ENT3I <3OP•13mo ago

This is the error:

{
    "status": "rejected",
    "reason": {
        "message": "getaddrinfo EAI_AGAIN xxxxxx-api.xxxxx.com",
        "name": "Error",
        "stack": "Error: getaddrinfo EAI_AGAIN xxxxxx-api.xxxxx.com at GetAddrInfoReqWrap.onlookup [as oncomplete] (node:dns:108:26)",
        "code": "EAI_AGAIN"
    }
}

{
    "status": "rejected",
    "reason": {
        "message": "getaddrinfo EAI_AGAIN xxxxxx-api.xxxxx.com",
        "name": "Error",
        "stack": "Error: getaddrinfo EAI_AGAIN xxxxxx-api.xxxxx.com at GetAddrInfoReqWrap.onlookup [as oncomplete] (node:dns:108:26)",
        "code": "EAI_AGAIN"
    }
}

Brody•13mo ago

(use triple backticks) but yes, that error message can only get us so far in debugging this, we need additional debugging logs, I want you to log the dns resolver in use for each of these 10k api calls

ENT3I <3OP•13mo ago

Does this helps?

{
    "exitCode": 0,
    "stderr": "",
    "stdout": "search internal railway.internal nameserver fd12::10 nameserver 8.8.8.8"
}

{
    "exitCode": 0,
    "stderr": "",
    "stdout": "search internal railway.internal nameserver fd12::10 nameserver 8.8.8.8"
}

Brody•13mo ago

no it doesn't, please review what I am asking

ENT3I <3OP•13mo ago

The thing is this repo runs through an app that I don't think it stores each connection dns resolver in use so it's hard to get that information

Brody•13mo ago

do you have access to the codebase that is making these 10k api calls per day?

ENT3I <3OP•13mo ago

I could make some changes to capture DNS traffic for new requests, but I don't think I can see past ones

Brody•13mo ago

yeah of course not, I don't expect you to be able to see the past ones, I wanna see the dns resolver that's used for all future requests

ENT3I <3OP•13mo ago

k, i'll let you know if I see again the error. I'll try to make the modifications later. Thanks

Brody•13mo ago

and you've mentioned a proxy, railway does not have any sort of proxy or firewall on the outbound connections, they only have metering

Gaming

Programming

getaddrinfo EAI_AGAIN