Server listen addr when deploying to fly.io
I'm on MacOS using wasp 0.13.0. For some reason when my server deploys (both with
wasp deploy fly deploy and the manual fly deployment flyctl deploy --remote-only --config ../../fly-server.toml I'm getting an errors about it not listening on the proper addr/port?
[error][PC01] instance refused connection. is your app listening on 0.0.0.0:8080? make sure it is not only listening on 127.0.0.1 (hint: look at your startup logs, servers often print the address they are listening on)
...
[info]waiting for machine to be reachable on 0.0.0.0:8080 (waited 10.399511506s so far)
...
[error][PM05] failed to connect to machine: gave up after 15 attempts (in 12.400552411s)
Everything works as expected locally. I've run wasp clean and done clean deploys both manually and using the wasp deploy wrapper with no luck. I can't see any changes I made that would affect this, any ideas?26 Replies
Hi @ellie. I'm kapa.ai, your friendly support bot. I will try to answer your question by searching through the knowledge base. :wave:
----------------------
Based on the error message you provided, it seems like your server is not listening on the correct address and port. When deploying to Fly.io, your server should be listening on
0.0.0.0:8080.
You can set the port by adding the following line to your environment variables on Fly.io:
This is mentioned in the Wasp documentation on manually deploying to Fly.io (https://wasp-lang.dev/docs/advanced/deployment/manually#set-up-a-flyio-app).
However, the error message also suggests that your server might be listening on 127.0.0.1 instead of 0.0.0.0. This could be due to a configuration issue in your server code. Unfortunately, the provided knowledge sources do not contain information on how to configure the server listen address in Wasp.
If the issue persists, I would recommend checking the server logs for more information. You can view the logs by running the following command:
This command is also mentioned in the Wasp documentation on manually deploying to Fly.io (https://wasp-lang.dev/docs/advanced/deployment/manually#deploy-to-a-flyio-app).
If you're still having trouble, it might be a good idea to reach out to the Wasp community on their Discord server (https://discord.com/channels/686873244791210014) for further assistance.
----------------------
Note: you can ask me a follow up question by @ mentioning me again :speech_balloon:
----------------------My server port is set in secrets, and my deployment had been working for a couple of months just fine until this week 😦 the outputs I included above are from flyctl logs, here's another warning that I see when I deploy manually (LOG_LEVEL=debug flyctl deploy --remote-only --config <path to fly-server.toml>):
WARNING The app is not listening on the expected address and will not be reachable by fly-proxy.
You can fix this by configuring your app to listen on the following addresses:
- 0.0.0.0:8080
Just tried a clean deployment starting from scratch with wasp deploy fly launch and I'm getting the same error. @kapa.ai where is the server addr set in the wasp project?Let me check the knowledge base for you. One moment! :books:
----------------------
The server address is typically set in the server's configuration, which is managed by the framework or library used to create the server. In the case of Wasp, the server is an Express.js server running on Node.js.
However, the provided knowledge sources do not contain specific information on how to set the server listen address in a Wasp project. The server listen address is usually set in the server's startup script, but the exact location can vary depending on the structure of your project and the libraries you're using.
In the Wasp documentation (https://wasp-lang.dev/docs/advanced/accessing-app-config), it's mentioned that you can access the server configuration object like this:
But it doesn't specify how to set the server listen address.
If you're still having trouble, I would recommend reaching out to the Wasp community on their Discord server (https://discord.com/channels/686873244791210014) or opening an issue on their Github page (https://github.com/wasp-lang/wasp/issues) for further assistance.
----------------------
Note: you can ask me a follow up question by @ mentioning me again :speech_balloon:
----------------------
Sorry @ellie, I seem to be having a bit of trouble. Don't worry though, I've pinged my support and they will try to fix me up! :technologist:
Perhaps try again in 15mins?
It turns out this error is collateral from a missing env var on fly.io, ugh! If you're reading this, thanks for making it this far 😅 all good now
Which env var did you miss? 🙂
"thanks for making it this far" cracked me up 😄
But yes, as @miho asked, please tell us what you missed so we can see if there's a way to emphasize it in the docs
It was an API key I had added myself for a new feature… nothing from WASP itself! The frustrating part is that the app will start and a helpful error will print when running locally… but no such luck when deployed 😅 I’m not sure if that’s related to it being dockerized?
Wohooo @ellie, you just became a Waspeteer level 1!
Definitely something to look into so thanks for reporting! 😄
Just one more question, what exactly got reported in development mode? Runtime error when your code attempted to read that variable?
Yep exactly, I have a null check at runtime. Seems like the missing var prevents successful startup altogether in prod
Ok, one fiiinal question 😅
Do you mind pasting that
null check so I can reproduce?Of course! Stepped away from my desk but I’ll follow up in 45 minutes or so 🙂
Excellent, thanks for helping out!
here it is!
export function setupAnthropic() {
if (!process.env.ANTHROPIC_API_KEY) {
throw new Error('Anthropic API key is not set');
}
return new Anthropic({ apiKey: process.env.ANTHROPIC_API_KEY });
} Hm so this crashed server in production in such way that this error message wasn't even logged @ellie ? If so that is a bit lousy, would be great if it was nicely logged!
Yeah, so weird! Even when I ran with verbose, I'd get entries like this (below) with some insight into the npm commands, etc., but the only hints about the crash were the exit code being 1 and then those misleading server addr bind errors
[info]> npm run db-migrate-prod && NODE_ENV=production npm run start
[info]> [email protected] db-migrate-prod
[info]> prisma migrate deploy --schema=../db/schema.prisma
[info]Prisma schema loaded from ../db/schema.prisma
[info]Datasource "db": PostgreSQL database "patentmapr_server", schema "public" at "patentmapr-db.flycast:5432"
[info]28 migrations found in prisma/migrations
[info]No pending migrations to apply.
[info]> [email protected] start
[info]> npm run validate-env && node --enable-source-maps -r dotenv/config bundle/server.js
[info]> [email protected] validate-env
2024-05-09T22:10:05Z app[82de77b793d118] ord [info]> node -r dotenv/config ./scripts/validate-env.mjs
[info]🔍 Validating environment variables...
[info]✅ Both STRIPE_CUSTOMER_PORTAL_LINK links defined
[info] INFO Main child exited normally with code: 1
[info] INFO Starting clean up.
[info][ 8.439734] reboot: Restarting systemSmall hint @ellie : if you put code into triple backtick, then it will show as a nice block. So first three backticks in one line, then code in lines below, then as the last line three backticks again.
Hm ok, so this is really lacking regarding error messages! At which place in Wasp app did you put that
setupAnthropic() check? I wonder when it is triggered exactly, that will help us figure out why it is not being show in Fly.io -> because it seems like app crashes before that check is triggered.
Also another thing: I see all logs have [info] in front of them. I wonder is it possible that errors are logged somewhere else? But now, I would expect them to also be logged here hm.Thanks for the triple backtick hint! I was wondering how to do that actually 😅
I have a separate file at the same level as
actions.ts called llm.ts, and actions.ts includes methods from that file. In llm.ts, I have a top level const defining anthropic like this:
Got it!
You mentioned
those misleading server addr bind errors -> would you mind pasting those, what do they look exactly like? I don't see them in the Fly logs above.@martinsos Leaving this one to you too as well, just FYI for when you scan messages
Ooop missed this for some reason, here are some of the errors:
^ this would show up during the
wasp deploy fly deploy and then the server logs after that would have entries like this:
Could you take a screenshot of this part of your Fly dashboard? Also check the
secrets that they are set?
Everything is working now! The problem was that an env var that I added wasn't set on fly, but the server errors were misleading (and different from what I might see in dev mode on localhost) so it took me a bit to get to the bottom of. Here's the config:
! The problem was that an env var that I added wasn't set on flyHow did you add the env var?
It was a new env var that I defined for a new feature, I had forgotten to add it to fly (which I eventually did via their Secrets tab on the console). When it was missing, I was seeing the errors above!
Wohooo @ellie, you just became a Waspeteer level 2!