W
Waspā€¢2w ago
Killshot

LemonSqueezy webhooks not working after deployment (VPS)

My Lemonsqueezy webhooks work fine in local environment with ngrox, but give error after deployment...
51 Replies
kapa.ai
kapa.aiā€¢2w ago
Hi @Killshot. I'm kapa.ai, your friendly support bot. I will try to answer your question by searching through the knowledge base. :wave: ---------------------- I apologize, but I don't have any specific information about LemonSqueezy webhooks in the provided knowledge sources. The documents mainly discuss Stripe webhooks and general deployment issues with Wasp applications. However, based on the information about Stripe webhooks, I can suggest a general approach that might be helpful: 1. Check your webhook endpoint URL: Make sure you've updated the webhook URL in your LemonSqueezy dashboard to point to your production server, not your local development environment. It should look something like <https://your-production-domain.com/lemonsqueezy-webhook> (adjust the path as needed for your application). 2. Verify environment variables: Ensure that any necessary environment variables for LemonSqueezy integration are properly set in your production environment. This might include API keys or webhook secrets. 3. Check server logs: Look at your server logs for any error messages related to the webhook requests. This could provide more insight into what's going wrong. 4. Verify server configuration: Make sure your server is properly configured to receive and process webhook requests. This might involve checking firewall settings, routing, etc. 5. Test with LemonSqueezy's tools: If LemonSqueezy provides any webhook testing tools (similar to Stripe's webhook testing), use those to debug the issue. If you continue to have issues, you may need to consult LemonSqueezy's documentation or support channels for more specific guidance, as the provided knowledge sources don't contain information about LemonSqueezy integration with Wasp. ---------------------- Note: you can ask me a follow up question by @ mentioning me again :speech_balloon: ---------------------- Sorry @Killshot, I seem to be having a bit of trouble. Don't worry though, I've pinged my support and they will try to fix me up! :technologist: Perhaps try again in 15mins?
Killshot
KillshotOPā€¢2w ago
Killshot
KillshotOPā€¢2w ago
@Vinny (@Wasp) can this be because in production i have to use production keys and i am still in Test mode with test keys in LemonSqueezy ?
miho
mihoā€¢2w ago
Could it be that your Nginx is set up differently on your VPS and there some additional things enabled e.g. parsing stuff as JSON or something like that?
Killshot
KillshotOPā€¢2w ago
i am using Caddy, but let me check this.
miho
mihoā€¢2w ago
My bad, I read ngrok as nginx šŸ‘
Killshot
KillshotOPā€¢2w ago
ohh so that can't be it i guess
Vinny (@Wasp)
Vinny (@Wasp)ā€¢2w ago
That sounds possible! But I don't know, to be honest. With Stripe you can deploy an app with test keys But seems odd that you're having this issue. Could it be something to do with node versions or something on your server. @martinsos might know more than me on this topic.
Killshot
KillshotOPā€¢2w ago
I have checked twice and the versions do match, but let's see what input martin has
martinsos
martinsosā€¢2w ago
@Killshot I wonder if this is connected with that other issue you had, also with buffers in lemon squeezy webhook, but at compile time (while running npm run build in web-app/ dir). Error is coming from the same part of the code: crypto.timingSafeEqual(Buffer.from(signature, 'utf8'), digest).Before it was complaining about types of Buffer, now it complains about their lengths in runtime -> there is some kind of issue here for sure. @Killshot I am guessing, that in order for you to get this far and get the runtime error, you had to get pass that compile time error first -> how did you get past it? @Vinny (@Wasp) you wrote this piece of code, any ideas what could be the issue here? Seems like some stuff is not vibing here. @Killshot can you check tha you have LEMONSQUEEZY_WEBHOOK_SECRET correctly set up there on VPS? @Killshot I would also advise you to put in some console.log()s in there, to log rawBody, signature, secret, hmac, digest. Don't share those here as some of it might be sensitive, but do log them out for yourself so you can take a look and see if anything there is weird, like an empty value or somethign suspicious.
Killshot
KillshotOPā€¢2w ago
Yes i had the same doubt regarding webhook secret and store id, both are confirmed and they are same as in production and local environment, but still local environment seems to be working fine including webhooks, but not production. i will add some logs as suggested and see what's the issue here. The reason i tagged vinny is because i remember he implemented lemonsqueezy here. Also... i solved the Type buffer error which was in the same code with manually editing the typescript version to the 5.1.0 as shown in wasp github repo.
martinsos
martinsosā€¢2w ago
Ohhhhh, so your version of Typescript was different than the one Wasp uses and that caused issues? Interesting that we didn't igve you a nicer error regarding that. @miho any thougths here, woud you expect we give a nicer error?
Killshot
KillshotOPā€¢2w ago
exactly, everytime there's an error if there is a version mismatch in any dependncy but this time there wasn't. also i didn't face any error while running wasp compile.
martinsos
martinsosā€¢2w ago
Ok thanks, we willl look into that (lack of error), I believe TypeScript is a bit different / special case so that is maybe why. Ok hm, so back to the current webhook problem -> maybe you are right, maybe issue is that you are using test setup/keys but your app is deployed. I will pass it back to @Vinny (@Wasp) , he knows lemon squeezy webhooks and has impelmented that part, I am here more for general stuff :). But I am curious, how does LS konw where to send webhooks? I woudl imagine that for each env (test, production, ...) you need to tell it an url/ip of server where the webhook is. So what do you have set there that works both for local developemtn and also the app when it is on VPS?
Killshot
KillshotOPā€¢2w ago
for that i am thinking of doing both, apply more logs in the code in both side of the signature verification and also will try to use production keys once and see what happens.
martinsos
martinsosā€¢2w ago
Yup, those sound like best bets at the moment
Killshot
KillshotOPā€¢2w ago
it allows to set the url yes, and for local testing i used ngrok as mentioned in the opensaas guide for payment integration.... https://docs.opensaas.sh/guides/payments-integration/ and for production i used api.myurl.com/payments-webhook
OpenSaaS.sh
Payments Integration
Open SaaS is a free, open-source, full-stack SaaS starter kit for React + NodeJS.
martinsos
martinsosā€¢2w ago
So you basically changed that URL now so it points to your app on VPS?
Killshot
KillshotOPā€¢2w ago
yes
martinsos
martinsosā€¢2w ago
ok, got it, that makes sense to me, but let's see what @Vinny (@Wasp) will say
Killshot
KillshotOPā€¢2w ago
yeah sure. appreciate the prompt support boss.
martinsos
martinsosā€¢2w ago
Btw sorry that this is taking so long and thanks for the patience hah both thanking each other
Killshot
KillshotOPā€¢2w ago
no worries, part of the job.
martinsos
martinsosā€¢2w ago
no worries, you are helping us figure out what we can do better (like error message for that TS situatino)
Killshot
KillshotOPā€¢2w ago
i just hope i am able to help others when they have issues on deploying via VPS i got a love hate relationship with typescript. :godfatherboi:
Vinny (@Wasp)
Vinny (@Wasp)ā€¢2w ago
Haha. Ok tomorrow Iā€™m gonna try out deploying the app with lemon squeezy test keys and see if I run into similar issues @martinsos
Killshot
KillshotOPā€¢2w ago
Oh that would be so good, Thanks in advance Boss.
miho
mihoā€¢2w ago
Changing the Typescript version in package.json gives you
- Error: Dependency conflict for user dependency (typescript, ^5.5.0): Version must be set to the exactly the same version as the one wasp is using: ^5.1.0
- Error: Dependency conflict for user dependency (typescript, ^5.5.0): Version must be set to the exactly the same version as the one wasp is using: ^5.1.0
I'm not sure how and where @Killshot you changed the Typescript version? If you could help me out to understand so I can maybe try doing the same šŸ™‚
Killshot
KillshotOPā€¢2w ago
while migrating from 0.13 to 0.15, I started getting Type Buffer error as shared here in questions earlier, which didn't solve for a while but then i noticed that there was TS wedges in typescript but not typescript and it wouldn't let me install it to via terminal, but then i visited wasp repo and copied the latest package.json version of typescript into mine and ran npm install which solved the issue. Up until 0.13 i had the typescript working fine and without any errors, but i guess migration cause some issues and never game me the above mentioned version conflict error. If i had gotten that i would have solved this much sooner šŸ˜… cause it clearly doesn't get any more obvious then that. I still have a minor webhook issue in lemonsqueezy going on which i gotta solve before moving into production.
miho
mihoā€¢2w ago
I don't mean to grill you about this, I'm really looking to understand what you did and tried to fix your app šŸ™‚ In 0.13.2, the TS version is "typescript": "^5.1.0", In 0.15.1, the TS version is "typescript": "^5.1.0", I suspect that something went wrong with npm and node_modules and maybe wasp clean could have helped. Did you try doing it as well?
Killshot
KillshotOPā€¢2w ago
ofcours, kapa suggested that first, i did both wasp clean and wasp build again and honestly multiple times during that issue. it didn't seem to fix it. is it possible that i may have removed Typescript all together from the package.json and it still didn't show error in 0.13 version too but gave me error when running npm run build in the wasp/build/web-app folder? i was doing that as part of deploying via VPS
miho
mihoā€¢2w ago
Removing the typescript bit from the package.json usually ends up like this šŸ˜„ so I think you didn't do that
No description
miho
mihoā€¢2w ago
While we are on the topic of VPS deployments, I suppose you read this by now: https://gist.github.com/infomiho/80f3f50346566e39db56c5e57fefa1fe but let me put it here just in case Ok, so the Typescript error is gone now, now we have to figure out the LemonSqueezy one next šŸ˜„
Killshot
KillshotOPā€¢2w ago
oh yeah loved that guide, very precise. yes :godfatherboi: Vinny was gonna test out a production app integrated with LemonSqueezy will see what he has to say
Vinny (@Wasp)
Vinny (@Wasp)ā€¢2w ago
No description
Vinny (@Wasp)
Vinny (@Wasp)ā€¢2w ago
no issues deploying lemon squeezy test keys and checking out after deploying to fly.io (cc: @martinsos @miho ) https://lemon-squeezy-test-client.fly.dev/ I did have to update my package.json to include react-dom and react-router-dom explicit versions, but I don't think this is the issue affecting you:
"react": "^18.2.0",
"react-dom": "^18.2.0",
"react-router-dom": "^6.26.2",
"react": "^18.2.0",
"react-dom": "^18.2.0",
"react-router-dom": "^6.26.2",
Killshot
KillshotOPā€¢2w ago
Any reason for it to not be working because i am on a VPS? I have configured Caddy server well too and all APIs are working fine
Vinny (@Wasp)
Vinny (@Wasp)ā€¢2w ago
I have no experience deploying to a VPS but it definitely sounds like something is going wrong in your Build process. @miho is the most experienced with deploying to VPS, so I'll let him take over from here.
miho
mihoā€¢2w ago
So you have: 1. VPS 2. Caddy as your reverse proxy (please share the config with us) 3. Your server running in Docker (how did you start it?) 4. Your other APIs are working (you tried GET and POST type of APIs?) 5. Your Lemon Squeezy webhook is not working (fails with the Input buffers must have the same byte length error
Killshot
KillshotOPā€¢2w ago
Here is my Caddy Configuration { # Global configuration acme_dns cloudflare {env.CLOUDFLARE_API_TOKEN} log { output file /var/log/caddy/error.log level ERROR } } mydomain.com, www.mydomain.com { root * /home/killshotroxs/projects/myproject/client encode gzip try_files {path} /index.html file_server tls { dns cloudflare {env.CLOUDFLARE_API_TOKEN} } log { output file /var/log/caddy/project_access.log } } api.mydomain.com { reverse_proxy localhost:3001 @cors_preflight { method OPTIONS } header @cors_preflight { Access-Control-Allow-Origin "https://mydomain.com" Access-Control-Allow-Methods "GET, POST, PUT, DELETE, OPTIONS" Access-Control-Allow-Headers "Content-Type, Authorization" Access-Control-Allow-Credentials "true" defer } respond @cors_preflight 204 } bot.mydomain.com { reverse_proxy localhost:3000 tls { dns cloudflare {env.CLOUDFLARE_API_TOKEN} } log { output file /var/log/caddy/bot_access.log } }
Killshot
KillshotOPā€¢2w ago
I start the container after setting up database and other steps mentioned in this guide -- https://gist.github.com/infomiho/80f3f50346566e39db56c5e57fefa1fe
Gist
Deploy Wasp to a VPS (reverse proxy + Docker)
Deploy Wasp to a VPS (reverse proxy + Docker). GitHub Gist: instantly share code, notes, and snippets.
Killshot
KillshotOPā€¢2w ago
And yes my other APIs are working fine both GET and POSt but as mentioned before, i get this error when my Webhooks are fired- https://discord.com/channels/686873244791210014/1315633636384571422/1315634500155215872 @miho Would love any insights you can offer here.
miho
mihoā€¢2w ago
Everything seems fine. Nothing screams that it's wrong. Look at the implementation:
const rawBody = request.body.toString('utf8');
const signature = request.get('X-Signature');
if (!signature) {
throw new HttpError(400, 'Lemon Squeezy Webhook Signature Not Provided');
}

const secret = requireNodeEnvVar('LEMONSQUEEZY_WEBHOOK_SECRET');
const hmac = crypto.createHmac('sha256', secret);
const digest = Buffer.from(hmac.update(rawBody).digest('hex'), 'utf8');

if (!crypto.timingSafeEqual(Buffer.from(signature, 'utf8'), digest)) {
throw new HttpError(400, 'Invalid signature');
}
const rawBody = request.body.toString('utf8');
const signature = request.get('X-Signature');
if (!signature) {
throw new HttpError(400, 'Lemon Squeezy Webhook Signature Not Provided');
}

const secret = requireNodeEnvVar('LEMONSQUEEZY_WEBHOOK_SECRET');
const hmac = crypto.createHmac('sha256', secret);
const digest = Buffer.from(hmac.update(rawBody).digest('hex'), 'utf8');

if (!crypto.timingSafeEqual(Buffer.from(signature, 'utf8'), digest)) {
throw new HttpError(400, 'Invalid signature');
}
I'd do the following:
console.log(rawBody);
console.log(signature);
console.log(secret);
console.log(hmac);
console.log(digest);
console.log(rawBody);
console.log(signature);
console.log(secret);
console.log(hmac);
console.log(digest);
Please share it here (omit the secret ofc) so maybe we can debug it further.
Killshot
KillshotOPā€¢6d ago
you're gonna think i am crazy but even after adding extensive logging, i don't see any logs, just the same error in the Docker logs. :sadboi:
const rawBody = request.body.toString('utf8');
const signature = request.get('X-Signature');

console.log('Raw Body:', rawBody);
console.log('Signature:', signature);

if (!signature) {
throw new HttpError(400, 'Lemon Squeezy Webhook Signature Not Provided');
}

const secret = requireNodeEnvVar('LEMONSQUEEZY_WEBHOOK_SECRET');


console.log('Secret Length:', secret.length); // Avoid logging actual secret

const hmac = crypto.createHmac('sha256', secret);
const digest = Buffer.from(hmac.update(rawBody).digest('hex'), 'utf8');

console.log('Computed HMAC:', hmac.digest('hex')); // Log the HMAC value

// Additional logging for debugging
console.log('Computed Digest:', digest.toString());
console.log('Received Signature:', signature);
console.log('Signature Comparison:', crypto.timingSafeEqual(Buffer.from(signature, 'utf8'), digest));

if (!crypto.timingSafeEqual(Buffer.from(signature, 'utf8'), digest)) {
throw new HttpError(400, 'Invalid signature');
}
const rawBody = request.body.toString('utf8');
const signature = request.get('X-Signature');

console.log('Raw Body:', rawBody);
console.log('Signature:', signature);

if (!signature) {
throw new HttpError(400, 'Lemon Squeezy Webhook Signature Not Provided');
}

const secret = requireNodeEnvVar('LEMONSQUEEZY_WEBHOOK_SECRET');


console.log('Secret Length:', secret.length); // Avoid logging actual secret

const hmac = crypto.createHmac('sha256', secret);
const digest = Buffer.from(hmac.update(rawBody).digest('hex'), 'utf8');

console.log('Computed HMAC:', hmac.digest('hex')); // Log the HMAC value

// Additional logging for debugging
console.log('Computed Digest:', digest.toString());
console.log('Received Signature:', signature);
console.log('Signature Comparison:', crypto.timingSafeEqual(Buffer.from(signature, 'utf8'), digest));

if (!crypto.timingSafeEqual(Buffer.from(signature, 'utf8'), digest)) {
throw new HttpError(400, 'Invalid signature');
}
martinsos
martinsosā€¢5d ago
Hm that means logs are getting lost somewhere? Is it possible you are not running the latest code btw? Or maybe you need to make sure those logs surface out, maybe the Docker logs you are looking at the moment are not showing them but only errors or something like that?
Killshot
KillshotOPā€¢5d ago
i used docker logs appname -f so i could see live logs. also put up a few logging in Caddy Configuration to see if i catch something, nothing unusual there too. Do we know someone who has a live project deployed on a VPS with Cloudflare, Caddy and Lemonsqueezy as server and payments? Yes i am on the latest code and i didn't make any changes to the webhook.ts file inside the lemonsqueezy folder. We know vinny tested the same code in Fly.io and it also works fine in my localhost. the only thing different is my VPS is Caddy and Cloudflare.
miho
mihoā€¢5d ago
You should be getting the console.logs. This means something is off. You can do a sanity check by replacing all the source code with just one console.log('Hey I'm here') and see if you get it. If you still get the same error - it's old code and you might need to rebuild your container from scratch or pull the latest code.
Killshot
KillshotOPā€¢4d ago
That worked miho, The issue was resolved by completely cleaning up the Docker environment: Removing the existing container and image: docker rm appName && docker rmi appName Cleaning up unused Docker resources: docker image prune -a Rebuilding the application from scratch: wasp build Creating a fresh Docker image: docker build . -t appName For now everything is working fine. Can't believe i didn't think of this before, but now the doubt is when will i need to do this again, would have loved a warning somewhere saying i might need to clear my docker environment. The webhook error ERR_CRYPTO_TIMING_SAFE_EQUAL_LENGTH persisted because: The old Docker image contained cached versions of the code Simply running wasp clean and wasp build wasn't enough because the Docker container was still using the cached image layers.
miho
mihoā€¢4d ago
Ooof, yeah, I got bit by this multiple times... It would be great if Docker told you - "Hey, I'm not even using your latest code" I'm not sure how you can detect this better, hm, there is the --no-cache Docker build option. It's very confusing that the old cache didn't get invalidated šŸ˜¦ https://docs.docker.com/build/cache/invalidation/
Docker Documentation
Build cache invalidation
Dig into the details about how cache invalidation works for Docker's build cache
Killshot
KillshotOPā€¢4d ago
Exactly... !! Anyways i truly appreciate the support and patience you folks showed, Now going back to building. Until then..... May the Force be with you :salute_right:.
Want results from more Discord servers?
Add your server