Detecting bot activity by email servers
Hey all, I have emails that get sent out to clients for them to commit a decision (accept / decline). The links go to an endpoint which reads an encrypted url parameter and commits the decision to a database, then serves a html page...
The problem: Some corporate emails use bots to determine whether or not the email is a security threat. Basically the bot is going to both the links and hitting the endpoints! Now an easy fix would be to add an extra step to the process and have clients click again once they reach to the page, but this is doubling the effort for users for the sake 1% of accounts that use a corporate emails with aggressive crawling.
Is there some sort of cloudflare product that I could use to confirm if the link clicked is by a human?
Any advice welcome
Cheers!
4 Replies
Now an easy fix would be to add an extra step to the process and have clients click again once they reach to the page, but this is doubling the effort for users for the sake 1% of accounts that use a corporate emails with aggressive crawling.The problem is you're kind of asking an impossible request: How can I identify bots without bothering real users/adding any resistance? Well if the bots are really trying they could bypass anything, let alone something that is designed to not add resistance. Cloudflare has solutions such as turnstile or even turnstile invisible but they would add friction. Cloudflare has Enterprise Bot stuff too but it's well Enterprise. Kind of a novel issue: https://www.mailsoar.com/blog/deliverability/bots-clicking/ You could try blacklisting specific IPs/and or User Agent. Making a list by adding an invisible third link that some might fall for/click and make it obvious they're bots, etc. (or if they click the link way too fast from email send, show an interstitial page just then) As far as I know the only foolproof solution is an interstitial page though asking for confirmation of that action.
Thanks for the considered response, yeah I was probably being too hopeful for a simple solution haha. theres a lot of mitigations we could build by timing the responses and/or adding invisible links .etc. for now i think the simplest approach is to identify the majority consumer emails (match gmail, hotmail, yahoo .etc) and serve them actionable links in the mail. Give corporate emails a two step process.
I'm going to give the CF bot fighting mode a shot next week, but I'll keep my expectations low haha.
Thanks again!
I'm going to give the CF bot fighting mode a shot next week, but I'll keep my expectations low haha.Just be careful, Free Bot Fight Mode (on free zones) is not configurable at all, no custom rules can skip or anything, it's on for entire domain/zone or off Super Bot Fight mode (pro or higher) is configurable via Custom Rules but is by default on for the entire zone SBFM is also considered a super high security thing and is going to have false postives. Cf's own docs push you towards enabling when under attack and disabling afterwards. Not saying it can't work, just saying be careful. If it false positives it'd force the user through a challenge
Thanks for the headsup, yeah not ideal, probably dont want landing pages to have challenges from PPC campaigns haha