Scapping/Datacenter IP
I got a question. Im on the pro plan for my website and the rules are all enabled. Im getting scrapped like crazy by datacenters IP.
Is there a way to block all datacenters IP or ask them to do a CF challenge?
Thanks !
11 Replies
There is no option to select "Datacenters" for a rule but there are 2 filters to play with:
- "Known Bots", this might capture what you want, you can make an exception for "Verified Bot Category" maybe to prevent Google from scraping for example. Depends on if you want to stop all scraping or just the excessive ones.
- "AS Num", lookup via WHOIS which AS number the IP belongs to and offer a challenge to those, you can for example challenge AS Num 14061 to challenge all IP's from the cloud provider Digital Ocean.
Keep in mind that a lot of traffic comes from datacenter IPs you might want to still want to allow like webhooks from other services or good bots uptime monitoring and the like. So be careful with sweeping rules that block large parts of the internet!
Good luck!
There are always bots lurking around in the web. They knock on every domain, IP address, and port constantly looking for security holes.
Most of them are rudimentary and try all exploits randomly, including ones for applications unrelated to your project, such as Wordpress or PHP.
These bots learn about your domains (including
.workers.dev and .pages.dev) near-instantly using Certificate Transparency logs, which cannot be disabled and are always issued (even on non-Cloudflare platforms).
Some options to help reduce the noise from these bots:
- Disable .workers.dev
- Redirect .pages.dev
- Enable Managed Rulesets (requires Pro plan or above)Is there a way to disable those .pages.dev ?
yeah i dont see the option they talking about
My question was can i disable them
Okay
Yeah i already do
But wanted to fully disable them
You can’t
Will it be something that will be possible in the future ?
No
You can use Workers Assets, which doesn't have this issue
What is the difference ?
Basically Workers with Asset serving. I'd recommend moving to it wherever possible. Also because you can deploy it as a route on your domain, which means that there is no
pages.dev/workers.dev to hitFor some reasons i have AS13335 CloudflareNET that spam me in russia
is that normal ???
Anyone using Cloudflare WARP (VPN), which Apple Private Relay uses as well, uses that asn
So its a vpn service. I can block the whole AS without any problem on my cloudflare service ?