Cloudflare Browser rendering Blocking
hey guys i am using browser rendering from cloudflare to scrape my customers website to generate embeddings for their sites
now one of my customer is protected by cloudflare itself
and now i am getting this error - "Please unblock challenges.cloudflare.com to proceed."
can anyone please help why this is coming, its like cloudflare is banning its own IP of worker?
how can i solve this? i tried adding headers while create new page, but nothing is helping
Please help.
3 Replies
whereas this guys crawler is almost same code but its able to get the data
https://md.dhr.wtf/?url=https://www.karsonslettings.com/property-search/
what is happening?
Look like you're trying to be cheeky...
To be clear: Cloudflare doesn't provide a service to bypass Cloudflare security products, so it's expected that you would be prevented from accessing a site with a challenge and in fact not being blocked by a site with a challenge configured would itself be the bug instead.