CloudFlare will show a cached version of my site for crawlerbots, no matter if I purge the cache
If I visit my site with the user agent "Mozilla/5.0 (compatible; AhrefsBot/7.0; +http://ahrefs.com/robot/)" for the ahrefs crawler, I keep getting an outdated version of my site, even though if I visit it with the browsers default user agent, work without a problem.
I have clicked "Purge everything" 20 times already, and no matter what, I cannot seem to get the right result for the crawler user agent. Why could that be???
2 Replies
probably is cached by the your browser
clear your browser cache
Figured it out. It was Netlify prerendering
Jesus, that was hard to debug. Whole day for this