How to cache request coming from bots / crawlers?
I tried setting a Page Rule Cache Everything + Edge TTL 1 Month for a specific endpoint. In the logs, I can see bots calling that endpoint but apparently their visit do not trigger Cloudflare to cache the result (a dynamic image).
Visiting the image from a browser would then result in a MISS and subsequent requests would be a HIT.
1 Reply
There's nothing special about a bot/crawler calling that endpoint unless you have something else (a cache rule, etc) checking for user agent/verified bots/etc. What's your page rule/what are you seeing exactly?