Caching by headers?
Hi, is it possible to cache pages based on the response headers or similar?
We have some "static" pages on our site that we'd like to cache in Cloudflare, but we don't know the URLs (as we support custom CNAMEs) so I'm struggling to find a way to "Cache everything" without using Page Rules?
I tried using a worker, but I couldn't seem to get that working either...
57 Replies
forgot to mention, we set some headers on these pages hoping we could either encourage CF to cache them, or we could use them in workers to cache the response:
// Set header to let Cloudflare know this request is static content and should be cached
$CI->output->set_header('X-Static-Site: 1');
// Set Cache-Control header
$CI->output->set_header('Cache-Control: public, max-age=60');
// Optionally set Expires header
$expiresDate = gmdate('D, d M Y H:i:s', time() + 60) . ' GMT';
$CI->output->set_header("Expires: $expiresDate");
// Ensure there's no conflicting Pragma header
$CI->output->set_header('Pragma: cache');
it's a static html page, do you mean I need to explicitly set the content type or something?
here's my response headers im getting:
Access-Control-Expose-Headers:
X-Country, X-Language
Alt-Svc:
h3=":443"; ma=86400
Cache-Control:
public, max-age=60
Cf-Cache-Status:
DYNAMIC
Cf-Ray:
8287ad8dee1e2df8-DPS
Content-Encoding:
br
Content-Type:
text/html; charset=UTF-8
Date:
Sun, 19 Nov 2023 10:12:59 GMT
Expires:
Sun, 19 Nov 2023 10:13:59 GMT
Nel:
{"success_fraction":0,"report_to":"cf-nel","max_age":604800}
Pragma:
cache
Report-To:
{"endpoints":[{"url":"https://a.nel.cloudflare.com/report/v3?s=XH%2FPJXa8RpPl5J19gE1EXyoybmxIyuHMZ228uI15PGN0hGhw1Wa1VbDzShravyn2X7gUsiLtda3HvxFF%2FHxRCcQlifiQtMxtqV09wN5kDazRHBQmXo7NHnqyrhQXgZO8SuWNTA%3D%3D"}],"group":"cf-nel","max_age":604800}
Server:
cloudflare
Set-Cookie:
ci_sessions=xxx; expires=Mon, 20-Nov-2023 10:12:59 GMT; Max-Age=86400; path=/; secure; HttpOnly
Strict-Transport-Security:
max-age=15552000; includeSubDomains; preload
Vary:
Accept-Encoding
X-Content-Type-Options:
nosniff
X-Country:
ID
X-Language:
en-GB
X-Static-Site:
1
Do you have a Cache Everything rule set?
in page rules? no
the URLs are dynamic so i cant use page rules (hence me trying to do it via headers or workers)
thanks but i dont think that will work for this page? https://demo.shoprocket.io/
Green Fingers Online Store
Browse & buy online from Green Fingers
we also support custom cnames, so it could also be something liek https://store.mahakatha.com/
Mahakatha Meditation Online Store
Official Store of Mahakatha Meditation Mantras and Ancient Chants from India, Shiva Mantras, Ganesha Mantras and more
i've been trying with workers all day and i THINK I have a solution
its not perfect
that's what i thought, but i cant figure out how
in that link you sent, you say:
"You can add a cache rule to force CF to cache stuff, it still wont cache dynamic content but will attempt to cache everything in caching/cache-rules/new"
pretty sure i tried that
but let me try again
Green Fingers Online Store
Browse & buy online from Green Fingers
like this?
i used a != to apply to all requests
because i wont know the hostnames (we support custom cnames)
is it wrong?
hows this?
whoops sorry, 1 moment
like this?
no effect
ive been on/off trying to get this workign with CF for a couple of years now
it never does
every time i get close i find some small print somehwere that say it requires an enterprise plan
i cant do that
because then it will cache every response from our zone (if it caches at all)
we have APIs etc that must not be cached
i'll try...
how dot CF know whats dynamic?
back to this again
welp
it worked
and cahced our APIs
great 🥲
yes
json
e.g **
https://api.shoprocket.io/storefront/v1/stores/sr_live_pk_0ac696797e14d49ca01f523ffb8d962e670f/products?search=&sort=product_order&page=1&limit=12
i just disbaled it and had to "purge everything"
because i disabled it
but it was caching
yeh
and age
no
you;re just seeing the old cache data i think
basically, when i enable "Use cache-control header if present" - no cache
with "Ignore cache-control header and use this TTL" - cache, but caches everything, even things i dong want cached
i just changed it to this
That's not what eligible for cache does
Eligible for cache just disables the filetype check(or any other default checks)
right
so....its not doable?
What you have above should be fine
doesnt seem to cache at all
maybe i havent set my headers right?
https://f90.shoprocket.io/
F90 Football Academy Store Online Store
Browse & buy online from F90 Football Academy Store
I'm getting
HIT on the demo pageAccess-Control-Expose-Headers:
X-Country, X-Language
Alt-Svc:
h3=":443"; ma=86400
Cache-Control:
public, max-age=600
Cf-Cache-Status:
BYPASS
Cf-Ray:
828855f37c2d2df9-DPS
Content-Encoding:
br
yeh the demo one is now cached with CF, i cant seem to purge it
but no other urls are caching
Dark Stry Online Store
Experience places differently with our self-guided walking and driving tours in Perth and the Coral Coast, Western Australia.
Cache-Control:
public, max-age=600
Cf-Cache-Status:
BYPASS
What happens if you trace it? https://dash.cloudflare.com/?to=/:account/trace
Does it show any Rules being applied?
checking
do you think the transform rules are preventing it being cached?
It shouldn't
If you click
View Rule Detail on use origin, what does it show?{
"step_name": "2bb8e71132f1486baccc181d2d25bf30",
"type": "rule",
"matched": true,
"action_parameter": {
"cache": true,
"edge_ttl": {
"mode": "respect_origin"
}
},
"expression": "(http.host ne "foo")",
"description": "use origin ",
"action": "set_cache_settings"
}
i didabled those header rules just to rule them out
no change
Alt-Svc:
h3=":443"; ma=86400
Cache-Control:
public, max-age=600
Cf-Cache-Status:
BYPASS
Cf-Ray:
828863155a342df9-DPS
Content-Encoding:
br
Content-Type:
text/html; charset=UTF-8
Date:
Sun, 19 Nov 2023 12:16:54 GMT
Expires:
Sun, 19 Nov 2023 12:26:54 GMT
Nel:
{"success_fraction":0,"report_to":"cf-nel","max_age":604800}
Pragma:
cache
status is alwasy BYPASS or DYNAMIC
i also have smart tiered cache enabled
and cache reserve
i have that same config:
yup - as expected, thats just page rules
which i cannot use
you cant do it by resposne header
only request header
and i dont know what URLs need caching until the response is available
origin
yeh
it doesnt
i dont believe it ever has
its been around 15 mins already
no change
Think I realised the issue:
Cloudflare also sets BYPASS when your origin web server sends cookies in the response header.
🥲
yeh i cant set in CF because i dont know the URLs
Can you get your origin to stop serving cookies?
possibly - i'll need to dig into that
but at least now i now why it never cached for me!
thanks
Header Transforms run closer to the origin than Cache Rules, so if you can scrub the Cookie Header there, then it might still be able to cache
Cloudflare Cookies shouldn't™️ be counted for cookies
At least in terms of cachability
done!
Green Fingers Online Store
Browse & buy online from Green Fingers
removed cookies from our hosted stores (they weren't need anyway)
thanks everyone 🙏