MasterScrat
CDCloudflare Developers
•Created by MasterScrat on 3/15/2024 in #general-help
Getting unexpected MISS when using Cache Reserve
We're using CDN with Cache Reserve to serve large binary files. I'm confused by the
HIT/MISS behavior.
Our setup
- We host large files on Google Cloud Storage in private buckets
- We serve it through Cloudflare using a CNAME record on a subdomain cdn.ourdomain.com
- We enabled Cache Reserve to also cache large files
- We use GCS signed URLs to allow access to these files
- We have tried with "Tiered Cache" either enabled or disabled, it didn't change the observations below
What I observe downloading a 2GB file from cdn.mydomain.com
- It always returns MISS, no matter how many times i download it
- However, after the first download, I see the "Egress savings (bandwidth)" and "Requests served by Cache Reserve" in the Cache Reserve page increase every time I hit the file, closely matching what I download
- If I delete the file from GCS, I instantly get a 404 when trying to download the file again
Why I'm confused
- Since the file seems to be served from Cache Reserved (according to the "Egress savings" metrics), I don't understand why I get cf-cache-status: MISS?
- Why do I immediately get 404s right after deleting the file in GCS: does it mean Cache Reserve does a HEAD request every time the file is downloaded?
Trying the exact same setup with an 80MB file, I get HITs after the initial requests, and if I delete the file from GCS I can still download it until the cache expires.5 replies
CDCloudflare Developers
•Created by MasterScrat on 12/13/2023 in #general-help
Serving TB of data from Google Cloud Storage (GCS)
Hello all,
We are serving TB of data per day from GCS (ML checkpoints, 2 GB to 10 GB per file).
Egress is starting to be a pain! 💸
I'm wondering if Cloudflare could help:
- CDN: Could we just proxy a GCS bucket using a domain CNAME, and enjoy both free egress from CF and cheaper egress between GCS and CF due to the Bandwidth Alliance?
- R2: Could we use R2 with Sippy for the same purpose? Would it be a better match?
23 replies