Notifying big number of clients about R2 file changes
I have a R2 bucket that one client uploads files to using a Worker. Then, I have a couple 1000 other clients that should be informed about these changes as soon as possible (within 10-60 seconds) so they can download the updated files.
What would be the best way to achieve this? Currently I have the clients poll for changes once a minute or so, which works but doesn't really scale well. I need a setup that works well with 1M+ clients :NyanThinking:
3 Replies
They just shipped an open beta for R2 event notifs -> https://blog.cloudflare.com/r2-events-gcs-migration-infrequent-access#event-notifications-open-beta
Might be worth a shot
The Cloudflare Blog
R2 adds event notifications, support for migrations from Google Clo...
We’re excited to announce three new features for Cloudflare R2: event notifications, support for migrations from Google Cloud Storage, and an infrequent access storage tier
posted 04/03/2024, so your mileage may vary
Interesting. Thanks a lot for the link! When that event fires, what would you say is the best way to inform clients?
I'm reading through the documentation right now, but would you say this is possible with durable objects and websockets? As far as I can tell, websocket hibernation sounds like what I'm looking for. Clients would register a websocket through the DO and the websocket hibernates until the R2 event notification is fired, then the worker wakes up all websockets and sends a signal to trigger the clients to download the new data.