Cloudflare Developers•4w ago

create TTL cache in CF worker

I need to throttle some code in a CF worker. i.e., if something is called 5 times in a 2 minutes, run it only once. I was thinking to use something like ttl cache to implement but this isn't working https://www.npmjs.com/package/@isaacs/ttlcache.

13 Replies

Chaika•4w ago

That's all in memory. Workers are spun up as needed on the same cdn machine as the connection hits. There's no affinity to machines already running or anything, and CF has 350+ PoPs with thousands of machines.. so not going to hit. It may seem like it work-ish in prod due to connection keep alive, but to any other people connecting or once your connection dies, you'll hit a different one The truest solution here would be Durable Objects and their persistent storage. But DOs don't run everywhere. Using CF Cache would be free & fast but only local to that PoP: https://developers.cloudflare.com/workers/runtime-apis/cache/. You'd just make responses to cache and pass in ttl in cache-control header

devinOP•4w ago

Thank you Chaika. Despite being in the Oxygen runtime I think the Cache API may indeed be an option. It seems to me that subsequent requests will likely be the same PoP, no? My alternative here is to spin up a new CF worker that leverages KV I think?

Chaika•4w ago

oh you're using the Oxygen runtime?

It seems to me that subsequent requests will likely be the same PoP, no?

For a single user, keep alive will keep them pinned to the same machine even, but only for the life of the connection KV's eventually consistent w/ a min cache of 60s, not a good use case for throttling really, it'd also be a bit expensive for this use case

Chaika•4w ago

It depends too what you want to return while it's running. You could just throw "1" in the cache api w/ a ttl and then check if it exists and abort. There's the rate limiting api too: https://developers.cloudflare.com/workers/runtime-apis/bindings/rate-limit/ but not much better then abusing cache, still PoP local & more restrictions on durations

Cloudflare Docs

Rate Limiting · Cloudflare Workers docs

Define rate limits and interact with them directly from your Cloudflare Worker

devinOP•4w ago

My scenario is more like: user may click a button repeatedly and try something which is fine (can go through expected code path). But in the code path I want to trigger an alert once and only once

Chaika•4w ago

well, to what degree do you need it to not run twice? I suppose it depends what you mean by Alert

devinOP•4w ago

e.g., trigger a network request which alerts team in slack

Chaika•4w ago

if it's not super critical you could even just have the client track presses and send along a "dontSendAlert" flag

devinOP•4w ago

I don't hate it but it still keeps us at the mercy of the network request which could be tampered with

Chaika•4w ago

Well if it's super critical, use Durable Objects, spin one up per team or some natural division to keep the rps below ~1k, then store the last alert time in storage, and check it. DOs are global singletons, single-threaded v8 isolates and storage is strongly consistent, you'll never send two within the timespan. Otherwise you have everything inbetween those consistency levels like abusing cache/rate limiting binding being per PoP, etc. Since those are entirely per PoP if someone wanted to, they could hit a bunch of diff CF locations with the request to trigger it faster then expected

devinOP•4w ago

Any idea why we're undefined here Chaika?

  const myCache = await caches.open('custom:cache');
  const key = 'https://bla.com';
  const cacheKey = new Request(key, {
    headers: new Headers({
      'cache-control': 'public, max-age=3600',
    }),
  });

  await myCache.put(cacheKey, new Response('hello'));

  const res = await myCache.match(cacheKey);

  console.log(res);

  const myCache = await caches.open('custom:cache');
  const key = 'https://bla.com';
  const cacheKey = new Request(key, {
    headers: new Headers({
      'cache-control': 'public, max-age=3600',
    }),
  });

  await myCache.put(cacheKey, new Response('hello'));

  const res = await myCache.match(cacheKey);

  console.log(res);

Chaika•4w ago

first thing that comes to mind is workers.dev or local dev don't have cache, need to custom domain Either via Workers Custom Domain or Workers Route

devinOP•4w ago

I got it to work. The cache control was on request not response

Gaming

Programming

create TTL cache in CF worker

Did you find this page helpful?