Cloudflare Developers•2mo ago

'Global' Worker for Logging (alternative for logflare App)

Cloudflare is discontinuing "Apps" at the end of August. The only "App" I am using is logflare (global http request / response logging) - https://cloudflareapps.com/apps/logflare I've been looking into the best way to migrate to using Workers directly instead of "Apps". The issue I'm running into is, I have many existing Workers for my domain, on many different routes. The one nice thing about "Apps" was, they essentially were a "global" Worker that ran before any other worker in your zone. Because of this, logflare was able to "wrap" every request, including timing information, regardless of it went directly to Origin or through a Worker. I'm wondering if anyone has suggestions for the best way to reproduce this behavior with my own Worker? As I said, for routes without existing workers, it is fairly straightforward. I can just create a custom logflare worker, which runs on those unused routes. My only guess on how to "wrap" existing workers (so that the logging includes their response timing), is turn off routes for all existing workers, and instead have on single "global" worker that inspects the incoming request and then calls the associated existing worker if a route exists for them. This seems... very messy, and would like to avoid that at all costs if I can! Definitely feels like working against the system, instead of with it. Thank you for any tips or advice you can give.

29 Replies

Hard@Work•2mo ago

A tail Worker? https://developers.cloudflare.com/workers/observability/logging/tail-workers/

Cloudflare Docs

Tail Workers · Cloudflare Workers docs

Track and log Workers on invocation by assigning a Tail Worker to your projects.

ArmoredCavalry•2mo ago

I actually did look into this as well (forgot to mention in the post). The trouble I ran into with tail-workers is they will pass all the request information over to use, but very little response information (only the HTTP status code, but not any other HTTP response headers, including whether it was cache hit, etc.)

ArmoredCavalry•2mo ago

https://developers.cloudflare.com/workers/runtime-apis/handlers/tail/#tailresponse

Cloudflare Docs

Tail Handler · Cloudflare Workers docs

The tail() handler is the handler you implement when writing a Tail Worker. Tail Workers can be used to process logs in real-time and send them to a …

Hard@Work•2mo ago

You could log those yourself? Not entirely ideal, but requires a good bit less work

ArmoredCavalry•2mo ago

I was thinking about that, but feel like I get into that weird chicken & egg issue again. On a route where the Worker handles the entire request, how do I log the "response" from that worker (so, when there's no 'fetch' called within that worker). I just get stuck trying to come up with a way to "observe" the worker from the outside. Only hacky thing I've seen would be something like, running the existing workers in an entirely different zone / domain, and "fetching" that from the actual zone / domain I want logging on... I guess this is kinda how the Cloudflare Logflare app functioned.

Hard@Work•2mo ago

Right before the return, slap a

// assuming the response is `res`
console.log({ status: res.status, headers: Object.fromEntries(res.headers.entries()) /* etc. */ });
return res;

// assuming the response is `res`
console.log({ status: res.status, headers: Object.fromEntries(res.headers.entries()) /* etc. */ });
return res;

ArmoredCavalry•2mo ago

So, in the case where a Worker doesn't make any "fetch" to the Origin (instead building the response within the worker), this would still log the eventual HTTP response headers? I might be confusing myself with the flow of things, as trying to think about how the Worker itself knows the eventual cache status (if it is determined later in the pipeline, by cache rules)

Hard@Work•2mo ago

Note that if you don't invoke the cache API yourself, then nothing will be cached Workers bypasses the CDN Cache Stage Yes, that is correct You can also add additional context on there if you want/need to

ArmoredCavalry•2mo ago

Hmm ok, I might have issues w/ cache control then, I was hoping it would still behave the same if I just added a simple pass-through worker that calls "fetch" on Origin, and then sends the logs over to Logflare... (that may just be another can of worms though) As far as the HTTP response headers, did a quick test and I see the content-type header of the response in Tail Worker now, however there's other values that make it to the client (web browser), but don't show in the Tail Worker log, e.g. "cf-ray" for instance.

Hard@Work•2mo ago

I believe the Ray ID should™️ be present in the Request Headers?

ArmoredCavalry•2mo ago

Yeah, you are correct (seems to be in both). I more just used as example though, there are other headers like 'date' that also seem to be missing from response at the time of the console.log

Hard@Work•2mo ago

Yeah, those might be added by higheer layers of the CDN unfortunately

ArmoredCavalry•2mo ago

Yeah, that's what I was afraid of. Which, I guess that's not really a Tail Worker issue, more of a "trying to gather data from within the Worker when it doesn't exist" issue, ha So, I think I'm gonna be stuck on that... Unless I can figure out some other way to "wrap" the call to workers. Just seems like there isn't a great replacement for the way Apps were working, even if I tried to build it myself.

rayberra•2mo ago

Perhaps Snippets + a worker? Haven't tried it, just took a look at the logflare source and the snippets docs out of curiosity. Snippets sounds a lot like " a "global" [limited] Worker that ran before any other worker in your zone". Something similar to https://developers.cloudflare.com/rules/snippets/examples/debugging-logs/.

ArmoredCavalry•2mo ago

@Raylight thank you, this seems very promising! I think I remember seeing snippets once, but it wasn't on my radar at all, so this is great. I did see they are much lower limits than full Workers (5 ms CPU time), but I think that should be ok. From my testing with Workers, the logging was only using ~2ms CPU average. I will definitely give snippets a shot when I get a chance, and report back on results. 😄

Gaming

Programming

'Global' Worker for Logging (alternative for logflare App)