what's the point of workers?
Hi, I'm new to cloudflare workers. Trying to figure out what even is the point if you can't seemingly import your libraries??
Like I'm trying to add a custom watermark to my image (cuz cloudflares apo didn't work for some reason) using canvas/jimp, and I can't even do that?
I think it's a skill issue but then how do I import node libraries?
62 Replies
Workers aren't node, they're v8 (chrome's js engine) with a web-like environment. There's some node compat: https://developers.cloudflare.com/workers/runtime-apis/nodejs/ but a lot of stuff isn't going to be supported no matter what, like file system or raw network access
Cloudflare Docs
Node.js compatibility · Cloudflare Workers docs
Implemented Node.js runtime APIs and enablement instructions for your Worker project.
You can import libraries which are built for the web/don't depend on node stuff, or ones that only depend on stuff which is supported like buffer
ok starting to get it. How can I do that? For example, import jimp
In a larger sense, Workers run on the same machine as your request hits, in all ~330 or however many CF Locations, so they need to be lean (even Paid is restricted at 10 mb), super fast to startup, and fast: https://developers.cloudflare.com/workers/reference/how-workers-works/
Cloudflare Docs
How Workers works · Cloudflare Workers docs
The difference between the Workers runtime versus traditional browsers and Node.js.
gotcha. I just need to add watermark to my image. In my nuzt3 serverit takes about 200ms in server-side with jimp
You can google and see if anyone got it working with Workers, or try to see what people are using, as jimp says though "written for node" and looks like its options are file system (doesn't exist) or http depending on a node network lib which isn't going to work either
aha, got it
what about cloudflares own API for adding watermarks? I tried it yesterdat, and for some reason it literally just returned the image, with no added watermark. I'd provide code, but can't at the moment sadly
(may worth mentioning that's going to be way too long for free, would need Workers Paid anyway, free only gets 10ms of CPU Time)
yeah, I'm on paid version
appreciate the mention
You mean Image Transformations? It'll only work on a route/custom domain, and then make sure you have it enabled for that website/zone (under Images -> Transformations)
if you don't have it enabled, it'll just return it untransformed
Okay. The other thing I worry about is how's that gonna work for my case.
I need to transform the image user uploaded BEFORE uploading it to the R2 I'm using. Can I somehow load an image using base64?
Or do I have to upload it to the server first, and then have the worker fetch it?
Cuz I'm using nuxthub with cloudflare r2 integration, and my media is settled up in a specifics domain media.jart.lol/"imagelink", which is cache configured and anything. So I'd like to keep the same domain media.jsrt.lol
I don't believe there's any way around uploading it first, you could try though via a data uri
fwiw's image transformation's billing is built around not doing that. It costs you the same to transform one unique image/settings once in a month as it does a million times
Meaning that I can't just fetch the new link workers will output, I need to send workers local image user uploaded, then receive the image, and upload it to r2
So, you mean I should upload it first, then have the workers upload transformed version, and use the transformed images link? And to delete the first uploaded non-transformed image?
Cuz my use case is getting images from the user and uploading them. It's not static images or anything. Huge image input per month. Maybe like 500-1000 images
And all images are gonna be unique
yea, upload to r2 -> fetch transformed -> put transformed into r2, sometimes you want to keep the original anyway
Do you think it's viable as an infrastructure? Uploading 2x amount the images?
keeping the original, or what exactly?
images are usually pretty small. It really depends on what you're doing. I can understand why for a free project you'd want to delete the originals, for one people are paying for, I've kept the original and then transformed version(s)
Got you. Really appreciate it, it helps a lot!
Will try tackling my way through it
sure, some of its just paranoria too "what if a user uploaded a copy of an important picture, deleted it from their end, and the one in a billion chance happened and the image library/api returned junk on trying to transform it, so we just lost it"
I've never had/heard of CF's Transformation return junk with no error/a useless image, but just saying
Haha, okay
Oh
And one more question if we're talking about cloudflares transformations
Ideally, I'd want to use nuxtimage to serve different sized images to people on mobiles and desktops
Does it also make copy of images on cloudflsre when I transform them like that?
If you're using CF's integration, you mean? https://image.nuxt.com/providers/cloudflare
Yes
no, it's just temp in cache based on the settings you input
Niceee
and if you're on the latest images plan, you're only billed once for a unique transformation per month, regardless of cache
https://developers.cloudflare.com/images/pricing/#images-transformed
That's really convenient
Got you
And can you advice me what baseURl stands for?
In nuxtImg
I have never used that component before but it looks like it's just the root your website will live on, ex: if you deploy your project on mycoolproject.mydomain.com, it'd be
https://mycoolproject.mydomain.com, so that it can use https://mycoolproject.mydomain.com/cdn-cgi/image/ for image transformationsGot you! I just noticed the pricing tags for transformations - goddamn. Will think about it, but prob gonna be tought as I have like 1000 images/user each month. On a free plan of my product that sucks
Appreciate it once again.
I also wanted to ask about how can I deploy my custom backend alongside my deployed cloudflare pages website. Can I ask you about it, or should I ask somewhere else?
I wanna deploy python, maybe rust. Or even a custom JavaScript without limitations of workers
just as a normal origin hosted on a vps somewhere/some other infra? Just a proxied record will do, and if you need SSL CF has Origin Certificates under SSL/TLS -> Origin Server which last for 15 years and are trusted by CF
If origin hosted means on the same domain with 0 ping then yes (sorry for my lack of knowledge)
So will the ping be 0, and will count as local if I deploy it with origin certificate?
Oh, when you say want to deploy, you were asking for providers/options? CF has nothing but Workers for deploying code, no container or vps offerings
You'd need to find a different provider, either more managed like fly.io/container level, or a vps provider, and throw rust/python on there.
CF Workers has some support for Python (But no packages) and Rust (but limited, all the other workers limits still apply)
I can deploy it on something like Railway (which is similar to fly.io), but will I be able to get the ping to 0 between my cloudflare pages deployed website, and this deployed backend?
If I use the same domain
0 ping is impossible, even you were running it on a server physically next to you, both connected to the same LAN, there would still be some latency
If your Pages app was calling your railway app/backend from a function/worker, there's going to be some latency to get to Railway yea
If your client calls the backend/you are using client-side js to do fetches to your backend, then you have
User -> ISP -> Railway (which is just GCP iirc)
User -> ISP -> CF
yeah and the main problem would be that while cloudflsre has a great cdn, so the ping for users will be minimal, but Railway only has one server. So for Europeans it's gonna be so slow
they don't have more then one still? rip
I mean that's the issue every app faces though. You have to give up something, or optimize for things
Think they do, but with highly payed plans. I'm using the cheapest for $5
Got you
It's my first app where I really gotta think about infrastructure, so thanks a lot for that!
I was thinking about adding watermark on client side
Which is insanely easy and takes like 80ms
Even if you had the backend in tons of locations, your database layer would then either have to be eventually consistent, or sharding data, or all going back to one location/slow
But if user uploads 100 images, the time piles up, and PC explodes
You're right
if it's client-side then you just queue and do it one at a time, slowly
Oooh wait
Youre right
In this case async parallelism would be worse
So I gotta cap it at like 50 images per thread and wait for them to continue
you just have to decide if it's something they'd do so regularly that them waiting <a while> for them to all be transformed is enough that you'd want to offload it (ex: use a queue on your end and process them) server-side, or slowly do them client-side
Or somehow make a buffer with max 50 concurrent watermartk processing
In Javascript (at least, web-side) you only have a single thread lol
Then how does it run in parallelism?
the magical event loop
MDN Web Docs
The event loop - JavaScript | MDN
JavaScript has a runtime model based on an event loop, which is responsible for executing the code, collecting and processing events, and executing queued sub-tasks. This model is quite different from models in other languages like C and Java.
ehh https://www.geeksforgeeks.org/what-is-an-event-loop-in-javascript/ might be a bit better/more imagines
GeeksforGeeks
What is an event loop in JavaScript ? - GeeksforGeeks
A Computer Science portal for geeks. It contains well written, well thought and well explained computer science and programming articles, quizzes and practice/competitive programming/company interview Questions.
Important thing: async != multi-threaded, two completely different things
Got you
And if you use gpt in web?
GPU*
like webgpu and such? The entire idea of a gpu is that you have something that you can send commands to, to do operations more efficiently. I have not worked with gpus on the web before, but I can say the sending side would still be sync/javascript, and the gpu would probably do multi-threading/its own work on its own time to handle commands, which shouldn't matter to you.
The actual page loading/drawing /renderer is a whole different complex topic which depends on the specific browser, mostly not your concern when you use JS though
There are ways to have background threads in js, using web workers, but my understanding is they're cumbersome to use and only fit some applications. May be worth looking into though
Got you. I will still think about it, but myaim options are:
Server side, which takes 500ms in total per image
Or client side with canvas with about 80ms, but obviously it can be too much for the client. Will prob try to make maximum concurrency buffer with 50 or smth. So it's gonna be 4 seconds per 50 images which is still not ideal, but worth looking into
Thank you so much. I only recently got into webdev (7months or so), and only thanks to people like you I can learn so much faster!
Canvas can be cpu or gpu accelerated, depends on os/browser/etc, it just handles it all for you. You're still stuck waiting for each action to complete though
Can I ask you what would you do were you in my situation? If you needed to add watermark to EVERY image user uploads and do it fast for 1000 images?
And if $0.5 per 1000 images was expensive for you?
Can I ask you what would you do were you in my situation? If you needed to add watermark to EVERY image user uploads and do it fast for 1000 images?Does it have to be fast? What do you mean by fast?
And if $0.5 per 1000 images was expensive for you?Depends on the product. To something people pay licenses for, $0.50 per 1k images is pretty cheap. For a free product/freenium product, could be expensive
Fast like if user uploads 1 image, adding watermark should be added in 200ms max. And, scaling up, if user is trying to upload 1000 images, ideally it would take about a 90 seconds.
I've got a freemium model, and gotta minimize the costs as much as possivle
I mean like I said above it really comes down to what matters for you and your product. Are these paying customers, and people you want to have a good experience even when they upload 1,000 images? Then you could process them server-side in a queue. Are these free customers/low budget/you don't care if it takes 9 years to process 1,000 images on their old intel 2 core?
then sounds like you'd want to just do it cilent-side
lol
Every customer is precious to me, but I get what you mean
yes they're all the equal amount of precious which still isn't enough to justify the amount of dev work and compute for doing it server-side for free lol, is how I'd view it
Then just gotta make sure the operations, while may be slow, but are still smooth with no lags or crashes
Cuz oh my, when my friend uploaded 2000 images, it wasn't looking too hot for his pc
Times for one function were like 40s cuz they all have been fighting among themselves for the computing power of his PC lol
It's like when I have one lane turn and 50 cars tryna fit in
Gonna limit that to 10 at a time, and tell other 10 to chill
Also if you wanna do it client side, you could put that operation into a webworker which a) does not block the main UI while creating the watermarks and b) you can utilize several CPU cores.
https://web.dev/articles/offscreen-canvas
web.dev
OffscreenCanvas—speed up your canvas operations with a web worker ...
This document explains how you can use the OffscreenCanvas API to achieve performance improvements when rendering graphics in your web app.
really appreciate it, will check it out
I made it work on the client side in about 30-40ms for now, not bad
@Murzin @Chaika it appears Jimp now works with CF workers https://github.com/jimp-dev/jimp/releases/tag/v1.0.2 (if it wasnt)
GitHub
Release v1.0.2 · jimp-dev/jimp
This release marks v1 of Jimp and there are some notable breaking changes.
Removed some odd api and made the main api easier to reason about
Rewrote it in TypeScript
Support for ESM/CJS/Browser
Al...
So good! I've also seen news about node_compatibility V2, thanks to unjs. Very cool