[VERY EXTREMELY CURSED] How can I access process.env (at start time)
Ahoy! I need .env to instantiate my jwt server. I've spent so much time of my life reading minified code and dealing with build tools, BUT I HAVE NO IDEA WHATS GOING ON!!! Solid start somehow uses vite, rollup, esbuild, vinxi, and nitro. and then I suppose cf is doing its own polyfills Has javascript gone too far???? But anyway
My first attempt is
but this didn't work, process.env is undefined despite node.js compat being enabled. Looking inside the bundled code, I see
.use(jwt({secret:J.env.JWT_SECRET
, what is J? const J=H.process
. Ok lets zoom out
ok so nitro is being "helpful" and injecting unenv into the game. Ok so its looking at globalThis.
Whats in globalThis? console.log(globalThis)
OH LAWDY WHAT IS THIS (hahah globalThis get it puns?)
1/?48 Replies
Ok so it should be
console.log(globalThis.process?.env || globalThis.__env__)
maybe like unenv does right?
Well that just gives me [Object: null prototype] {} [Object: null prototype] {}
[Object: null prototype] {}
, so there's obviously nothing there, right?
Ok so I saw https://developers.cloudflare.com/workers/runtime-apis/nodejs/process/
Which says
In the Workers implementation, there is no process-level environment, so env is an empty objectSo maybe we can directly acess it, but I think this is saying that I don't get this :( Still, lets try:
console.log(globalThis.process?.env.JWT_SECRET)
gives undefined
when the server starts. Not looking good. But wait! When I do a request, I see it! Wait, so what if I directly acess process?.env.JWT_SECRET
? OH LAWD WOULD YOU LOOK AT THAT! IT WORKS!
But it doesn't work outside of that request.
When the server boots up, it throws an error because process.env.JWT is undefined and it needs that to start.
Ok, so what other ways do we have? Well you can see import { env, nextTick } from "node:process";
in that doc page above. Does anything change?
Ow. Ok, Solid provides getRequestEvent()?.nativeEvent.context.cloudflare.env.JWT_SECRET
, and again this is undefined in the initialization as expected because there's no request event when initializing.Cloudflare Docs
process · Cloudflare Workers docs
To use Node.js APIs in your Worker, add the nodejs_compat compatibility flag to your wrangler.toml file.
next-on-cloudlare
How does
next-on-cloudflare
do it with getRequestContext().env
?
Woah, __cloudflare-request-context__
, we're in some deep stuff, but why is it undefined? Oooh, next on pages does... I don't even want to know:
Now this is JavaScript
So we're looking in
globalThis
for __cloudflare-request-context__
. But this still doesn't explain how env is being populated
- https://github.com/cloudflare/next-on-pages/blob/main/internal-packages/next-dev/src/shared.ts
But tbh maybe it doesn't even matter, I don't have any reason to believe this works outside of a request, given that its... requestContext.
---
So yeah, the ultimate question is is it possible to access env outside of a request context?GitHub
next-on-pages/internal-packages/next-dev/src/shared.ts at main · cl...
CLI to build and develop Next.js apps for Cloudflare Pages - cloudflare/next-on-pages
So no, I don't believe that you can get environment variables outside of a request context because they're only passed into the actual request handler by the runtime itself (i.e. global code can't access them).
Frameworks like
next-on-pages
use a somewhat-hacky method of putting the environment variables into the global context but I believe it's very much next-on-pages-specific and has a lot of moving parts (i.e. probably not something that would be easy to do yourself for Solid Start).
@Better James (hope the ping is OK) would probably have more info on how next-on-pages do it specifically, I'm not well-versed with the internals of that framework.I believe too that only works because those scripts are loaded when a Request Context has already been initialized
next-on-pages runs your requests inside an AyncLocalStorage that holds the values for the env vars, and process.env is overridden with a proxy object to interact with the ALS. And request context is done in a similar way.
Outside of a request context is not possible. How does code run outside of a request context? It doesn't, everything should be run within the context of a request if it is supposed to depend on that request's context. Maybe you're trying to do something build-time related, in which case it will be up to how your framework does things at build-time. But run-time, everything runs within the lifecycle of a request.
How does code run outside of a request context?This would make sense, but when I start miniflare it crashes instantly. The command run was
pnpm build && wrangler pages dev dist/ --compatibility-flags nodejs_compat
so maybe there's a bug with miniflare, because I agree that it makes sense that code shouldn't be running outside of a request context
How are you ensuring that this code isn't running outside of the request context?
Well 1. Wouldn't all code technically be running inside of a request context as cloudflare doesn't have long running processes?
but more importantly
2. The specific answer is I'm not ensuring this. Its plainly imported
import client from "../../../server/queries/index";
which causes this code to run on import. But the thing is that its running again whenever I get a req, as I would expect because workers isn't long runningfor all I know this is a miniflare problem and doesn't happen in prod
A Worker is given a few milliseconds to boot up before an event is actually passed in. In this case, the outcome is expected, as the variables are not available in the bootup phase
but the miliseconds to bootup you describe never happened. When the server started, this code ran without a request, and caused the entire server to crash. Subsequent requests, despite theoretically running before being passed an event, correctly have the env set
That sounds right. The bootup runs once, so it fails on the first request, but on subsequent ones it passes
does this mean that workers are long running?
Sort of? They can be reused for multiple requests, but you aren't ever guaranteed this will happen
Basically though, for stuff where you need
env
bindings, we recommend something like this: Adjusted for whatever framework you are using
Or just creating the Thing
on each requestI can't do that, because I'm instantiating middleware. High level, something like
now theoretically, I could wrap all this in closures, like you propose: and I hope you can see how this scales terribly, especially when considering factors like type safety.
Why not this?
because now I'm forced to write my entire codebase inside of
which, again, does not scale
Well no, only adding the handlers to the router:
Otherwise, there are a bunch of other frameworks that are more suited to this kind of thing
I recommend
itty-router
/hono
Thats assuming that authPlugin is only used inside one route group
Basically, you need some way of passing the context(not the request) into the router when a request is run, which at least from a cursory skimming of the docs, Elysia does not support
hono has the exact same problem. I took this code out straight out of the docs, except i replaced the hardcoded string with process.env.JWT_TOKEN and added comments
This works fine, no?
and here's the kicker. I was confused how hono could possibly be making this secure when any other implementation would be blindly believing that the getter will always be defined inside a request context when we already know its not defined at all points in the lifecycle, so I went into the source code.
and, hmm this looks very similar to the code running in Elysia, which also basically throws very early if there's nothing. But you see the bug? Look at that if statement! its not actually checking if secret is defined
Maybe a PR on that one then?
I absolutely will, but when it is merged there WILL be breakage
because this will break for every single person running this code FROM THE HONO DOCS inside production
@yusukebe could you check ^ out please?
You can try this syntax:
theoretically, this would work, I think but this is
1. an ergonomic problem for the actual library
2. I need to deliberate on this solution more to ensure it is actually secure. As we see above, these mistakes are incredibly easy to make
Make no mistake, the real problem (barring the fact that hono is currently shipping code with very questionable security) imo is just that process.env isn't defined in all contexts, which seems like a simple fix as process.env cannot change without rebuilding the worker anyway
Well that's just it, afaik Environment Bindings can be changed at runtime, without a full reboot of the isolate
So a
proces.env
that works now may not work for the next requestyeah so similar to the solution above, but also you also need to check to make sure that JWT is defined properly (see issue in code snippet above because the library isn't properly checking), and then you need to update the docs to this new method
based on my extensive knowledge (/s) from this chaos ^^^
its my understanding that process.env cannot be changed outside of the worker (ie in the dashboard) without rebuilding
so again, why not seed it initially with the values there? (the same seed that the first request gets)
I don't think that is the case, or at least that code is supposed to keep that in mind
Earlier versions of Workers did actually have global env vars
But they made them scoped to the request to allow changes to the environment without require a full reboot of the isolate
I'm 99% sure that it was that way at least very recently. But let me check, I have code deployed right now that could help. But in any case I don't see why its relevant
currently test is changed to w, now i changed it to wff in the dashboardand then I reload the page, no change
I still don't see why it can't be seeded with the dashboard values outside of a request
That would
1. significantly improve ergonomics in many cases
2. allow hono to fix this issue without making people rewrite their code to be something objectively ergonomically worse
3. allow elysia to work (though I could make it work in the same method described above)
The second parameter is an object that contains your environment variables (also known as "bindings"). Previously, each variable was inserted into the global scope of the Worker. While a simple solution, it was confusing to have variables magically appear in your code. Now, with an environment object, you can control which modules and libraries get access to your environment variables. This mechanism is more secure, as it can prevent a compromised or nosy third-party library from enumerating all your variables or secrets.https://blog.cloudflare.com/workers-javascript-modules
The Cloudflare Blog
JavaScript modules are now supported on Cloudflare Workers
Now you can use JavaScript modules, also known as ECMAScript or “ES” modules, on Cloudflare Workers. This replaces the old “addEventListener” syntax with a new “import” and “export” semantics that makes it really easy to write reusable, modular code.
Here's one
Also this
Workers using ES modules format do not rely on any global bindings. This means the Workers runtime does not need to set up fresh execution contexts, making Workers safer and faster to run.
Cloudflare Docs
Migrate from Service Workers to ES Modules · Cloudflare Workers docs
Write your Worker code in ES modules syntax for an optimized experience.
Doesn't matter with node.js compatibility which adds a global process.env back
its just... a flakey global process.env
so basically the worst of both worlds
same security ramifications, with ergonomic failures
I think the fact that this can happen is probably problematic given ^^^^
becuase they basically say that only code running inside request context can modify env without mentioning the nuance of process.env
and to be honest, I think its best to keep it that way. That is compatibility
but it should be documented
and the flakeyness should be fixed
and also, again, I maintain that reinitializing the JWT reference every request is questionable because I can't think of a valid use case for why the value of the secret would change between requests which is something that requiring it inside a function enables
So there is a use-case.
If you change a Worker's environment variables, existing isolates may (or may not) be re-used with the new env vars.
That's partly why the old service workers format (which had env vars as globals) was deprecated/is no longer recommended.
I can't think of many reasons I'd actually want to change env at runtime. Seems like a much better use for kv. Now, if it is true you can cache the result of a function in the top level using the pattern described above (
let result;
, thats cool, but has little bearing on actual environment (which is forcefully serialized to a string), especially if the workaround tanks ergonomics of both libraries and end user code.
But all of these opinions on proper ergonomics literally doesn't matter.
Node.js compatibility is broken. Node.js code expects process.env to be defined throughout the entire lifecycle. This current implementation involves environment variables at only some, apparently inconsistent parts of the lifecycle.Workers are not meant to have Node compatibility. Unlike frameworks like Bun which promise full compat, Workers are meant to be their own thing with some optional node compatibility in the form of the
nodejs_compat
flag and some Node-specific APIs and modules (see the docs for more info)
Is it possibly less ergonomic to not use process.env
and use the env
on the handler instead? Possibly, depending on your application, but that's the way the Workers API was designed to allow it to be used specifically for the Workers runtime on Cloudflare.
Sorry if that came across as rude: I wasn't trying to be, just trying to explain why it's not exactly node-compatible directly.no, not at all
and I hope im not coming across that way either
Nope, not at all. 🙂
Right, And I have
nodejs_compat
enabled. One of the modules listed is process. https://developers.cloudflare.com/workers/runtime-apis/nodejs/process/. Currently, process is defined at certain points in the lifecycle, but I haven't seen a clear answer for
1. lessso: why (like the actual development choices that lead to that) it is only defined at some parts of the lifecycle
2. moreso: the exact way I can predict when it will and it won't be populatedHmm... so process itself isn't defined, or
process.env
is empty/not defined?Its empty. So when the worker boots up, it instantly fails if you have code like
it crashes and does not answer subsequent requests. but, if it makes it past initialization, logging
process?.env.JWT_SECRET
at the very top level (like it actually logs the correct value. So basically if you had a handler that just console.logged "got request", and at the top level you logged JWT_SECRET, you would see , so code that expects to always have an env will crash on initialization, but if you can make it past that first second, it will work forever