Topics

Sapphire - Imagine a framework•9mo ago

Getting Data from .env File

I created a new Sapphire app using the CLI. I am using swc. And, I want to ask how do access env variables in my bot? The project contains a src/.env file . So, naturally, I thought to put my discord token there and access it in my src/index.ts file as follows.

const main = async () => {
    try {
        client.logger.info('Logging in');
        await client.login(process.env.DISCORD_TOKEN);
        client.logger.info('logged in');
    } catch (error) {
        client.logger.fatal(error);
        await client.destroy();
        process.exit(1);
    }
};

const main = async () => {
    try {
        client.logger.info('Logging in');
        await client.login(process.env.DISCORD_TOKEN);
        client.logger.info('logged in');
    } catch (error) {
        client.logger.fatal(error);
        await client.destroy();
        process.exit(1);
    }
};

However, running this throws an error : Error [TokenInvalid]: An invalid token was provided.. However, it runs if I pass the discord token as a string literal to the client.login() method. So, this tells me the src/.env file isn't getting included in the compiled source code. And, upon reviewing the outputted 'dist folder, it appears this is the case. How would I fix this? Or, is there a different way I am supposed to access environment variables?

Solution:

which is why i was saying to try export const rootDir = join(__dirname, '..', '..', '..');

Jump to solution

89 Replies

MRDGH2821•9mo ago

https://nodejs.org/en/learn/command-line/how-to-read-environment-variables-from-nodejs

Node.js — How to read environment variables from Node.js

Node.js® is a JavaScript runtime built on Chrome's V8 JavaScript engine.

MRDGH2821•9mo ago

Can be done with Node 20 and above

MRDGH2821•9mo ago

Or use this https://www.npmjs.com/package/dotenv

npm

Loads environment variables from .env file. Latest version: 16.4.5, last published: 11 days ago. Start using dotenv in your project by running npm i dotenv. There are 40121 other projects in the npm registry using dotenv.

MRDGH2821•9mo ago

On a side note, Sapphire CLI generates a bot template for you to use which takes care of loading env files 🤔

Je Suis Un AmiOP•9mo ago

I am using the template Sapphire CLI generated. How do I get it to load the env files like you said? The generated project structure came with a sec/.env file. So I put all my env vars there (discord token, FireBase config info, etc…). The Sapphire docs didn’t mention anything about handling .env variables. How could I access the values from the env file?

KaydaFox•9mo ago

env things get loaded to process.env so if you named your discord token "DISCORD_TOKEN" inside of your env file, you an access it with process.env.DISCORD_TOKEN if youre using one of the templates, they should come with @skyra/env-utilities which handles the loading of the env file, which is done inside of src/lib/setup.{.js | .ts}

Je Suis Un AmiOP•9mo ago

Yeah. I saw the code in the setup.ts that loads the .env. Let me check my config again. On a side note, is it normal for the dist output directory to not contain the .env file from src/.env? Because mine doesn’t. And I am tempted to think that could be the source of my error. There was also a little error in the template because it was looking for “dist/index.js” to start the bot, when the compiled output actually had it in dist/src/index.js. I fixed that by just changing the script in package.json.

KaydaFox•9mo ago

that is normal kind of odd how it put it stuff into dist/src though isntead of just dist/ im assuming that's something with your tsconfig

Je Suis Un AmiOP•9mo ago

I haven’t changed anything in the tsconfig. Only thing I set that isn’t “recommended” is to use swc. That’s what’s throwing me off.

KaydaFox•9mo ago

ive used swc for one of my bots and it didnt have any effect like this and i dont think it should either since it just compiles the code to js, the env is loaded at runtime can you log process.env somewhere in your code? (after the setup script has run)

KaydaFox•9mo ago

https://github.com/sapphiredev/examples/tree/main/examples/with-swc also i dont think using swc is that not recommended since there is an official example with it

GitHub

examples/examples/with-swc at main · sapphiredev/examples

Various examples of setting up your bot with the Sapphire Framework - sapphiredev/examples

Je Suis Un AmiOP•9mo ago

Here's what it spat out

{"TERM_PROGRAM":"Apple_Terminal","PROJECT_CWD":"<my-swd>","INIT_CWD":"<my-init-cmd>","TERM":"xterm-256color","SHELL":"/bin/zsh","TMPDIR":"<some-value>","TERM_PROGRAM_VERSION":"445","NODE_OPTIONS":"","TERM_SESSION_ID":"<session-id>","USER":"foo","SSH_AUTH_SOCK":"some-value","__CF_USER_TEXT_ENCODING":"foo","npm_execpath":"foo","PATH":"foo","_":"/usr/local/bin/yarn","LaunchInstanceID":"B5175E11-3F97-4120-B00F-D1C7479EB1B1","__CFBundleIdentifier":"com.apple.Terminal","PWD":"value","npm_lifecycle_event":"start","npm_package_name":"arenamate","LANG":"en_US.UTF-8","XPC_FLAGS":"0x0","npm_package_version":"1.0.0","XPC_SERVICE_NAME":"0","HOME":"/Users/patrickluy","SHLVL":"1","LOGNAME":"patrickluy","BERRY_BIN_FOLDER":"/private/var/folders/ps/2dk1s5tx0bl85q8y2qbr_kw40000gn/T/xfs-6e28e087","npm_config_user_agent":"yarn/4.1.0 npm/? node/v18.17.1 darwin x64","SECURITYSESSIONID":"186a4","npm_node_execpath":"/private/var/folders/ps/2dk1s5tx0bl85q8y2qbr_kw40000gn/T/xfs-6e28e087/node","NODE_ENV":"development"}

{"TERM_PROGRAM":"Apple_Terminal","PROJECT_CWD":"<my-swd>","INIT_CWD":"<my-init-cmd>","TERM":"xterm-256color","SHELL":"/bin/zsh","TMPDIR":"<some-value>","TERM_PROGRAM_VERSION":"445","NODE_OPTIONS":"","TERM_SESSION_ID":"<session-id>","USER":"foo","SSH_AUTH_SOCK":"some-value","__CF_USER_TEXT_ENCODING":"foo","npm_execpath":"foo","PATH":"foo","_":"/usr/local/bin/yarn","LaunchInstanceID":"B5175E11-3F97-4120-B00F-D1C7479EB1B1","__CFBundleIdentifier":"com.apple.Terminal","PWD":"value","npm_lifecycle_event":"start","npm_package_name":"arenamate","LANG":"en_US.UTF-8","XPC_FLAGS":"0x0","npm_package_version":"1.0.0","XPC_SERVICE_NAME":"0","HOME":"/Users/patrickluy","SHLVL":"1","LOGNAME":"patrickluy","BERRY_BIN_FOLDER":"/private/var/folders/ps/2dk1s5tx0bl85q8y2qbr_kw40000gn/T/xfs-6e28e087","npm_config_user_agent":"yarn/4.1.0 npm/? node/v18.17.1 darwin x64","SECURITYSESSIONID":"186a4","npm_node_execpath":"/private/var/folders/ps/2dk1s5tx0bl85q8y2qbr_kw40000gn/T/xfs-6e28e087/node","NODE_ENV":"development"}

Doesnt look like it loaded the DISCORD_TOKEN, or any of the other .env values. hmm.

KaydaFox•9mo ago

in that case, seems like the path may not be correct also is your setup script definitely being called? in my bots at the top of the main file i always have import '#lib/setup'; be first

Je Suis Un AmiOP•9mo ago

My index.ts looks like this:

ts import './lib/setup';

import { LogLevel, SapphireClient } from '@sapphire/framework';
import { GatewayIntentBits, Partials } from 'discord.js';

const client = new SapphireClient({
    defaultPrefix: '!',
    regexPrefix: /^(hey +)?bot[,! ]/i,
    caseInsensitiveCommands: true,
    logger: {
        level: LogLevel.Debug
    },
    shards: 'auto',
    intents: [GatewayIntentBits.GuildMembers, GatewayIntentBits.GuildMessages, GatewayIntentBits.Guilds, GatewayIntentBits.MessageContent],
    partials: [Partials.Channel],
    loadMessageCommandListeners: true
});

const main = async () => {
    try {
        client.logger.info('Logging in');
        console.log('env: ' + JSON.stringify(process.env));
        await client.login(process.env.DISCORD_TOKEN);
        client.logger.info('logged in');
    } catch (error) {
        client.logger.fatal(error);
        await client.destroy();
        process.exit(1);
    }
};

void main();

ts import './lib/setup';

import { LogLevel, SapphireClient } from '@sapphire/framework';
import { GatewayIntentBits, Partials } from 'discord.js';

const client = new SapphireClient({
    defaultPrefix: '!',
    regexPrefix: /^(hey +)?bot[,! ]/i,
    caseInsensitiveCommands: true,
    logger: {
        level: LogLevel.Debug
    },
    shards: 'auto',
    intents: [GatewayIntentBits.GuildMembers, GatewayIntentBits.GuildMessages, GatewayIntentBits.Guilds, GatewayIntentBits.MessageContent],
    partials: [Partials.Channel],
    loadMessageCommandListeners: true
});

const main = async () => {
    try {
        client.logger.info('Logging in');
        console.log('env: ' + JSON.stringify(process.env));
        await client.login(process.env.DISCORD_TOKEN);
        client.logger.info('logged in');
    } catch (error) {
        client.logger.fatal(error);
        await client.destroy();
        process.exit(1);
    }
};

void main();

KaydaFox•9mo ago

okay so you do have that imported. id check to make sure your path for the env file is correct then

Je Suis Un AmiOP•9mo ago

Only thing I have changed so far here are the intents inserting process.env.DISCORD_TOKEN in the login() function.

Je Suis Un AmiOP•9mo ago

It should be correct. This is all from the template the CLI spun up.

No description

Je Suis Un AmiOP•9mo ago

Here's my dist directory though. As you can see, the .env file is missing from this directory. But, you did say that shouldn't be the problem right?

No description

KaydaFox•9mo ago

i am assuming it is to do with the path, since youre nested an extra level can you get your code to compile down into dist/ instead of dist/src/ (or add an extra level back to the path youre using)? the default cli projects all compile down to dist/ and are setup for that nest level

Je Suis Un AmiOP•9mo ago

which path are you referring to? in the tsconfig?

KaydaFox•9mo ago

the path for the env file

MRDGH2821•9mo ago

Woah, you have to put the env file in the root folder not src

KaydaFox•9mo ago

the default templates all have it in src/ the location of it doesnt matter too much as long as the path name is correct

MRDGH2821•9mo ago

I.e. besides the package.json

Je Suis Un AmiOP•9mo ago

Oh? well, the template put it in src. lol. Let me move it

MRDGH2821•9mo ago

Try it once

KaydaFox•9mo ago

chances are moving it would work too since its moving it back a level

MRDGH2821•9mo ago

Either it's a mistake or the build step is not copying the env file in dist Or my knowledge is outdated

KaydaFox•9mo ago

the env file shouldnt be copied

KaydaFox•9mo ago

this is a project setup with the sapphire cli, its by default inside of src/

No description

MRDGH2821•9mo ago

Well, how is the compiled code going to read from src/.env.local ?

KaydaFox•9mo ago

the way its handled in mine

// constants.ts
export const rootDir = new URL('../../', import.meta.url);
export const srcDir = new URL('src/', rootDir);

// setup.ts
setup(new URL('.env.local', srcDir));

// constants.ts
export const rootDir = new URL('../../', import.meta.url);
export const srcDir = new URL('src/', rootDir);

// setup.ts
setup(new URL('.env.local', srcDir));

MRDGH2821•9mo ago

Fair point

Je Suis Un AmiOP•9mo ago

It's still saying an invalid token was provided.

KaydaFox•9mo ago

and with the sapphire swc template this is handled in the same level

export const rootDir = join(__dirname, '..', '..');
export const srcDir = join(rootDir, 'src');

setup({ path: join(srcDir, '.env') });

export const rootDir = join(__dirname, '..', '..');
export const srcDir = join(rootDir, 'src');

setup({ path: join(srcDir, '.env') });

MRDGH2821•9mo ago

You may have to modify your setup.ts file OR If you are using node v20: After the build step, use this: node --env-file <env file path> ./dist/index.js

Je Suis Un AmiOP•9mo ago

No description

MRDGH2821•9mo ago

Yea in this case the code in .dist will travel back to root then to src, thus correctly reading the .env file

KaydaFox•9mo ago

yeah but in their code its nested an extra level inside of dist/

Je Suis Un AmiOP•9mo ago

Alright. Let try. Actually, I'm gonna try to updating Node to see if that helps.

Solution

KaydaFox•9mo ago

which is why i was saying to try export const rootDir = join(__dirname, '..', '..', '..');

KaydaFox•9mo ago

so a third level back or just have it all compile directly to dist/

Je Suis Un AmiOP•9mo ago

Well, here is my setup file.

ts // Unless explicitly defined, set NODE_ENV as development:
process.env.NODE_ENV ??= 'development';

import { ApplicationCommandRegistries, RegisterBehavior } from '@sapphire/framework';
import '@sapphire/plugin-api/register';
import '@sapphire/plugin-editable-commands/register';
import '@sapphire/plugin-logger/register';
import '@sapphire/plugin-subcommands/register';
import { setup, type ArrayString } from '@skyra/env-utilities';
import * as colorette from 'colorette';
import { join } from 'path';
import { inspect } from 'util';
import { srcDir } from './constants';

// Set default behavior to bulk overwrite
ApplicationCommandRegistries.setDefaultBehaviorWhenNotIdentical(RegisterBehavior.BulkOverwrite);

// Read env var
setup({ path: join(srcDir, '.env') });

// Set default inspection depth
inspect.defaultOptions.depth = 1;

// Enable colorette
colorette.createColors({ useColor: true });

declare module '@skyra/env-utilities' {
    interface Env {
        OWNERS: ArrayString;
    }
}

ts // Unless explicitly defined, set NODE_ENV as development:
process.env.NODE_ENV ??= 'development';

import { ApplicationCommandRegistries, RegisterBehavior } from '@sapphire/framework';
import '@sapphire/plugin-api/register';
import '@sapphire/plugin-editable-commands/register';
import '@sapphire/plugin-logger/register';
import '@sapphire/plugin-subcommands/register';
import { setup, type ArrayString } from '@skyra/env-utilities';
import * as colorette from 'colorette';
import { join } from 'path';
import { inspect } from 'util';
import { srcDir } from './constants';

// Set default behavior to bulk overwrite
ApplicationCommandRegistries.setDefaultBehaviorWhenNotIdentical(RegisterBehavior.BulkOverwrite);

// Read env var
setup({ path: join(srcDir, '.env') });

// Set default inspection depth
inspect.defaultOptions.depth = 1;

// Enable colorette
colorette.createColors({ useColor: true });

declare module '@skyra/env-utilities' {
    interface Env {
        OWNERS: ArrayString;
    }
}

For you guys' reference. Its just the default that came with template. And here's lib/constants

import { join } from 'path';

export const rootDir = join(__dirname, '..', '..');
export const srcDir = join(rootDir, 'src');

export const RandomLoadingMessage = ['Computing...', 'Thinking...', 'Cooking some food', 'Give me a moment', 'Loading...'];

import { join } from 'path';

export const rootDir = join(__dirname, '..', '..');
export const srcDir = join(rootDir, 'src');

export const RandomLoadingMessage = ['Computing...', 'Thinking...', 'Cooking some food', 'Give me a moment', 'Loading...'];

KaydaFox•9mo ago

may you try this please? if that doesnt work then i have no idea what else to try tbh, but it definitely feels like a pathing issue

Je Suis Un AmiOP•9mo ago

Add an extra '..' in the rootDir = join() call? alright. Gotta wait for brew to finish updating node first though.

MRDGH2821•9mo ago

In the older versions of the sapphire template, build output was directly in ./dist/<compiled source code> I'm wondering why there's ./dist/src/<compiled source code>

KaydaFox•9mo ago

its still the same in the current versions wait no, for the swc version it does go to dist/src/

KaydaFox•9mo ago

i just missed that cos of the way vscode does this

KaydaFox•9mo ago

i wonder why it does that all the others go straight to dist/ and i think that's the issue

Je Suis Un AmiOP•9mo ago

tsconfig file looks right though.

json {
    "extends": ["@sapphire/ts-config", "@sapphire/ts-config/extra-strict", "@sapphire/ts-config/decorators"],
    "compilerOptions": {
        "rootDir": "src",
        "outDir": "dist",
        "baseUrl": "src",
        "tsBuildInfoFile": "dist/.tsbuildinfo",
        "paths": {
            "#lib/*": ["./lib/*"]
        }
    },
    "include": ["src"]
}

json {
    "extends": ["@sapphire/ts-config", "@sapphire/ts-config/extra-strict", "@sapphire/ts-config/decorators"],
    "compilerOptions": {
        "rootDir": "src",
        "outDir": "dist",
        "baseUrl": "src",
        "tsBuildInfoFile": "dist/.tsbuildinfo",
        "paths": {
            "#lib/*": ["./lib/*"]
        }
    },
    "include": ["src"]
}

MRDGH2821•9mo ago

I don't have baseUrl property 🤔

Je Suis Un AmiOP•9mo ago

This came with the template. So, idk. Let me try removing it to see if that helps.

KaydaFox•9mo ago

after doing this on a freshly setup swc template it works fine but before that it doesnt so im going to assume that'll solve it for you unsure why swc puts stuff into dist/src/ though its not the tsconfig it seems since i switched it out for the custom one i use in all my bots and its the same output

Je Suis Un AmiOP•9mo ago

Okay, that fixed it.

MRDGH2821•9mo ago

Swc's build options may need a change Apart from that if nothing works, Node v20+ itself got a nice feature built in. https://nodejs.org/en/learn/command-line/how-to-read-environment-variables-from-nodejs This should work in any case node --env-file <env file path> index.js (Unless you are building a docker image, then there's different way to provide variables)

Node.js — How to read environment variables from Node.js

Node.js® is a JavaScript runtime built on Chrome's V8 JavaScript engine.

Je Suis Un AmiOP•9mo ago

So, I guess the rootDir in lib/constants.ts just needed to point up one level.

KaydaFox•9mo ago

glad to hear it fixed it, sorry it took so long. seems like a bug with the layout of some stuff in the swc template

Je Suis Un AmiOP•9mo ago

Thanks for the help. Not a problem at all. Just glad I can get off using Akairo for my bots. lol

MRDGH2821•9mo ago

On a side note, can you see what is the output directory structure of this command: npx swc src -d disttest

KaydaFox•9mo ago

still the same

KaydaFox•9mo ago

No description

KaydaFox•9mo ago

that was straight after running it

No description

MRDGH2821•9mo ago

npx swc src -d disttest --strip-leading-paths

MRDGH2821•9mo ago

https://swc.rs/docs/usage/cli#--strip-leading-paths

@swc/cli – SWC

SWC is an extensible Rust-based platform for the next generation of fast developer tools. It's used by tools like Next.js, Parcel, and Deno, as well as companies like Vercel, ByteDance, Tencent, Shopify, and more.

KaydaFox•9mo ago

ah that does it probably an easier fix for what i did in a pull request which was this

MRDGH2821•9mo ago

This should do it Because I have seen people using ts-node to test the bot. And that setup.ts file with triple .. will send the env file searching outside the source code's directory

KaydaFox•9mo ago

i keep forgetting docs exist ;-; instead i was playing around manually with swcs config

MRDGH2821•9mo ago

I.e. rootDir will be the parent folder of source code if ts-node is used

KaydaFox•9mo ago

i cant even get ts-node to work with it in the first place

D:\code\dev\test-bot\node_modules\ts-node\dist\util.js:62
    return value.replace(backslashRegExp, directorySeparator);
                 ^

TypeError: value.replace is not a function

D:\code\dev\test-bot\node_modules\ts-node\dist\util.js:62
    return value.replace(backslashRegExp, directorySeparator);
                 ^

TypeError: value.replace is not a function

lol

MRDGH2821•9mo ago

Try tsc-watch 🤔

Spinel•9mo ago

TL;DR: Do not use ts-node, use tsc-watch instead. We very strongly discourage using ts-node because it was never meant to be used for bots. ts-node is designed for REPL purposes. That's short for Read Eval Print Loop. Which means to read some code, dump it in an eval() statement, print the result, and loop. A discord bot is not that. A Discord bot sets up a permanent web socket connection to the Discord server and connects to the rest gateway. There is read yes, but no eval, no print, and no loop. So what should you use instead? The most ideal way is to just use the watch flag of tsc (tsc --watch) and run node dist/index.js to run your bot, then cancel that process and restart it when you have changes that require restarting. You would open 2 terminal tabs, 1 in which you run tsc --watch and another in which you run the bot. This is, in particular, the most ideal way, because Discord has a limit to the amount of times you can log in with your bot, or register commands, per day. Constantly logging in over and over again due to an auto-restarting process will get you close to that limit very quickly and once you exceed it, your development will be halted entirely for the current day. However, this can be quite tedious so a great package to use instead is tsc-watch.

KaydaFox•9mo ago

I mean, that would take us back to using regular tsc instesd of swc like the OP was using

MRDGH2821•9mo ago

Oh right 🤔

MRDGH2821•9mo ago

Try this actually https://github.com/swc-project/swc-node

GitHub

GitHub - swc-project/swc-node: Faster ts-node without typecheck

Faster ts-node without typecheck. Contribute to swc-project/swc-node development by creating an account on GitHub.

KaydaFox•9mo ago

then yeah that does present us with the issue you mentioned I don't personally see that as a big issue since using stuff like that for long running things is bad practice anyways, but regardless your fix is better I think

Je Suis Un AmiOP•9mo ago

@KaydaFox @MRDGH2821 Okay. One other small issue with the swc template I should probably let you guys know about. So, after I fixed the .env problem, like mentioned, the bot was running. But it wasn’t registering any commands. To fix this, I needed to edit the build script in package.json to swc src -d dist —strip-leading-paths, as mentioned by @MRDGH2821 That’s all from me. Don’t really use ts-node. And, frankly don’t have the time right now to tinker with it. Everything (as in everything included in the template) seems to be working for me as of testing today. I’ll open another thread if I have other issues as I start building out this bot. 🙂

MRDGH2821•9mo ago

Nice!

KaydaFox•9mo ago

cool, glad you got it all working in the end ^^

Je Suis Un AmiOP•9mo ago

Thanks… don’t think that swc template was very well tested lol.

KaydaFox•9mo ago

I hadn't used the template before until last night. Last time I used swc I just replaced tscs build with it manually lol Unsure how used that template is

Je Suis Un AmiOP•9mo ago

First time using it too (and first time using Sapphire). It definitely has its quirks. Even after the changes I made, it still doesn’t like my start script pointing to dist/index.js (can’t find env vars again… but it can load commands now). So, it still needs to point to dist/src/index.js. Might just file a bug report later today tbh. Can’t be the only one facing this.

MRDGH2821•9mo ago

About this option tho, I see this as a new change. It wasn't there when I was using it back in 2022 I was using: "@swc/core": "^1.3.14", "@swc/helpers": "^0.4.12",

KaydaFox•9mo ago

remove the thing i asked you to do for the rootDir so replace it with what it was originally, since we altered it to account for the directory nesting, but since --strip-leading-paths alters that back to what its expected to be, and then envVar should load again last i used SWC was mid to late 2023, although i also didnt have this issue back then

Je Suis Un AmiOP•9mo ago

Okay. So, remove the extra ”..”? Kk. Will try later. Working on something else rn.

KaydaFox•9mo ago

Yeah ^^

Je Suis Un AmiOP•9mo ago

Just tried it. Looks like it worked. So, I guess the solution here is to change the build script to add the —strip-leading-paths flag.

KaydaFox•9mo ago

yeah, they did that in this commit ^^ https://github.com/sapphiredev/examples/commit/81ff2a4cb69a61046eaff1ec33b2b443049c4eec

GitHub

fix(swc): add --strip-leading-paths · sapphiredev/examples@81ff2a4

closes #195

Je Suis Un AmiOP•9mo ago

Oh cool. So I don’t need to submit a bug report 🙂

Favna•9mo ago

It wasn't. Most of use either either tsup (nearly just as fast tbh) or tsc. tsup has the advantage of using esbuild in the background which is a compiler that's much more popular and has much bigger community behind it.

Sapphire - Imagine a framework Join

Welcome to the Sapphire Discord server! The next-gen object-oriented Discord.js bot framework can be found here.

2.2KMembers

View on Discord

Want results from more Discord servers?

Add your server