Kevin Powell - Community•2y ago

More file structure, this time webpack flavoured!

Thought I'd make another thread as this is kind of a different, but very related, question I've been developing locally using webpack and vanillajs. My server is hosted on digitalocean with ubuntu. As mentioned in my last question I build locally, push to a github repo, and then pull that repo in to my server. I plan to use github actions from next week thanks to @joao6246 advice, but for the time being that's the (maybe less than ideal?) set-up I learnt from frontend masters My Server folder structure:

www
|--site-folder
    |--src
        |-- css
        |-- js
        |-- index.html
        // etc
    |--dev
        |-- index.html
        |-- index.bundle.js
        |-- page2.html
        |-- page2.bundle.js
        |-- image.png
        //... assets/more js/html files etc...

www
|--site-folder
    |--src
        |-- css
        |-- js
        |-- index.html
        // etc
    |--dev
        |-- index.html
        |-- index.bundle.js
        |-- page2.html
        |-- page2.bundle.js
        |-- image.png
        //... assets/more js/html files etc...

I'm just wondering both where should I put my express app that's going to serve my html, and how I can better structure webpack's output? As you can see it has just output everything in to one folder. Should I have a script that moves files after every git pull? Or should I be specifying better paths in my webpack config? This is my webpack config at the moment, I guess I could add path.resolve(__dirname, 'dist/js') for my js content, look for an output path option for each of the relevant module rules (for example for images), and do similar with HtmlWebpackPlugin (or add a path before the filename)? I guess I could then place a simple express app in the route directory? Sorry, I'm just a little confused! Appreciate any help, am so close to getting my first self-hosted website up and running!

67 Replies

JWodeOP•2y ago

// webpack config

module.exports = {
    entry: {//snip},
    output: {
        filename: '[name].bundle.[chunkhash].js',
        path: path.resolve(__dirname, 'dist'),
        clean: true,
    },
    module: {
        rules: [
            {
                test:/\.css$/i,
                use:  ['style-loader', 'css-loader', 'postcss-loader']
            },
            {
                test: /\.js$/i,
                use: ['babel-loader'],
                exclude: /node_modules/,
             },
            {
                test: /\.(png|svg|jpg|jpeg|gif|webp)$/i,
                type: 'asset/resource'
            },
        ]
    },
    plugins: [
        new HtmlWebpackPlugin({
            filename: 'index.html',
            template: './src/index.html',
            chunks: ['index']
        }),
        new HtmlWebpackPlugin({
            filename: 'admin.html',
            template: './src/admin.html',
            chunks: ['admin']
        })
    ]
}

module.exports = {
    entry: {//snip},
    output: {
        filename: '[name].bundle.[chunkhash].js',
        path: path.resolve(__dirname, 'dist'),
        clean: true,
    },
    module: {
        rules: [
            {
                test:/\.css$/i,
                use:  ['style-loader', 'css-loader', 'postcss-loader']
            },
            {
                test: /\.js$/i,
                use: ['babel-loader'],
                exclude: /node_modules/,
             },
            {
                test: /\.(png|svg|jpg|jpeg|gif|webp)$/i,
                type: 'asset/resource'
            },
        ]
    },
    plugins: [
        new HtmlWebpackPlugin({
            filename: 'index.html',
            template: './src/index.html',
            chunks: ['index']
        }),
        new HtmlWebpackPlugin({
            filename: 'admin.html',
            template: './src/admin.html',
            chunks: ['admin']
        })
    ]
}

Joao•2y ago

If you are using express to serve content (html, css, images, etc...) you need to specify a folder using express.static (I think that's the function, right?). In that case all you need to do is make sure the output of the build process from webpack is inside this folder. As an aside, you can apply conditional logic on your webpack.config.js file based on whether you are running on development or for production. Although in this case, as far as deployment is concerned, it shouldn't matter too much.

JWodeOP•2y ago

Hmm... interesting. So I guess res.sendFile() won't work as I expect then? Yeah I guess it won't serve the files needed in the html... And yeah, I use a different webpack config file for production vs dev with webpack merge, so I think I've got that side of things covered 🙂 Problem is, how can I tell webpack to put files into that folder XD

// /var/www/example

|-dev
|-build
  |--js
  |--html
|-express-app
  |--public
    |--js

// /var/www/example

|-dev
|-build
  |--js
  |--html
|-express-app
  |--public
    |--js

Like this?

    output: {
        filename: '../express-app/public/js/[name].bundle.[chunkhash].js',
        path: path.resolve(__dirname, 'dist'),
        clean: true,
    },

    output: {
        filename: '../express-app/public/js/[name].bundle.[chunkhash].js',
        path: path.resolve(__dirname, 'dist'),
        clean: true,
    },

The javascript gods don't like me that much XD Edit: whisper it quietly: they do! lol, the dist folder is kind of redundant now but has to exist (I think).

Joao•2y ago

The problem, in this case, with res.sendFile() is that it only sends one file. You could send the index.html file but then it will make requests for additional files for things like images, fonts or scripts. Then you'd have to setup routes for each of those files, or create some route handler that is flexible enough to detect this sort of requests... which of course is just unnecessarily complex. Setting up a static folder that Express.js knows to look into is much better (and also what other web servers do). so you got it working then? That config looks fine to me. Since it's all in the same repo Webpack should have access to whatever folder you want to use as the public to serve files from the server.

JWodeOP•2y ago

I'm just trying to implement it now. The problem is using those paths then breaks my devServer, so I've just split the config between dev and prod.webpack.js, which seems to work. Ideally I'd have some way to move express-app.js to the correct folder, but as it's not a webpack entry point, I'm not entirely sure how to do that - I think there's a copy plugin or loader, but I've said to my friend I'd show him the site in 2 hours so that can wait! 😆 (I'm just editing that file locally on the server for the time being) Fingers crossed I'm on track! (I don't want to jinx it, but I'm getting on with webpack ok this morning!)

// Local

|-dev
  |--src
    |--css
    |--js 
    |--express-app.js
|-build
  |--js
  |--html
|-express-app
  |--public
  | |--js
  |--express-app.js

// Local

|-dev
  |--src
    |--css
    |--js 
    |--express-app.js
|-build
  |--js
  |--html
|-express-app
  |--public
  | |--js
  |--express-app.js

of course pm2 now keeps erroring 😭 edit- now fixed, and replaced with 504 Gateway Time-out. The route works, but sending the file doesn't:

const express = require('express');
const path = require('path');
const app = express();
const port = 3000;

//// Works:
// app.get('/', (req, res, next) => {
//   res.send('hello')
// });

//// Doesn't work
app.get('/', (req, res, next) => {
  res.sendFile(path.join(__dirname, './public/html/index.html'));
});

app.use((error, req, res, next) => {
  res.status(500)
})

app.listen(port, () => console.log(`Listening on port ${port}`));

const express = require('express');
const path = require('path');
const app = express();
const port = 3000;

//// Works:
// app.get('/', (req, res, next) => {
//   res.send('hello')
// });

//// Doesn't work
app.get('/', (req, res, next) => {
  res.sendFile(path.join(__dirname, './public/html/index.html'));
});

app.use((error, req, res, next) => {
  res.status(500)
})

app.listen(port, () => console.log(`Listening on port ${port}`));

Well, that was just the path 😄 Now I've got the problem that I'm cache busting the html. I guess for the moment I can stop doing that on webpack's side.

JWodeOP•2y ago

Joao•2y ago

res.sendFile(path.join(__dirname, './public/html/index.html')); You don't need to dot when joining paths

JWodeOP•2y ago

Yeah, without the dot also gets the same error does the mime type error mean at least it's finding the file, right?

JWodeOP•2y ago

without the dot:

JWodeOP•2y ago

// express-app/public/html/index.html

<!doctype html><html lang="en">
  <head>
    <meta charset="UTF-8">
    <meta http-equiv="X-UA-Compatible" content="IE=edge"><meta name="viewport" content="width=device-width,initial-scale=1">
    <link rel="preconnect" href="https://fonts.googleapis.com">
    <link rel="preconnect" href="https://fonts.gstatic.com" crossorigin><link href="https://fonts.googleapis.com/css2?family=Montserrat:wght@400;500;600;700&display=swap" rel="stylesheet">
    <title>Document</title>
    <script defer="defer" src="../../../dist/../express-app/public/js/index.bundle.90c93fa382b7dac7021d.js"></script>
  </head>
      <body><div class="app"></div></body>
  </html>

<!doctype html><html lang="en">
  <head>
    <meta charset="UTF-8">
    <meta http-equiv="X-UA-Compatible" content="IE=edge"><meta name="viewport" content="width=device-width,initial-scale=1">
    <link rel="preconnect" href="https://fonts.googleapis.com">
    <link rel="preconnect" href="https://fonts.gstatic.com" crossorigin><link href="https://fonts.googleapis.com/css2?family=Montserrat:wght@400;500;600;700&display=swap" rel="stylesheet">
    <title>Document</title>
    <script defer="defer" src="../../../dist/../express-app/public/js/index.bundle.90c93fa382b7dac7021d.js"></script>
  </head>
      <body><div class="app"></div></body>
  </html>

Well, that's an interesting script tag... I guess my webpack config wasn't as clever as I thought So it was both the path above & that when I pulled the repo it overwrote use(express.static(...)). It's now loading the site, but without images. I think I'm going to strip my stupid production webpack config and just copy the files across from the build folder manually ---------------------- Update: Success? I dumped all my build files on the server as they were in one big list in the public folder. The weird thing now is I've this list:

// express-app/public
index.html
index.js
admin.html
admin.js
...more asset files

// express-app/public
index.html
index.js
admin.html
admin.js
...more asset files

and an express-app.js file like this:

const express = require('express');
const path = require('path');
const app = express();
const port = 3000;

app.use(express.static('./public'));

app.get('/', (req, res, next) => {
  res.sendFile(path.join(__dirname, '/public/index.html'));
});

app.use((error, req, res, next) => {
  res.status(500).json({msg: 'Please contact your web administrator'});
});

app.listen(port, () => console.log(`Listening on port ${port}`));

const express = require('express');
const path = require('path');
const app = express();
const port = 3000;

app.use(express.static('./public'));

app.get('/', (req, res, next) => {
  res.sendFile(path.join(__dirname, '/public/index.html'));
});

app.use((error, req, res, next) => {
  res.status(500).json({msg: 'Please contact your web administrator'});
});

app.listen(port, () => console.log(`Listening on port ${port}`));

and it's serving ADMIN.HTML 😆 😭

Joao•2y ago

Do you have a git repo to take a look at?

JWodeOP•2y ago

yeah, but the express file above isn't part of it I can link it anyway if it helps? but in my build folder if I click click on index.html and choose 'live-server' locally it serves the right file so it feels like an issue with express/the server rather than my build files?

Joao•2y ago

That's what I want to understand, why are you using webpack at all?

JWodeOP•2y ago

I'm not for the express side of things any more. I'm manually copying them into the public folder

Joao•2y ago

Try this:

app.use(express.static(path.join(__dirname, 'public'));

app.get('/', (req, res, next) => {
  res.sendFile(path.join(__dirname, 'public', 'index.html'));
});

app.use(express.static(path.join(__dirname, 'public'));

app.get('/', (req, res, next) => {
  res.sendFile(path.join(__dirname, 'public', 'index.html'));
});

JWodeOP•2y ago

still the admin.html XD

Joao•2y ago

Ok, let's take a look at the repo 😄

JWodeOP•2y ago

I literally have to laugh at this point 😁 1 second thanks btw

JWodeOP•2y ago

https://github.com/NickWoodward/sub1

GitHub

GitHub - NickWoodward/sub1

Contribute to NickWoodward/sub1 development by creating an account on GitHub.

JWodeOP•2y ago

there is a rogue express-app.js file in there, but it's not om the server

JWodeOP•2y ago

^ the site on my server. basically the repo + an express-app folder

JWodeOP•2y ago

// public:

Joao•2y ago

So the public folder is /var/www/sub1/express-app/public? And the node.js server is the file at /var/www/sub1/express-app/app.js?

JWodeOP•2y ago

yup

JWodeOP•2y ago

/var/www/sub1/express-app/public:

JWodeOP•2y ago

I mean, I'm very close to switching the names of the files round 😆

Joao•2y ago

You mentioned you are using pm2 right?

JWodeOP•2y ago

yeah

Joao•2y ago

Try shutdown and restart. It's probably some caching issue somewhere

JWodeOP•2y ago

i could just shut down the whole server, just to make sure ?

Joao•2y ago

Also try app.use(express.static(path.join(__dirname, 'public'))) why not, better be safe at this point 😄

JWodeOP•2y ago

no luck, still the admin page 😦 let me delete the pm2 app nope 🤷‍♂️ /etc/nginx/sites-available/default:

       index index.html index.htm index.nginx-debian.html;

        server_name sub1.io www.sub1.io;

        location / {
                proxy_pass http://127.0.0.1:3000/;
        }

       index index.html index.htm index.nginx-debian.html;

        server_name sub1.io www.sub1.io;

        location / {
                proxy_pass http://127.0.0.1:3000/;
        }

Joao•2y ago

Are you sure admin.html and index.html aren't the same file?

JWodeOP•2y ago

Pretty sure. I'm just copying across the file from the build folder, and locally it opens the main index page. Also it references index.js:

Joao•2y ago

Well, try to setup different routes then and see if they work:

app.get('/', (req, res, next) => {
  res.sendFile(path.join(__dirname, 'public', 'index.html'));
});

app.get('/admin', (req, res, next) => {
  res.sendFile(path.join(__dirname, 'public', 'admin.html'));
});

app.get('/', (req, res, next) => {
  res.sendFile(path.join(__dirname, 'public', 'index.html'));
});

app.get('/admin', (req, res, next) => {
  res.sendFile(path.join(__dirname, 'public', 'admin.html'));
});

Also try to run the server with node, instead of pm2, as a test node app.js

JWodeOP•2y ago

Not run with node yet, but / goes to admin, /admin is cannot get, /admin.html goes to admin let me try node

JWodeOP•2y ago

Just in case I've made a silly error:

Joao•2y ago

Btw, is there a reason why you are using express AND nginx? You can do the same with nginx only

JWodeOP•2y ago

I was following a guide, and thought i'd stick to what I knew. That's worked out well XD

Joao•2y ago

Make sure it's the right path, console.log the result of path.join(__dirname, public, admin.html)

JWodeOP•2y ago

(same problem with node app.js)

Joao•2y ago

And also try removing the / route now that you have one for admin.

JWodeOP•2y ago

sub1.io/admin:

JWodeOP•2y ago

wait...

JWodeOP•2y ago

JWodeOP•2y ago

sub1.io: no output tried with an index route too. Cannot GET /index (but it said that about /admin to start off with) restarted node. Both routes point to the correct path

JWodeOP•2y ago

If I remove admin.html express throws the error from my error handler on the /admin route... but serves the admin page on every other route ------------------- So I deleted the code in index.html and it started serving the right content, so it's an issue with how it's being bundled I guess? I've reworked the issue here, but I'm so tired of fighting this for 8 hours now it's probably terribly written and about to be closed 😁

JWodeOP•2y ago

https://stackoverflow.com/questions/75338674/webpack-building-the-wrong-file

Stack Overflow

Webpack building the wrong file?

So I've a bug I've been chasing all day, and I've narrowed it down to my webpack config (I think). I run my app locally using webpack server and index.html is displayed correctly (with index.js). I

Joao•2y ago

Hey, sorry I had to go earlier... I'm not sure what's going on, I'm suspecting it has to do with the static folder. Either webpack or express it's probably some silly typo somewhere that we missed Try setting up a route in nginx instead:

root /var/www/sub1/express-app/public;

location / {
  try_files $uri $uri/ index.html;
}

root /var/www/sub1/express-app/public;

location / {
  try_files $uri $uri/ index.html;
}

JWodeOP•2y ago

Oh no worries at all, I appreciate whatever help I can get!. I'm starting to think it isn't Express though. I now think it was serving the correct file (I deleted the contents of the file and replace it with new code it works) let me try it again quickly before we start looking at switching to nginx -------------- So I've just got index.html and index.js in my public folder now, and I've just changed the contents to a boilerplate html doc and it's displaying it. So it must be webpack, right? I just don't understand how webpack-server and live-server can both display the file correctly? got to sub1.io XD

Joao•2y ago

I think there was something hard-coded in there for an api endpoint

JWodeOP•2y ago

Joao•2y ago

But I don't think it's the problem

JWodeOP•2y ago

either way express/nginx is displaying index.html at least that's something (for 7 hours bug testing!)

Joao•2y ago

Well if you figure it out let me know I'm curious now 😄

JWodeOP•2y ago

Will do! It'll be one of those things where I'll have a good sleep and it'll take me 3 minutes in the morning 😁 (I hope)

Joao•2y ago

That's how it works 😂

JWodeOP•2y ago

I wish I had a proper answer for you @joao6246 . From what I can tell it was serving the right html file, but the wrong js file (which produces the content). That doesn't explain why not finding the index.js file (indicated by the MIME type error) would result in it defaulting to admin.js though 🤷‍♂️ . I still think that it was a bug in webpack. I read a couple of accounts of somesort of 'blank space' bug in the config that could produce errors, so maybe I removed part of the config that was causing an issue. Either way, after messing about in the config, chrome started complaining about the config file not being found at .src/blah/blah which obviously wasn't right and indicated that html-webpack-plugin was injecting the wrong script links, and it was. Still, I've no idea why that meant it was serving the wrong file. BUT IT'S WORKING! 🥳 ...and that's all that matters, right? 😆 (I also managed this fix while drunk 😁 )

Joao•2y ago

If it works for martial arts, no reason why it shouldn't work for programming right? 😂 I still think it wassome caching issue somewhere (it always is...)

JWodeOP•2y ago

"works"

Joao•2y ago

Anyway, good job getting things to work is the most important part

JWodeOP•2y ago

yeah, it's still now complaining about that MIME type every other time I run locally now. Refreshing it fixes it. TBH I'm ok with that as long as it's working! thanks again for your help bud, appreciate it

Joao•2y ago

Try adding this to the express server if you are still using it: res.setHeader('X-Content-Type-Options', 'nosniff');

JWodeOP•2y ago

I'll give that a try, thanks 🙂

Gaming

Programming

More file structure, this time webpack flavoured!