Wasp-lang•4w ago

Serve /public files in prod (build)

Hello, I have a weird scenario in which I download a file on my API, save it in /public so the frontend can access it, and then I spin remotion to render the video in the background. Issue is I keep getting errors saying Error: ENOENT: no such file or directory, open '/app/public/0b985bfc-3d43-49d6-8d4c-7cc4bff1e24e.mp4' Some of the code used for this:

const videoPath = path.join(process.cwd(), '..','..','..', 'public', `${filename}.mp4`);

       const x = await downloadFromPresignedUrl(s3, videoPath)
           .then(() => console.log('Download completed'))
           .catch(err => console.error('Download failed', err));
       const compositionId = 'CaptionedVideo';
       const compPath = path.join(process.cwd(),'..','..','..', "src/demo-ai-app/index.ts")

       const bundleLocation = await bundle({
           // entryPoint: path.resolve('src/demo-ai-app/index.ts'),
           entryPoint: compPath,
           webpackOverride: (config) => config,
       });
       const allWords = output.segments.flatMap((segment: { words: any; }) => segment.words);
       const inputProps = {
           videoSrc: videoPath,
           src: `/public/${filename}.mp4`,
           s3: s3,
           words: allWords
       };
       const composition = await selectComposition({
           serveUrl: bundleLocation,
           id: compositionId,
           inputProps,
       });
       const captioned = path.join(process.cwd(), '..','..','..', 'public',`${filename}_captioned.mp4`);
       await renderMedia({
           composition,
           serveUrl: bundleLocation,
           codec: 'h264',
           outputLocation: `${captioned}`,
           inputProps,
       });

const videoPath = path.join(process.cwd(), '..','..','..', 'public', `${filename}.mp4`);

       const x = await downloadFromPresignedUrl(s3, videoPath)
           .then(() => console.log('Download completed'))
           .catch(err => console.error('Download failed', err));
       const compositionId = 'CaptionedVideo';
       const compPath = path.join(process.cwd(),'..','..','..', "src/demo-ai-app/index.ts")

       const bundleLocation = await bundle({
           // entryPoint: path.resolve('src/demo-ai-app/index.ts'),
           entryPoint: compPath,
           webpackOverride: (config) => config,
       });
       const allWords = output.segments.flatMap((segment: { words: any; }) => segment.words);
       const inputProps = {
           videoSrc: videoPath,
           src: `/public/${filename}.mp4`,
           s3: s3,
           words: allWords
       };
       const composition = await selectComposition({
           serveUrl: bundleLocation,
           id: compositionId,
           inputProps,
       });
       const captioned = path.join(process.cwd(), '..','..','..', 'public',`${filename}_captioned.mp4`);
       await renderMedia({
           composition,
           serveUrl: bundleLocation,
           codec: 'h264',
           outputLocation: `${captioned}`,
           inputProps,
       });

29 Replies

MEE6•4w ago

Wohooo @fustuk., you just became a Waspeteer level 3!

kapa.ai•4w ago

Hi @fustuk.. I'm kapa.ai, your friendly support bot. I will try to answer your question by searching through the knowledge base. :wave: ---------------------- Based on the information provided, it seems like the issue might be related to the way you are handling static assets in your Wasp project. In Wasp, static assets that are not referenced in the source code and must retain the exact same file name can be placed in the public directory at the root of your project. These assets will be served at the root path / during development and copied to the root of the dist directory as-is. However, assets in the public directory cannot be imported from the source code. You should always reference public assets using root absolute path. For example, public/icon.png should be referenced in source code as /icon.png source (https://wasp-lang.dev/docs/project/static-assets#the-public-directory). However, there is an open issue in Wasp's GitHub repository about the inability to serve and upload dynamic binary assets like images and PDFs. This might be related to the issue you are facing source (https://github.com/wasp-lang/wasp/issues/494). In your case, it seems like you are trying to access a file in the public directory that might not exist at the time of access. The error ENOENT: no such file or directory typically means that Node.js can't find the file you are trying to access. This could be due to a number of reasons such as the file not existing at the time of access, the file being deleted before access, or the file not being properly written to disk. Without more information, it's hard to provide a definitive solution. However, you might want to check the following: - Ensure that the file you are trying to access actually exists in the public directory at the time of access. - Check the file permissions of the public directory and the file you are trying to access. Make sure that the Node.js process has the necessary permissions to read the file. - If you are running your application in a Docker container or some other kind of isolated environment, make sure that the public directory is correctly mapped in your container. If none of these suggestions help, you might want to consider opening an issue in the Wasp GitHub repository to get help from the Wasp community. ---------------------- Note: you can ask me a follow up question by @ mentioning me again :speech_balloon: ---------------------- Sorry @fustuk., I seem to be having a bit of trouble. Don't worry though, I've pinged my support and they will try to fix me up! :technologist: Perhaps try again in 15mins?

fustuk.•4w ago

if so how should I test localy, currently /public/icon.png would work just fine

Filip•4w ago

Hey @fustuk., let's figure this out. My first question is: have you checked whether the file is there? In other words, is Wasp wrong that there's no such file, or is there really no such file.

fustuk.•4w ago

I am trying to download it at that destination, so I am not sure how to check if its really there after download. But yes either the path when I am downloading it is wrong or the path that I try to load it in remotion is And also not sure if this words properly @Filip

const compPath = path.join(process.cwd(),'..','..','..', "src/demo-ai-app/index.ts")

const compPath = path.join(process.cwd(),'..','..','..', "src/demo-ai-app/index.ts")

Reviewing the docs again I see that its probably on download since it says WriteStream

Error: ENOENT: no such file or directory, open '/app/public/1505001d-8e22-47b4-a0f8-8e1c0b6f2f6b.mp4'

Emitted 'error' event on WriteStream instance at:

Error: ENOENT: no such file or directory, open '/app/public/1505001d-8e22-47b4-a0f8-8e1c0b6f2f6b.mp4'

Emitted 'error' event on WriteStream instance at:

main issue becomes that this is hosted on railways in 2 different services (server/client) how can I connect the file system, is there a way or do I have to upload to s3 and re-download it on the client side :/

Filip•4w ago

main issue becomes that this is hosted on railways in 2 different services (server/client)

Yes, that's the correct diagnosis. The public dir you wish to save it to is in the client service (and doesn't really exist after the app is built). So, you have to save your file somewhere on the server service, and then serve it to the use on an HTTP request. Wasp currently provides no first-class support for this, so you'll have to implement it yourself. But there's no need to go to s3, you can use the local file system, especially if it's temporary. If the files have to be permanent, then you're better off using s3 anyway. Anyway, if the files are temporary, here's what I'd do: - Pick a directory where to save the files (e.g., /app/videos) and create it - Write the file to that directory (as you're currently doing). - Make a custom API endpoint that servers these file to the client on request (check the docs). In the function, read the file from /app/videos and send it back to the client. If the files are permanent, I'd either ensure my server file system is persisted, or use s3. @kapa.ai Can you give a suggestion on how to implement this custom API in Wasp to give @fustuk. a starting point?

kapa.ai•4w ago

Let me check the knowledge base for you. One moment! :books: ---------------------- Sure, I can provide a basic example of how you might implement a custom API in Wasp to serve files from a directory. First, you would declare the API in your Wasp code. This could look something like this:

// main.wasp
api downloadFile {
  fn: import { downloadFile } from "@src/apis",
  httpRoute: (GET, "/downloads/:filename")
}

// main.wasp
api downloadFile {
  fn: import { downloadFile } from "@src/apis",
  httpRoute: (GET, "/downloads/:filename")
}

In this example, :filename is a placeholder for the name of the file you want to download. This will be part of the URL when you make a request to this endpoint. Next, you would define the implementation of the API in Node.js. This function will be responsible for reading the file from the filesystem and sending it as a response. Here's a basic example:

// src/apis.js
import fs from 'fs'
import path from 'path'

export const downloadFile = (req, res, context) => {
  const filename = req.params.filename
  const filepath = path.join('/app/videos', filename)

  fs.readFile(filepath, (err, data) => {
    if (err) {
      res.status(500).send('Error reading file')
    } else {
      res.setHeader('Content-Type', 'video/mp4')
      res.send(data)
    }
  })
}

// src/apis.js
import fs from 'fs'
import path from 'path'

export const downloadFile = (req, res, context) => {
  const filename = req.params.filename
  const filepath = path.join('/app/videos', filename)

  fs.readFile(filepath, (err, data) => {
    if (err) {
      res.status(500).send('Error reading file')
    } else {
      res.setHeader('Content-Type', 'video/mp4')
      res.send(data)
    }
  })
}

In this example, we're using the fs.readFile function to read the file from the filesystem. If there's an error (for example, if the file doesn't exist), we send a 500 response. Otherwise, we set the Content-Type header to video/mp4 (you might need to adjust this depending on the type of files you're serving), and send the file data as the response. Finally, you would call this API from your client code using the Axios wrapper provided by Wasp:

// src/pages/SomePage.jsx
import React, { useEffect } from "react";  
import { api } from "wasp/client/api";  

async function fetchCustomRoute() {  
  const res = await api.get("/downloads/test.mp4");  
  console.log(res.data);  
}
export const Foo = () => {  
  useEffect(() => {  
    fetchCustomRoute();  
  }, []);  

  return <>// ...</>;  
};

// src/pages/SomePage.jsx
import React, { useEffect } from "react";  
import { api } from "wasp/client/api";  

async function fetchCustomRoute() {  
  const res = await api.get("/downloads/test.mp4");  
  console.log(res.data);  
}
export const Foo = () => {  
  useEffect(() => {  
    fetchCustomRoute();  
  }, []);  

  return <>// ...</>;  
};

In this example, we're making a GET request to the /downloads/test.mp4 endpoint, which will trigger the downloadFile API we defined earlier, and log the response data. Please note that this is a basic example and might need to be adjusted based on your specific use case. For example, you might need to handle different types of files, add error handling, or adjust the file paths. You can find more information about creating custom APIs in Wasp in the official documentation (https://wasp-lang.dev/docs/advanced/apis). Also, remember to enable CORS if you are making these requests from a different origin. You can do this by defining custom middleware for your APIs in the Wasp file. More information about this can be found in the official documentation (https://wasp-lang.dev/docs/advanced/apis#making-sure-cors-works). ---------------------- Note: you can ask me a follow up question by @ mentioning me again :speech_balloon: ---------------------- Sorry @Filip, I seem to be having a bit of trouble. Don't worry though, I've pinged my support and they will try to fix me up! :technologist: Perhaps try again in 15mins?

Filip•4w ago

Here you go @fustuk., this might not be 100% correct, but will hopefully give you the right idea

fustuk.•4w ago

Thank you, I forgot that there are 2 services there my badd

Filip•4w ago

Still, if the files are permanent, s3 is the way to go (even if Wasp had a first-class support for doing this) No worries!

fustuk.•4w ago

Hey so I have another concern, even If I make it work like you sugested, how can I invoke my client, or when I build it is it even at the client side?heres the folder structure, I am invoking the index.ts in /demo-ai-app, when we build where does this even go @Filip

Filip•4w ago

Hm, not sure I understand the qustion. What do you mean by invoking the client?

fustuk.•4w ago

const compositionId = 'CaptionedVideo';
       const compPath = path.join(process.cwd(),'..','..','..', "src/demo-ai-app/index.ts")

       const bundleLocation = await bundle({
           // entryPoint: path.resolve('src/demo-ai-app/index.ts'),
           entryPoint: compPath,
           webpackOverride: (config) => config,
       });
       const allWords = output.segments.flatMap((segment: { words: any; }) => segment.words);
       const inputProps = {
           videoSrc: videoPath,
           // src: `/public/${filename}.mp4`,
           src: s3,
           s3: s3,
           words: allWords
       };
       const composition = await selectComposition({
           serveUrl: bundleLocation,
           id: compositionId,
           inputProps,
       });
       const captioned = path.join(process.cwd(), '..','..','..', 'public',`${filename}_captioned.mp4`);
       await renderMedia({
           composition,
           serveUrl: bundleLocation,
           codec: 'h264',
           outputLocation: `${captioned}`,
           inputProps,
       });

const compositionId = 'CaptionedVideo';
       const compPath = path.join(process.cwd(),'..','..','..', "src/demo-ai-app/index.ts")

       const bundleLocation = await bundle({
           // entryPoint: path.resolve('src/demo-ai-app/index.ts'),
           entryPoint: compPath,
           webpackOverride: (config) => config,
       });
       const allWords = output.segments.flatMap((segment: { words: any; }) => segment.words);
       const inputProps = {
           videoSrc: videoPath,
           // src: `/public/${filename}.mp4`,
           src: s3,
           s3: s3,
           words: allWords
       };
       const composition = await selectComposition({
           serveUrl: bundleLocation,
           id: compositionId,
           inputProps,
       });
       const captioned = path.join(process.cwd(), '..','..','..', 'public',`${filename}_captioned.mp4`);
       await renderMedia({
           composition,
           serveUrl: bundleLocation,
           codec: 'h264',
           outputLocation: `${captioned}`,
           inputProps,
       });

See my paths here compPath and compositionId are those folders/files I just mentioned getting this when I try to render using this file path

[Error: ENOENT: no such file or directory, open '/app/src/demo-ai-app/index.ts'] {
  errno: -2,
  code: 'ENOENT',
  syscall: 'open',
  path: '/app/src/demo-ai-app/index.ts'
}

[Error: ENOENT: no such file or directory, open '/app/src/demo-ai-app/index.ts'] {
  errno: -2,
  code: 'ENOENT',
  syscall: 'open',
  path: '/app/src/demo-ai-app/index.ts'
}

miho•4w ago

You should consider your server and the client as two separate worlds. Uploading things to the public dir of the client is a no-no since they are two separate worlds. Ideally, you'd prepare the file somewhere the server can access it (some path you decide on) and then make it available for download with the Express static middleware. Ideally, you'd upload the file to S3 as @Filip mentioned not to lose the file after the server restarts / redeploys etc. (or if you have permanent storage, then it's fine to keep it on the server disk - but you need to be sure about this). I see that you are trying to use the Remotion lib, that uses React to produce videos, right? So you are trying to load React components on the server?

fustuk.•4w ago

I can see those files here tho, those should be deployed to the server correct? /src But I still get not found err [Error: ENOENT: no such file or directory, open '/app/.wasp/build/src/demo-ai-app/index.ts']

Gaming

Programming

Serve /public files in prod (build)