Wasp-lang•3mo ago

Serve /public files in prod (build)

Hello, I have a weird scenario in which I download a file on my API, save it in /public so the frontend can access it, and then I spin remotion to render the video in the background. Issue is I keep getting errors saying Error: ENOENT: no such file or directory, open '/app/public/0b985bfc-3d43-49d6-8d4c-7cc4bff1e24e.mp4' Some of the code used for this:

const videoPath = path.join(process.cwd(), '..','..','..', 'public', `${filename}.mp4`);

       const x = await downloadFromPresignedUrl(s3, videoPath)
           .then(() => console.log('Download completed'))
           .catch(err => console.error('Download failed', err));
       const compositionId = 'CaptionedVideo';
       const compPath = path.join(process.cwd(),'..','..','..', "src/demo-ai-app/index.ts")

       const bundleLocation = await bundle({
           // entryPoint: path.resolve('src/demo-ai-app/index.ts'),
           entryPoint: compPath,
           webpackOverride: (config) => config,
       });
       const allWords = output.segments.flatMap((segment: { words: any; }) => segment.words);
       const inputProps = {
           videoSrc: videoPath,
           src: `/public/${filename}.mp4`,
           s3: s3,
           words: allWords
       };
       const composition = await selectComposition({
           serveUrl: bundleLocation,
           id: compositionId,
           inputProps,
       });
       const captioned = path.join(process.cwd(), '..','..','..', 'public',`${filename}_captioned.mp4`);
       await renderMedia({
           composition,
           serveUrl: bundleLocation,
           codec: 'h264',
           outputLocation: `${captioned}`,
           inputProps,
       });

const videoPath = path.join(process.cwd(), '..','..','..', 'public', `${filename}.mp4`);

       const x = await downloadFromPresignedUrl(s3, videoPath)
           .then(() => console.log('Download completed'))
           .catch(err => console.error('Download failed', err));
       const compositionId = 'CaptionedVideo';
       const compPath = path.join(process.cwd(),'..','..','..', "src/demo-ai-app/index.ts")

       const bundleLocation = await bundle({
           // entryPoint: path.resolve('src/demo-ai-app/index.ts'),
           entryPoint: compPath,
           webpackOverride: (config) => config,
       });
       const allWords = output.segments.flatMap((segment: { words: any; }) => segment.words);
       const inputProps = {
           videoSrc: videoPath,
           src: `/public/${filename}.mp4`,
           s3: s3,
           words: allWords
       };
       const composition = await selectComposition({
           serveUrl: bundleLocation,
           id: compositionId,
           inputProps,
       });
       const captioned = path.join(process.cwd(), '..','..','..', 'public',`${filename}_captioned.mp4`);
       await renderMedia({
           composition,
           serveUrl: bundleLocation,
           codec: 'h264',
           outputLocation: `${captioned}`,
           inputProps,
       });

29 Replies

MEE6•3mo ago

Wohooo @fustuk., you just became a Waspeteer level 3!

kapa.ai•3mo ago

Hi @fustuk.. I'm kapa.ai, your friendly support bot. I will try to answer your question by searching through the knowledge base. :wave: ---------------------- Based on the information provided, it seems like the issue might be related to the way you are handling static assets in your Wasp project. In Wasp, static assets that are not referenced in the source code and must retain the exact same file name can be placed in the public directory at the root of your project. These assets will be served at the root path / during development and copied to the root of the dist directory as-is. However, assets in the public directory cannot be imported from the source code. You should always reference public assets using root absolute path. For example, public/icon.png should be referenced in source code as /icon.png source (https://wasp-lang.dev/docs/project/static-assets#the-public-directory). However, there is an open issue in Wasp's GitHub repository about the inability to serve and upload dynamic binary assets like images and PDFs. This might be related to the issue you are facing source (https://github.com/wasp-lang/wasp/issues/494). In your case, it seems like you are trying to access a file in the public directory that might not exist at the time of access. The error ENOENT: no such file or directory typically means that Node.js can't find the file you are trying to access. This could be due to a number of reasons such as the file not existing at the time of access, the file being deleted before access, or the file not being properly written to disk. Without more information, it's hard to provide a definitive solution. However, you might want to check the following: - Ensure that the file you are trying to access actually exists in the public directory at the time of access. - Check the file permissions of the public directory and the file you are trying to access. Make sure that the Node.js process has the necessary permissions to read the file. - If you are running your application in a Docker container or some other kind of isolated environment, make sure that the public directory is correctly mapped in your container. If none of these suggestions help, you might want to consider opening an issue in the Wasp GitHub repository to get help from the Wasp community. ---------------------- Note: you can ask me a follow up question by @ mentioning me again :speech_balloon: ---------------------- Sorry @fustuk., I seem to be having a bit of trouble. Don't worry though, I've pinged my support and they will try to fix me up! :technologist: Perhaps try again in 15mins?

fustuk.OP•3mo ago

if so how should I test localy, currently /public/icon.png would work just fine

Filip•3mo ago

Hey @fustuk., let's figure this out. My first question is: have you checked whether the file is there? In other words, is Wasp wrong that there's no such file, or is there really no such file.

fustuk.OP•3mo ago

I am trying to download it at that destination, so I am not sure how to check if its really there after download. But yes either the path when I am downloading it is wrong or the path that I try to load it in remotion is And also not sure if this words properly @Filip

const compPath = path.join(process.cwd(),'..','..','..', "src/demo-ai-app/index.ts")

const compPath = path.join(process.cwd(),'..','..','..', "src/demo-ai-app/index.ts")

Reviewing the docs again I see that its probably on download since it says WriteStream

Error: ENOENT: no such file or directory, open '/app/public/1505001d-8e22-47b4-a0f8-8e1c0b6f2f6b.mp4'

Emitted 'error' event on WriteStream instance at:

Error: ENOENT: no such file or directory, open '/app/public/1505001d-8e22-47b4-a0f8-8e1c0b6f2f6b.mp4'

Emitted 'error' event on WriteStream instance at:

main issue becomes that this is hosted on railways in 2 different services (server/client) how can I connect the file system, is there a way or do I have to upload to s3 and re-download it on the client side :/

Filip•3mo ago

main issue becomes that this is hosted on railways in 2 different services (server/client)

Yes, that's the correct diagnosis. The public dir you wish to save it to is in the client service (and doesn't really exist after the app is built). So, you have to save your file somewhere on the server service, and then serve it to the use on an HTTP request. Wasp currently provides no first-class support for this, so you'll have to implement it yourself. But there's no need to go to s3, you can use the local file system, especially if it's temporary. If the files have to be permanent, then you're better off using s3 anyway. Anyway, if the files are temporary, here's what I'd do: - Pick a directory where to save the files (e.g., /app/videos) and create it - Write the file to that directory (as you're currently doing). - Make a custom API endpoint that servers these file to the client on request (check the docs). In the function, read the file from /app/videos and send it back to the client. If the files are permanent, I'd either ensure my server file system is persisted, or use s3. @kapa.ai Can you give a suggestion on how to implement this custom API in Wasp to give @fustuk. a starting point?

kapa.ai•3mo ago

Let me check the knowledge base for you. One moment! :books: ---------------------- Sure, I can provide a basic example of how you might implement a custom API in Wasp to serve files from a directory. First, you would declare the API in your Wasp code. This could look something like this:

// main.wasp
api downloadFile {
  fn: import { downloadFile } from "@src/apis",
  httpRoute: (GET, "/downloads/:filename")
}

// main.wasp
api downloadFile {
  fn: import { downloadFile } from "@src/apis",
  httpRoute: (GET, "/downloads/:filename")
}

In this example, :filename is a placeholder for the name of the file you want to download. This will be part of the URL when you make a request to this endpoint. Next, you would define the implementation of the API in Node.js. This function will be responsible for reading the file from the filesystem and sending it as a response. Here's a basic example:

// src/apis.js
import fs from 'fs'
import path from 'path'

export const downloadFile = (req, res, context) => {
  const filename = req.params.filename
  const filepath = path.join('/app/videos', filename)

  fs.readFile(filepath, (err, data) => {
    if (err) {
      res.status(500).send('Error reading file')
    } else {
      res.setHeader('Content-Type', 'video/mp4')
      res.send(data)
    }
  })
}

// src/apis.js
import fs from 'fs'
import path from 'path'

export const downloadFile = (req, res, context) => {
  const filename = req.params.filename
  const filepath = path.join('/app/videos', filename)

  fs.readFile(filepath, (err, data) => {
    if (err) {
      res.status(500).send('Error reading file')
    } else {
      res.setHeader('Content-Type', 'video/mp4')
      res.send(data)
    }
  })
}

In this example, we're using the fs.readFile function to read the file from the filesystem. If there's an error (for example, if the file doesn't exist), we send a 500 response. Otherwise, we set the Content-Type header to video/mp4 (you might need to adjust this depending on the type of files you're serving), and send the file data as the response. Finally, you would call this API from your client code using the Axios wrapper provided by Wasp:

// src/pages/SomePage.jsx
import React, { useEffect } from "react";  
import { api } from "wasp/client/api";  

async function fetchCustomRoute() {  
  const res = await api.get("/downloads/test.mp4");  
  console.log(res.data);  
}
export const Foo = () => {  
  useEffect(() => {  
    fetchCustomRoute();  
  }, []);  

  return <>// ...</>;  
};

// src/pages/SomePage.jsx
import React, { useEffect } from "react";  
import { api } from "wasp/client/api";  

async function fetchCustomRoute() {  
  const res = await api.get("/downloads/test.mp4");  
  console.log(res.data);  
}
export const Foo = () => {  
  useEffect(() => {  
    fetchCustomRoute();  
  }, []);  

  return <>// ...</>;  
};

In this example, we're making a GET request to the /downloads/test.mp4 endpoint, which will trigger the downloadFile API we defined earlier, and log the response data. Please note that this is a basic example and might need to be adjusted based on your specific use case. For example, you might need to handle different types of files, add error handling, or adjust the file paths. You can find more information about creating custom APIs in Wasp in the official documentation (https://wasp-lang.dev/docs/advanced/apis). Also, remember to enable CORS if you are making these requests from a different origin. You can do this by defining custom middleware for your APIs in the Wasp file. More information about this can be found in the official documentation (https://wasp-lang.dev/docs/advanced/apis#making-sure-cors-works). ---------------------- Note: you can ask me a follow up question by @ mentioning me again :speech_balloon: ---------------------- Sorry @Filip, I seem to be having a bit of trouble. Don't worry though, I've pinged my support and they will try to fix me up! :technologist: Perhaps try again in 15mins?

Filip•3mo ago

Here you go @fustuk., this might not be 100% correct, but will hopefully give you the right idea

fustuk.OP•3mo ago

Thank you, I forgot that there are 2 services there my badd

Filip•3mo ago

Still, if the files are permanent, s3 is the way to go (even if Wasp had a first-class support for doing this) No worries!

fustuk.OP•3mo ago

Hey so I have another concern, even If I make it work like you sugested, how can I invoke my client, or when I build it is it even at the client side?heres the folder structure, I am invoking the index.ts in /demo-ai-app, when we build where does this even go @Filip

Filip•3mo ago

Hm, not sure I understand the qustion. What do you mean by invoking the client?

fustuk.OP•3mo ago

const compositionId = 'CaptionedVideo';
       const compPath = path.join(process.cwd(),'..','..','..', "src/demo-ai-app/index.ts")

       const bundleLocation = await bundle({
           // entryPoint: path.resolve('src/demo-ai-app/index.ts'),
           entryPoint: compPath,
           webpackOverride: (config) => config,
       });
       const allWords = output.segments.flatMap((segment: { words: any; }) => segment.words);
       const inputProps = {
           videoSrc: videoPath,
           // src: `/public/${filename}.mp4`,
           src: s3,
           s3: s3,
           words: allWords
       };
       const composition = await selectComposition({
           serveUrl: bundleLocation,
           id: compositionId,
           inputProps,
       });
       const captioned = path.join(process.cwd(), '..','..','..', 'public',`${filename}_captioned.mp4`);
       await renderMedia({
           composition,
           serveUrl: bundleLocation,
           codec: 'h264',
           outputLocation: `${captioned}`,
           inputProps,
       });

const compositionId = 'CaptionedVideo';
       const compPath = path.join(process.cwd(),'..','..','..', "src/demo-ai-app/index.ts")

       const bundleLocation = await bundle({
           // entryPoint: path.resolve('src/demo-ai-app/index.ts'),
           entryPoint: compPath,
           webpackOverride: (config) => config,
       });
       const allWords = output.segments.flatMap((segment: { words: any; }) => segment.words);
       const inputProps = {
           videoSrc: videoPath,
           // src: `/public/${filename}.mp4`,
           src: s3,
           s3: s3,
           words: allWords
       };
       const composition = await selectComposition({
           serveUrl: bundleLocation,
           id: compositionId,
           inputProps,
       });
       const captioned = path.join(process.cwd(), '..','..','..', 'public',`${filename}_captioned.mp4`);
       await renderMedia({
           composition,
           serveUrl: bundleLocation,
           codec: 'h264',
           outputLocation: `${captioned}`,
           inputProps,
       });

See my paths here compPath and compositionId are those folders/files I just mentioned getting this when I try to render using this file path

[Error: ENOENT: no such file or directory, open '/app/src/demo-ai-app/index.ts'] {
  errno: -2,
  code: 'ENOENT',
  syscall: 'open',
  path: '/app/src/demo-ai-app/index.ts'
}

[Error: ENOENT: no such file or directory, open '/app/src/demo-ai-app/index.ts'] {
  errno: -2,
  code: 'ENOENT',
  syscall: 'open',
  path: '/app/src/demo-ai-app/index.ts'
}

miho•3mo ago

You should consider your server and the client as two separate worlds. Uploading things to the public dir of the client is a no-no since they are two separate worlds. Ideally, you'd prepare the file somewhere the server can access it (some path you decide on) and then make it available for download with the Express static middleware. Ideally, you'd upload the file to S3 as @Filip mentioned not to lose the file after the server restarts / redeploys etc. (or if you have permanent storage, then it's fine to keep it on the server disk - but you need to be sure about this). I see that you are trying to use the Remotion lib, that uses React to produce videos, right? So you are trying to load React components on the server?

fustuk.OP•3mo ago

I can see those files here tho, those should be deployed to the server correct? /src But I still get not found err [Error: ENOENT: no such file or directory, open '/app/.wasp/build/src/demo-ai-app/index.ts']

fustuk.OP•3mo ago

yes my end goal is to render using remotion from the server/background somehow I am not really sure how its usually suppose to be used for stuff like that.It just adds some animated captions to videos, I can display the video usint there Player component but it cannot be rendered and thus downloaded by the user, this is the bigger picture of the issue

miho•3mo ago

Could you share some minimal example of using Remotion with Wasp so I can try doing it. I guess the package.json file and some client and server code that you feel comfortable sharing. We can talk back and forth and maybe we arrive at a solution, but it'll be easier if I try doing it and then I can share how I did it so you can have a working example.

fustuk.OP•3mo ago

remotion code is modified version of this https://github.com/remotion-dev/template-tiktok this is the endpoint I have which purpose is to just render this (add captions to the video) I was firstly looking for a way to just display it on the frontend, while it works (plays captioned video) it is not rendered thus cant be downloaded which is why Im currently trying to do it this way

GitHub

GitHub - remotion-dev/template-tiktok: Generate TikTok-style captio...

Generate TikTok-style captions with Whisper.cpp. Contribute to remotion-dev/template-tiktok development by creating an account on GitHub.

MEE6•3mo ago

Wohooo @fustuk., you just became a Waspeteer level 4!

fustuk.OP•3mo ago

const output: any = await replicate.run(
           "thomasmol/whisper-diarization:aae6db69a923a6eab6bc3ec098148a8c9c999685be89f428a4a6072fca544d26",
           {
               input: {
                   file: s3,
                   language: "en",
                   translate: false,
                   num_speakers: 2,
                   group_segments: false,
                   offset_seconds: 0,
                   transcript_output_format: "words_only"
               }
           }
       );



       // const x = await downloadFromPresignedUrl(s3, videoPath)
       //     .then(() => console.log('Download completed'))
       //     .catch(err => console.error('Download failed', err));
       const compositionId = 'CaptionedVideo';
       const compPath = path.join(process.cwd(),'..', "src/demo-ai-app/index.ts")
       // const compPath = path.join(process.cwd(),'..','..','..', "src/demo-ai-app/index.ts")
       console.log(process.cwd())
       console.log(compPath)

const output: any = await replicate.run(
           "thomasmol/whisper-diarization:aae6db69a923a6eab6bc3ec098148a8c9c999685be89f428a4a6072fca544d26",
           {
               input: {
                   file: s3,
                   language: "en",
                   translate: false,
                   num_speakers: 2,
                   group_segments: false,
                   offset_seconds: 0,
                   transcript_output_format: "words_only"
               }
           }
       );



       // const x = await downloadFromPresignedUrl(s3, videoPath)
       //     .then(() => console.log('Download completed'))
       //     .catch(err => console.error('Download failed', err));
       const compositionId = 'CaptionedVideo';
       const compPath = path.join(process.cwd(),'..', "src/demo-ai-app/index.ts")
       // const compPath = path.join(process.cwd(),'..','..','..', "src/demo-ai-app/index.ts")
       console.log(process.cwd())
       console.log(compPath)

const bundleLocation = await bundle({
           // entryPoint: path.resolve('src/demo-ai-app/index.ts'),
           entryPoint: compPath,
           webpackOverride: (config) => config,
       });
       const allWords = output.segments.flatMap((segment: { words: any; }) => segment.words);
       const inputProps = {
           videoSrc: videoPath,
           // src: `/public/${filename}.mp4`,
           src: s3,
           s3: s3,
           words: allWords
       };
       const composition = await selectComposition({
           serveUrl: bundleLocation,
           id: compositionId,
           inputProps,
       });
       const captioned = path.join(process.cwd(), '..','..','..', 'public',`${filename}_captioned.mp4`);
       await renderMedia({
           composition,
           serveUrl: bundleLocation,
           codec: 'h264',
           outputLocation: `${captioned}`,
           inputProps,
       });

const bundleLocation = await bundle({
           // entryPoint: path.resolve('src/demo-ai-app/index.ts'),
           entryPoint: compPath,
           webpackOverride: (config) => config,
       });
       const allWords = output.segments.flatMap((segment: { words: any; }) => segment.words);
       const inputProps = {
           videoSrc: videoPath,
           // src: `/public/${filename}.mp4`,
           src: s3,
           s3: s3,
           words: allWords
       };
       const composition = await selectComposition({
           serveUrl: bundleLocation,
           id: compositionId,
           inputProps,
       });
       const captioned = path.join(process.cwd(), '..','..','..', 'public',`${filename}_captioned.mp4`);
       await renderMedia({
           composition,
           serveUrl: bundleLocation,
           codec: 'h264',
           outputLocation: `${captioned}`,
           inputProps,
       });

this is playing a video on the client side

import React, {useState, useEffect} from 'react';
import {Player} from '@remotion/player';
import {
    CaptionedVideo,
} from './CaptionedVideo/index';

const ClipPlayer = ({clip}) => {
    const [durationInFrames, setDurationInFrames] = useState(2000); // Default value

    return (
        <div>asd</div>

        // <Player
        //     component={CaptionedVideo}
        //     durationInFrames={durationInFrames}
        //     fps={100}
        //     compositionWidth={303}
        //     compositionHeight={540}
        //     controls={true}
        //     inputProps={{
        //         src: clip.url
        //     }}
        // />
    );
};

import React, {useState, useEffect} from 'react';
import {Player} from '@remotion/player';
import {
    CaptionedVideo,
} from './CaptionedVideo/index';

const ClipPlayer = ({clip}) => {
    const [durationInFrames, setDurationInFrames] = useState(2000); // Default value

    return (
        <div>asd</div>

        // <Player
        //     component={CaptionedVideo}
        //     durationInFrames={durationInFrames}
        //     fps={100}
        //     compositionWidth={303}
        //     compositionHeight={540}
        //     controls={true}
        //     inputProps={{
        //         src: clip.url
        //     }}
        // />
    );
};

miho•3mo ago

Ok noice! So you are building a tool for automatically captioning videos with Whisper 🙂 Is this using Remotion servers to render the video?

fustuk.OP•3mo ago

Remotion servers ?

miho•3mo ago

I'm not that familiar with Remotion, I thought you are maybe using some sort of Remotion API to render the videos. But now I see it's done locally. I'll play around with this later today / tomorrow and report back.

fustuk.OP•3mo ago

alright thanks alot , will also try to resolve it oh are we only deploying build/server ? if so thats where things go wrong since im looking for /build/src... im using railway up inside .wasp/build , question is can I deploy .wasp/build/src as well somehow @miho how does the Dockerfile inside .wasp/build gets generated and can I change it somehow this should resolve my problem

miho•3mo ago

https://wasp-lang.dev/docs/advanced/deployment/overview#customizing-the-dockerfile

Overview | Wasp

Wasp apps are full-stack apps that consist of:

fustuk.OP•3mo ago

thanks love ya oh well, I fixed it, but now I get error due to browser support and when I read remotion's docs I see this: Alpine Linux Not supported due to to unsupported Libc symbols.

miho•3mo ago

The base image should be then something else 🙂 instead of alpine, put maybe slim or smth else

fustuk.OP•3mo ago

made it work with node:18.18.0-bookworm-slim, thank you

miho•3mo ago

Noice! I'm glad it worked. You can also use node 20 (I think it has some performance benefits)

Gaming

Programming

Serve /public files in prod (build)