Wasp•2mo ago

Streaming Text From OpenAI : Best Approach?

Hi there! As the title mentions, I am looking for ways to implement a text streaming chatbot in Wasp. However, the only example I could find on streaming text was done through API, and the user seems to have a bit of trouble tying it to a button on the frontend. Not sure if I can do it with Async Generator / Event Emitter / Websocket / Other Methods. I can probably work it out with enough time. Still, I want to ask for some advice from people who are more familiar with wasp, as text streaming feels like a common feature. Which would be the easiest option? Is there some native support or tools that I'm missing?

9 Replies

kapa.ai•2mo ago

Hi @DavidFeng. I'm kapa.ai, your friendly support bot. I will try to answer your question by searching through the knowledge base. :wave: ---------------------- Based on the knowledge sources provided, I can offer some guidance on implementing text streaming for a chatbot in Wasp, particularly using WebSockets. While there isn't specific information about streaming text from OpenAI, Wasp does provide built-in support for WebSockets, which can be a good approach for real-time communication like text streaming. Here's what you can do: 1. Use WebSockets: Wasp has integrated WebSocket support using Socket.IO. This can be an effective method for streaming text in real-time. 2. Set up WebSockets in your Wasp file: You'll need to enable WebSockets in your Wasp file by adding the webSocket configuration to your app. Docs > Advanced > Web Sockets provides an example:

   app todoApp {
     // ...
     webSocket: {
       fn: import { webSocketFn } from "@src/webSocket",
       autoConnect: true, // optional, default: true
     },
   }

   app todoApp {
     // ...
     webSocket: {
       fn: import { webSocketFn } from "@src/webSocket",
       autoConnect: true, // optional, default: true
     },
   }

3. Implement the server-side logic: Define your WebSocket events and handlers in the webSocketFn function. This is where you would implement the logic for streaming text from your AI model. 4. Use WebSockets on the client: Wasp provides useSocket and useSocketListener hooks for using WebSockets in React components. These can be used to send and receive streaming text data. For a detailed example of how to set up and use WebSockets in Wasp, you can refer to the Web Sockets documentation. While this approach doesn't specifically address OpenAI integration, it provides a foundation for implementing real-time text streaming in your Wasp app. You would need to integrate your OpenAI calls within this WebSocket setup. For OpenAI-specific integration, you might want to check out some open-source Wasp apps that use OpenAI's API, as mentioned in the Discord support thread. These examples, while not focused on streaming, could provide insights on how to work with OpenAI in a Wasp context: 1. https://OpenSaaS.sh 2. https://CoverLetterGPT.xyz Remember, implementing streaming with OpenAI might require additional setup beyond what's directly provided by Wasp, but the WebSocket functionality should give you a good starting point for real-time communication between your server and client. ---------------------- Note: you can ask me a follow up question by @ mentioning me again :speech_balloon: ---------------------- Sorry @DavidFeng, I seem to be having a bit of trouble. Don't worry though, I've pinged my support and they will try to fix me up! :technologist: Perhaps try again in 15mins?

matijash•2mo ago

tagging @Vinny (@Wasp) as he has first-hand experience with web sockets and open AI integrations in a Wasp app. @DavidFeng have you already looked into web sockets or some other way of doing it? Any thoughts or restrictions we should be aware of when proposing a solution?

DavidFengOP•2mo ago

I'm still doing research so nothing implemented so far. I am implementing something relatively simple - frontend press a button -> backend generates text from openAI -> stream it to a textbox on frontend I don't think there are any restrictions, just don't want to shoot myself on the foot and check out what's best supported

matijash•2mo ago

awesome, thanks for the contest! Vinny and Miho have the most knowledge here so I'll let them chime in :). Just keep in mind it's the end of day where we are (Europe), so it might have to wait until tomorrow.

DavidFengOP•2mo ago

OK, I found some support docs for websocket (seems pretty low latency too!) so I can just work on that for now Will send update on how it goes

matijash•2mo ago

awesome! here's also an example chat app using websockets, made with wasp: https://github.com/wasp-lang/wasp/tree/main/examples/websockets-realtime-voting

GitHub

wasp/examples/websockets-realtime-voting at main · wasp-lang/wasp

The fastest way to develop full-stack web apps with React & Node.js. - wasp-lang/wasp

Vinny (@Wasp)•2mo ago

So @DavidFeng as I understand it youre interested in streaming the text from the LLM to the user in real time, right? @miho Don't you have an example Wasp app for this?

miho•2mo ago

Here's the example app: https://github.com/wasp-lang/wasp/tree/main/examples/streaming

GitHub

wasp/examples/streaming at main · wasp-lang/wasp

The fastest way to develop full-stack web apps with React & Node.js. - wasp-lang/wasp

Vinny (@Wasp)•2mo ago

Thank you sir

Gaming

Programming

Streaming Text From OpenAI : Best Approach?

Did you find this page helpful?