How do I implement text streaming from the openai api
How do I implement text streaming from the openai api
12 Replies
Wohooo @haarismian, you just became a Waspeteer level 2!
Hi @haarismian. I'm kapa.ai, your friendly support bot. I will try to answer your question by searching through the knowledge base. :wave:
----------------------
I'm sorry, but the provided knowledge sources do not contain specific information on how to implement text streaming from the OpenAI API using the Wasp framework. However, you can refer to the OpenAI API documentation for more details on how to use their API for text streaming.
In the context of Wasp, you would typically interact with external APIs within your actions or operations. For example, in one of the Discord threads, a user was trying to use the OpenAI API within an action in their Wasp project. They imported the OpenAI module and attempted to create an assistant using
openai.beta.assistants.create()
. However, they encountered an issue because the 'beta' property did not exist on the imported 'openai' module. This suggests that you would need to correctly import and use the OpenAI module within your Wasp actions or operations.
Remember to add the OpenAI module to your dependencies in your main.wasp
file.
Here is the relevant extract for reference: Discord support thread: openai gpt Assistant api, or embeddings template? How to select version when git cloning? > Answers Part 3 (https://discord.com/channels/686873244791210014/1196842660938399744)
For more information on how to define actions and operations in Wasp, you can refer to the Wasp documentation here: Wasp Data Model Operations Overview (https://wasp-lang.dev/docs/data-model/operations/overview)
----------------------
Note: you can ask me a follow up question by @ mentioning me again :speech_balloon:
----------------------
Sorry @haarismian, I seem to be having a bit of trouble. Don't worry though, I've pinged my support and they will try to fix me up! :technologist:
Perhaps try again in 15mins?@martinsos any guidance here would be appreciated. I want to stream the results from open ai to my frontend
thanks!
can you give more context on your setup?..
here's how i would usually do it>>
and then do for loops on set state until the stream done
thanks @Emil , so what does that look like on your backend? my struggle is that in actions I don't know how to repeatedly get it to return the stream chunks. do I just put a return in the for await?
![No description](https://cdn.answeroverflow.com/1249055592480505969/image.png)
I am basically I guess trying to implement server sent events?
if you return. you only get once. if you yield. you will return multiple times.
try change return to yield
Here is an example by @miho on how to do streaming in Wasp, check it out! https://github.com/wasp-lang/wasp/tree/main/examples/streaming
Thank you very much!
@martinsos any way I can make that a post call?
@haarismian you should be able to, not sure how it vibes with streaming, but you can make it POST in Wasp, check it out here, the docs on API: https://wasp-lang.dev/docs/advanced/apis
Custom HTTP API Endpoints | Wasp
In Wasp, the default client-server interaction mechanism is through Operations. However, if you need a specific URL method/path, or a specific response, Operations may not be suitable for you. For these cases, you can use an api. Best of all, they should look and feel very familiar.
thank you @martinsos , wanted to report it did indeed work with streaming. In case you need it for future reference here's what I did
1. Followed the example and created a custom API and middleware but with http request as a post
2. passed the body object I wanted into the usetext stream functon in the react component as so:
useTextStream('/api/streaming-test', {stateVars});
3. updated the use textstream function to accept the state
4. updated fetchstream with the variable
Let me know if I can help with knowledge transfer in any way.
Thanks for yoru help
![No description](https://cdn.answeroverflow.com/1250693400110432318/image.png)
![No description](https://cdn.answeroverflow.com/1250693400345448558/image.png)
Thanks @haarismian that is awesome! This alreayd helps, these questions are searcheable by others and also published online so it will enable others to find your solution!