SolidJS•6mo ago

Does streaming work in SolidJS with signals?

Hello! I was trying to update my OpenAI Assistant to use the new streaming API and I can successfully get it to work and log to the console, but when I update a signal instead of console.loging...the UI never updates after the first item? I'm guessing there is something on the "fine-grain" reactivity playing a part of this? I'm new to API streaming but would love to add this feature to my app and be more performant (also have a cleaner codebase)? Anything I'm missing? Context: https://platform.openai.com/docs/assistants/overview?lang=node.js&context=with-streaming

20 Replies

caseybaggz•6mo ago

Basically, doing this:

createEffect(() => {
    if (data()?.run) {
      data()
        ?.run.on('textCreated', () => {
          // Assistant begins typing...
          setAssistantPending(false);
        })
        .on('textDelta', (textDelta, snapshot) => {
          setStream((prev) => prev + textDelta.value);
        })
        .on('textDone', (text) => {
          // Assistant has finished typing...
          setThread(
            produce((state) => {
              state.messages.push({
                id: state.messages.length,
                role: 'assistant',
                value: text.value,
              });
            })
          );
          updateScrollView();
        });
});

createEffect(() => {
    if (data()?.run) {
      data()
        ?.run.on('textCreated', () => {
          // Assistant begins typing...
          setAssistantPending(false);
        })
        .on('textDelta', (textDelta, snapshot) => {
          setStream((prev) => prev + textDelta.value);
        })
        .on('textDone', (text) => {
          // Assistant has finished typing...
          setThread(
            produce((state) => {
              state.messages.push({
                id: state.messages.length,
                role: 'assistant',
                value: text.value,
              });
            })
          );
          updateScrollView();
        });
});

Then in the JSX, using stream() only updates the first item on the stream and never again.

thetarnav•6mo ago

can you create a minimal reproduction in the playground without the open ai stuff? if the problem is solid api

caseybaggz•6mo ago

But wouldn't that defeat the purpose? If I remove the open ai stuff it's just an effect and signal update?

thetarnav•6mo ago

is the problem open ai api or solid api? you say that the ui doesn't update so I'm guessing that it's a problem with understanding and debugging solid api reproducing that in a clear environment will help with that you can also share more code about how the store is getting setup and used becuase updating looks fine also it's an api for node (the open ai one) is is meant to be useed in the browser?

caseybaggz•6mo ago

Right. I guess what I'm asking is if there is anything particular in the signal/store logic that prevents streaming updates? I can log all day and it works but the UI will never re-render the signal even though an effect will log it correctly. So, I'm assuming there is something deeper in the "fine grain" details on the Solid side that is not so compatible with streaming updates?

thetarnav•6mo ago

what do you mean by "streaming" in this context?

caseybaggz•6mo ago

https://platform.openai.com/docs/assistants/overview?lang=node.js&context=with-streaming

caseybaggz•6mo ago

It's a readable stream: https://developer.mozilla.org/en-US/docs/Web/API/Streams_API/Using_readable_streams

MDN Web Docs

Using readable streams - Web APIs | MDN

That explains the basics of "default" readable streams.

thetarnav•6mo ago

it looks like some nodejs api for communication with open ai so are you using it on the server? are you using solidstart?

caseybaggz•6mo ago

Vanilla Solid

thetarnav•6mo ago

ok and youre using this on the client?

caseybaggz•6mo ago

Client

thetarnav•6mo ago

ok so every time you get a chunk of text you update a signal and that doesn't work? if you remove the stream and just set the signal in setInterval does it work?

caseybaggz•6mo ago

Oh, I wonder if I need to do something with this? 🤔 https://docs.solidjs.com/reference/rendering/render-to-stream

thetarnav•6mo ago

this is for ssr are you using ssr? from what I'm understanding stream is just a way to get chunks of data over time from a single long request it has nothing to do with solid solid just cares how you use signals so you should be a be able to reproduce that without using a stream or open ai, it's just an issue with susing signals proparely

Gaming

Programming

Does streaming work in SolidJS with signals?