SolidJS•9mo ago

Does streaming work in SolidJS with signals?

Hello! I was trying to update my OpenAI Assistant to use the new streaming API and I can successfully get it to work and log to the console, but when I update a signal instead of console.loging...the UI never updates after the first item? I'm guessing there is something on the "fine-grain" reactivity playing a part of this? I'm new to API streaming but would love to add this feature to my app and be more performant (also have a cleaner codebase)? Anything I'm missing? Context: https://platform.openai.com/docs/assistants/overview?lang=node.js&context=with-streaming

20 Replies

caseybaggzOP•9mo ago

Basically, doing this:

createEffect(() => {
    if (data()?.run) {
      data()
        ?.run.on('textCreated', () => {
          // Assistant begins typing...
          setAssistantPending(false);
        })
        .on('textDelta', (textDelta, snapshot) => {
          setStream((prev) => prev + textDelta.value);
        })
        .on('textDone', (text) => {
          // Assistant has finished typing...
          setThread(
            produce((state) => {
              state.messages.push({
                id: state.messages.length,
                role: 'assistant',
                value: text.value,
              });
            })
          );
          updateScrollView();
        });
});

createEffect(() => {
    if (data()?.run) {
      data()
        ?.run.on('textCreated', () => {
          // Assistant begins typing...
          setAssistantPending(false);
        })
        .on('textDelta', (textDelta, snapshot) => {
          setStream((prev) => prev + textDelta.value);
        })
        .on('textDone', (text) => {
          // Assistant has finished typing...
          setThread(
            produce((state) => {
              state.messages.push({
                id: state.messages.length,
                role: 'assistant',
                value: text.value,
              });
            })
          );
          updateScrollView();
        });
});

Then in the JSX, using stream() only updates the first item on the stream and never again.

thetarnav•9mo ago

can you create a minimal reproduction in the playground without the open ai stuff? if the problem is solid api

caseybaggzOP•9mo ago

But wouldn't that defeat the purpose? If I remove the open ai stuff it's just an effect and signal update?

thetarnav•9mo ago

is the problem open ai api or solid api? you say that the ui doesn't update so I'm guessing that it's a problem with understanding and debugging solid api reproducing that in a clear environment will help with that you can also share more code about how the store is getting setup and used becuase updating looks fine also it's an api for node (the open ai one) is is meant to be useed in the browser?

caseybaggzOP•9mo ago

Right. I guess what I'm asking is if there is anything particular in the signal/store logic that prevents streaming updates? I can log all day and it works but the UI will never re-render the signal even though an effect will log it correctly. So, I'm assuming there is something deeper in the "fine grain" details on the Solid side that is not so compatible with streaming updates?

thetarnav•9mo ago

what do you mean by "streaming" in this context?

caseybaggzOP•9mo ago

https://platform.openai.com/docs/assistants/overview?lang=node.js&context=with-streaming

caseybaggzOP•9mo ago

It's a readable stream: https://developer.mozilla.org/en-US/docs/Web/API/Streams_API/Using_readable_streams

MDN Web Docs

Using readable streams - Web APIs | MDN

That explains the basics of "default" readable streams.

thetarnav•9mo ago

it looks like some nodejs api for communication with open ai so are you using it on the server? are you using solidstart?

caseybaggzOP•9mo ago

Vanilla Solid

thetarnav•9mo ago

ok and youre using this on the client?

caseybaggzOP•9mo ago

Client

thetarnav•9mo ago

ok so every time you get a chunk of text you update a signal and that doesn't work? if you remove the stream and just set the signal in setInterval does it work?

caseybaggzOP•9mo ago

Oh, I wonder if I need to do something with this? 🤔 https://docs.solidjs.com/reference/rendering/render-to-stream

thetarnav•9mo ago

this is for ssr are you using ssr? from what I'm understanding stream is just a way to get chunks of data over time from a single long request it has nothing to do with solid solid just cares how you use signals so you should be a be able to reproduce that without using a stream or open ai, it's just an issue with susing signals proparely

caseybaggzOP•9mo ago

Not sure how to really reproduce this since openAI is using an actual fetch stream (not the same as just faking it) but this is kind of what it is doing - but it still doesn't explain why the real deal logs fine in an effect but doesn't update in the rendered JSX like the example does? https://stackblitz.com/edit/solidjs-templates-3g8e9e?file=src%2FApp.tsx

Casey Baggz

StackBlitz

Solidjs - Templates (forked) - StackBlitz

Vite + solid templates

thetarnav•9mo ago

so the reproduction works fine but when using open ai it doesn't? then it must be something specific to your local env what happens if you do the same as on stackblitz but locally - reduce the code to a simple case does is work then? I'm not sure what else can be causing this can you show more code than the effect in which you are updating the signal/store? it doesn't show how your data structure really looks like and how it's used

caseybaggzOP•9mo ago

Yeah, the code comes in chunks and not streaming. I feel like I read somewhere that signals defer for performance which I think it is the reason. The reproduction doesn't work fine though - should be loading very quickly at every 200ms (or whatever I put) but instead loads slower.

bigmistqke•9mo ago

could you share the code of whole component? including jsx and the like.

caseybaggzOP•9mo ago

I cannot outside of the example I have because it is a private code base

Gaming

Programming

Does streaming work in SolidJS with signals?