Streaming partial extraction
I want to replicate this streaming partial extraction: https://x.com/jxnlco/status/1755723172228350056
The basic idea is to use an LLM to extract some particular information (for example, name, e-mail and twitter login) from a text. I think I'm able to do it inside the solara chatbot example. That is, I send a message (which will not be displayed) just to activate the extraction and then I will stream the response within the solara.lab.ChatBox.
I don't like this solution because ideally I would like to activate the extraction with a button, not with solara.lab.ChatInput. Any hint about how to accomplish this?
jason liu (@jxnlco) on X
instructor also supports partial streaming of json so:
1. don't wait for the full payload
2. no stress about json.loads
what do you think @ollama
Twitter
7 Replies
This is what I have for the moment: https://huggingface.co/spaces/alonsosilva/Extractor
You can send a message (which will not be displayed) just to activate the extraction and then I will stream the response. For the moment, I just have lorem ipsum not to complexify the code.
Hi Alonso,
I am not sure what the question is. Do you want instead of the ChatInput a separate input and a button?
Or do you want an example with streaming output? Because the chat bot example we have already does that right?
Thank you @MaartenBreddels
Indeed if you look at the huggingface space I just shared I am using the chat box example but I want to have a text area (which I already know how to build and a button). When I click the button it can extract entities from the text inside the text area.
Indeed if you look at the huggingface space I just shared I am using the chat box example but I want to have a text area (which I already know how to build and a button). When I click the button it can extract entities from the text inside the text area.
Would something like
work, where in send, you craft your prompt?
Let me think about it. I guess it could work with a ChatInput where instead of InputText there is a ChatTextArea. Let me see and I’ll update you
Thank you @MaartenBreddels Actually you were right all along. It was as simple as that. The application now extract names and ages from the text:
https://huggingface.co/spaces/alonsosilva/Extractor
nice 🙂
