siggmus Comments - Answer Overflow

siggmus

•Created by siggmus on 9/17/2023 in #workers-help

How to make a Worker stream (an OpenAI) response back to the front-end?

@bkyerv Thank you. It would have taken me some time to figure that out. That solved it. For others who have the same issue, here is the modified workers code:

const { OpenAI } = require("openai");

export default {
    async fetch(request, env, ctx) {
        const openai = new OpenAI({
            apiKey: "##-#####"
        })

        // make our request to the OpenAI API
        const stream = await openai.chat.completions.create({
            model: "gpt-3.5-turbo",
            messages: [{ role: "user", content: "Tell me a story using 1000 chars." }],
            stream: true
        })

        const headerCode = {
            'Access-Control-Allow-Origin': '*',
            'Access-Control-Allow-Headers': 'Origin, X-Requested-With, Content-Type, Accept',
        }


        // Using our readable and writable to handle streaming data
        let { readable, writable } = new TransformStream()

        const writer = writable.getWriter();
        const encoder = new TextEncoder();
        const response = new Response(readable, {
            headers: headerCode
        });
        (async () => {
            try {
                for await (const chunk of stream) {
                    const content = chunk['choices'][0].delta.content;
                    const encodedMessage = encoder.encode(content);
                    writer.write(encodedMessage);
                }
                writer.close();
            } catch (e) {
                console.error(e);
                // writer.abort(e);
                return new Response('bad bad error', {
                    headers: headerCode
                });
            }
        })();
        return response;
    }
}

4 replies

Gaming

Programming