CF CALLS & Workers AI

I've seen a demo on twitter that looks really cool: https://x.com/kwindla/status/1815892810584723609 it uses daily.co on the backend to enable a webRTC connection and https://github.com/pipecat-ai/pipecat (python) to handle the pipeline of speech-to-text -> LLM -> text-to-speech and function calling etc. The webRTC stuff helps makes streaming audio really really fast obviously and that's kind of the killer feature I'm after, since it needs to be as seamless as possible. I'm trying to get my head around how to build something like this perhaps with cloudflare calls? I'd probably still use grok for the llm/whisper api calls simply because it's so fast, but I do currently use a cloudflare worker for whisper. Like in a cloudflare only setup should I be thinking about connecting the client and a durable object to a call and having the durable object orchestrate the whisper -> llm -> tts, is that possible? Any help or thoughts appreciated!

kwindla (@kwindla) on X

Very, very fast voice bots. Llama 3.1 running on @GroqInc. 🚀 500ms voice-to-voice response times

Twitter

GitHub

GitHub - pipecat-ai/pipecat: Open Source framework for voice and mu...

Open Source framework for voice and multimodal conversational AI - pipecat-ai/pipecat