tkpop777
Explore posts from serversCDCloudflare Developers
•Created by tkpop777 on 7/27/2024 in #general-help
Does it make sense to use workers to host a chat endpoint?
Thanks for the response @Erisa , that makes me feel more at ease! I understand there is no need for the open socket connection since I just use the message the user sent as a prompt to query Open AI's api and return the response after doing a little bit of server side processing in the middle. The whole thing takes just a few seconds over the course of a standard https connection (as it would take hitting any regular REST api endpoint that wasn't AI related.
But I am still a little unsure if I should be using a serverless solution like workers here or a server that's running 24/7? Obviously the workers solution works great, it's super fast too (I already tried) but for something like this, would I be saving a ton more money/resources with the regular continuously running server option vs serverless? Assuming we're talking about massive scale?
Obviously the live server solution will always be cheaper at the expense of not having any of the great dev ops that is auto handled for you with serverless, but is it about the same as it would be for a regular REST endpoint, or would it be a lot worse since in a chat app you'd have lots of people constantly hitting the message endpoint non stop throughout the course of a regular convo?
I would feel a bit better if it's about the same or if anyone can say other big chat apps use serverless and the costs aren't astronomical relative to regular servers, then I want to go with the workers/serverless route and call it a day. Just want to confirm it's appropiate for this kind fo app, with this kind of usage characteristics..
4 replies