Real time transcription using Serverless
creation of handler file for real time transcription app
Solution:Jump to solution
heres the high level function handler
- edit template and add ports to it, range can be anything, e.g. 5000-5010, depends if your workload can do parallel inference or its 1 per gpu (you will need our help to do this, its not available to public yet)
- start with first port 5000, open websocket server
- look at env for external port # for 5000, and get ip from env
- use inprogress hook in our sdk to send ip, port and any other info you want...
4 Replies
Solution
heres the high level function handler
- edit template and add ports to it, range can be anything, e.g. 5000-5010, depends if your workload can do parallel inference or its 1 per gpu (you will need our help to do this, its not available to public yet)
- start with first port 5000, open websocket server
- look at env for external port # for 5000, and get ip from env
- use inprogress hook in our sdk to send ip, port and any other info you want
- keep port and job in progress, until a message comes back in websocket server that says "x" and shut down websocket server and return the job marking it completed
client side:
- /run a job
- wait until job is in progress and has the ip and port
- connect the local client directly to ip and port for live communication
- keep in mind any security you need to setup for ip and port layer
@Justin Merrell can help with anything python specific
Just fyi:
https://discord.com/channels/912829806415085598/1194042962049503274
(Some other thread that isn't related to the real time socket, but related to another discussion about whisper / other alternatives)
please confirm the working is same as the below.
1. The handler.py will have code that creates a websocket server that listens on a channel
2. Front end connects to this websocket channel and sends chunks
3. Whisper processes the chunks and sends back the text on the websocket channel to the front end
4. When user stops the recording on the front end, the front end sends some kind of signal or command on the websocket channel to close the socket/channel and stop the websocket server, and handler,py will have no more operations to run, and the serverless instance stops.
5. there is no significant use for json input.
that looks about right, for 4. you need to detect close signal in web socket server, then stop the websocket and return out of handler