Faster-Whisper worker template is not fully up-to-date
Hi,
We're using the Faster-Whisper worker (https://github.com/runpod-workers/worker-faster_whisper) on Serverless.
I saw that Faster-Whisper itself is currently on version
1.0.2, whereas the Runpod template is still on 0.10.0.
There are a few changes that have been introduced in Faster-Whisper (now using CUDA 12) since, that we would like to benefit from, especially the language_detection_threshold setting, since it seems like most of our transcriptions done by people with British accent are being transcribed into Welsh (with a language detection confidence of around 0.51 to 0.55) - which could be circumvented by increasing the threshold.
Are there any plans for this? It's kind of a blocker for us at the moment.8 Replies
I suggest forking the repo and changing it yourself.
Easier said than done 😉 We did actually make some attempts, but are not able to get it working. Seeing that this is a template provided by a paid service I thought it wouldn't hurt to ask.
RunPod sunset the managed endpoints, so they probably aren't maintaining the repos anymore.
https://blog.runpod.io/refocusing-core-strengths-shift-managed-ai-apis-serverless-flexibility/
RunPod Blog
Refocusing on Core Strengths: The Shift from Managed AI APIs to Ser...
RunPod is transitioning from Managed AI APIs to focusing on Serverless solutions, offering users more control and customization with comprehensive guidance.
Maybe, still it's a Quick Deploy template on Serverless.
As mentioned in the article you've shared:
Simplified Guide to Leveraging Serverless AI Functionalities [...] 2. Deploy with Ease: Use our Serverless platform to deploy your AI models without the complexity of managing infrastructure. Explore our easy-to-use quick deploy option.I would like to "deploy with ease" and said template is part of the section called "Quick Deploy" in the console 😅
Yeah you can quick deploy it, but to customise it thats your job.
Perhaps I'll be better of with Customer Support, thanks anyways.
Yeah log a ticket on website.
We did another attempt and it actually turned out to be a non-issue. We simply didn't make enough memory available to Docker in order to build the image.