esho
RRunPod
•Created by esho on 4/12/2024 in #⚡｜serverless
Faster Whisper Latency is High
I am using RTX3090 and RTX4090.
22 replies
RRunPod
•Created by esho on 4/12/2024 in #⚡｜serverless
Faster Whisper Latency is High
Although it is called serverless, only me is using the server after cold start. This should be really fast.
22 replies
RRunPod
•Created by esho on 4/12/2024 in #⚡｜serverless
Faster Whisper Latency is High
I am using the default config. I think it should run as fast as the local machine.
22 replies
RRunPod
•Created by esho on 4/12/2024 in #⚡｜serverless
Faster Whisper Latency is High
the executionTime is in the response, about 800ms. I think this is also high.
22 replies
RRunPod
•Created by esho on 4/12/2024 in #⚡｜serverless
Faster Whisper Latency is High
I also tested it on GCP
22 replies
RRunPod
•Created by esho on 4/12/2024 in #⚡｜serverless
Faster Whisper Latency is High
it's from my PC.
22 replies
RRunPod
•Created by esho on 4/12/2024 in #⚡｜serverless
Faster Whisper Latency is High
"import time
start = time.time()
response = requests.post(url, json=payload, headers=headers)
print("Time taken: ", time.time() - start)"
a very simple scirpt, and there is "executionTime" in the respone. "executionTime" is about 800ms.
22 replies
RRunPod
•Created by Cereberus on 4/1/2024 in #⚡｜serverless
Can we use serverless faster Whisper for local audio?
I think you can use base64, here is a example:
audio_file = "audio2.wav"
audio_base64 = base64.b64encode(open(audio_file, "rb").read()).decode("utf-8")
payload = {
"input": {
"audio_base64": audio_base64,
"model": "tiny",
"transcription": "plain_text",
"translate": False,
"language": "en",
"temperature": 0,
"best_of": 1,
"beam_size": 1,
"patience": 1,
"suppress_tokens": "-1",
"condition_on_previous_text": False,
"temperature_increment_on_fallback": 0.2,
"compression_ratio_threshold": 2.4,
"logprob_threshold": -1,
"no_speech_threshold": 0.6,
"word_timestamps": False
},
"enable_vad": False
}
8 replies