R
RunPod•5mo ago
vipul

Issue while running FastAPI TTS(Text To Speech) Docker Image on Serverless

We have made a TTS model that converts text input into an audio file. To serve this model, we created a REST API using the FastAPI framework with multiple endpoints, each performing specific tasks based on the text input. After building the API, we created a Dockerfile, pushed the image to Docker Hub, and attempted to run it on a serverless platform. However, while the container works perfectly on a local setup, it fails to run on the serverless environment. Could you assist with how to successfully run the RestAPI container on a serverless platform, and once the container is running, how to access the FastAPI endpoints to send text input and get audio responses? Docker File: 
FROM ubuntu:22.04

# Install system dependencies and Python 3.10
RUN apt-get update && apt-get install -y --no-install-recommends \
    python3 \
    python3-venv \
    python3-pip \
    python3-dev \
    curl \
    bash \
    build-essential \
    libsndfile1 \
    && apt-get clean \
    && rm -rf /var/lib/apt/lists/*

RUN apt-get update && apt-get install -y ffmpeg

# Set up virtual environment
ENV VIRTUAL_ENV=/opt/openvoice
RUN python3 -m venv $VIRTUAL_ENV
ENV PATH="$VIRTUAL_ENV/bin:$PATH"

# Upgrade pip
RUN pip install --no-cache-dir --upgrade pip

# RUN apt-get update && apt-get install -y python3-dev build-essential

# Install torch and other dependencies
RUN pip install --no-cache-dir torch setuptools>=40.8.0 langid fastapi soundfile librosa inflect unidecode eng_to_ipa pypinyin jieba
RUN pip install cn2an pydub faster_whisper whisper_timestamped
RUN pip install --upgrade setuptools

# Expose the ports used by the application
EXPOSE 8000

# Copy application code
COPY . .

# Create a startup script
RUN echo '#!/bin/bash\n\
source /opt/openvoice/bin/activate\n\
python tts_2.py' > /start.sh && chmod +x /start.sh

# Run the application
CMD ["/bin/bash", "/start.sh"]
FROM ubuntu:22.04

# Install system dependencies and Python 3.10
RUN apt-get update && apt-get install -y --no-install-recommends \
    python3 \
    python3-venv \
    python3-pip \
    python3-dev \
    curl \
    bash \
    build-essential \
    libsndfile1 \
    && apt-get clean \
    && rm -rf /var/lib/apt/lists/*

RUN apt-get update && apt-get install -y ffmpeg

# Set up virtual environment
ENV VIRTUAL_ENV=/opt/openvoice
RUN python3 -m venv $VIRTUAL_ENV
ENV PATH="$VIRTUAL_ENV/bin:$PATH"

# Upgrade pip
RUN pip install --no-cache-dir --upgrade pip

# RUN apt-get update && apt-get install -y python3-dev build-essential

# Install torch and other dependencies
RUN pip install --no-cache-dir torch setuptools>=40.8.0 langid fastapi soundfile librosa inflect unidecode eng_to_ipa pypinyin jieba
RUN pip install cn2an pydub faster_whisper whisper_timestamped
RUN pip install --upgrade setuptools

# Expose the ports used by the application
EXPOSE 8000

# Copy application code
COPY . .

# Create a startup script
RUN echo '#!/bin/bash\n\
source /opt/openvoice/bin/activate\n\
python tts_2.py' > /start.sh && chmod +x /start.sh

# Run the application
CMD ["/bin/bash", "/start.sh"]
2 Replies
Encyrption
Encyrption•5mo ago
You will need to write a handler using Python using the RunPod module. When you do this your handler will accept POST (JSON) requests to the RUN and RUNSYNC endpoints and returns JSON to the end user. Because of this you don't really need FastAPI. You can use the RunPod interface. If you do not want to remove the FastAPI you can code your handler to connect to it on 127.0.0.1 and act as a proxy between the user and the FastAPI. You can get started with this: https://docs.runpod.io/serverless/workers/handlers/overview
Overview | RunPod Documentation
Create and deploy serverless Handler Functions with RunPod, processing submitted inputs and generating output without managing server infrastructure, ideal for efficient, cost-effective, and rapid deployment of code.
vipul
vipulOP•5mo ago
Okay let me try this and get back to you , thanks for help

Did you find this page helpful?