Deploying MIGAN model to Serverless.
Hi, could someone recommend how to deploy the MIGAN model in a RunPod serverless wrapper?
There are 2 areas where I need advice:
Downloading the model
Invoking the model from Python to expose it in the RunPod wrapper.
I already have an implementation, but it is very bad and slow, when it should be very fast, as the model is very lightweight (<50MB) and even runs well on CPU.
I'm taking this to Docker and from there, setting up the serverless environment in RunPod. What tips can you share to implement this model effectively?
The model: https://github.com/Picsart-AI-Research/MI-GAN
Thanks!
GitHub
GitHub - Picsart-AI-Research/MI-GAN: [ICCV 2023] MI-GAN: A Simple B...
[ICCV 2023] MI-GAN: A Simple Baseline for Image Inpainting on Mobile Devices - Picsart-AI-Research/MI-GAN
20 Replies
read this
GitHub
GitHub - Picsart-AI-Research/MI-GAN: [ICCV 2023] MI-GAN: A Simple B...
[ICCV 2023] MI-GAN: A Simple Baseline for Image Inpainting on Mobile Devices - Picsart-AI-Research/MI-GAN
Hi, thanks for your anwser @nerdylive , would you mind to explain your pov? That's how already i've deployed on serverless, i mean, this is the work flow right now:
-A job is fired.
-The serverless worker starts running.
-The container is up.
-Inside the container, the iopaint API is mounted.
-It looks for the Model weights (can't figure out how to persist them).
-No Model present so it will download it.
-There, the job params are passed to the iopaint API.
-Iopaint process the request.
-The response (in happy path) all way back until container and worker are released.
How it should be done in my opinion:
-A job is fired.
-The serverless worker starts running.
-The container is up.
-MIGAN model is ready (no download required) and it *directly *process the job's req/params.
-The response (in happy path) all way back until container and worker are released.
In other words, I want to avoid to use iopaint wraper. I don't know much about python and the iopaint 'wraper' is a bit complex to me.
If someone can help or advice I would be very gratefull!
I can share snippets of code if needed of course
Ask on iopaint?
Im not really familiar with them so try exploring codes, docs
So sorry can't help this further except if you got the code that accepts some kind of input and outputs Ill try to help implementing it on runpod for this migan model
@giustiro bake the model into the container image
yeah or put it on network storage
Hi. thanks for your inputs, I've deployed a new docker version with updates.
As I mentioned before, I had a working example, sometimes it was filing, but working anyways.
Now I need some help, the exact same endpoint template, with new Docker tag (updated from 0.2.4 to 0.2.7) it is throwing a lot of logs and it always fails!
There were no significant changes, only tried to place the model relative to the
app & src folders, so the model is ready to use.
please @nerdylive @flash-singh can you take a look?
attaching the logs.Observations:
#1
HTTPConnectionPool(host='0.0.0.0', port=5000): its expected to be there until the internal FastAPI builds. Then the retry mechanism will hit it with job parameters. I repeat, THIS WAS WORKING around a month ago.
But there are a lot of other errors I haven't seen before, even the 0.0.0.0:5000/v1/api/inapint fails over and over.
#2
When serving the docker locally (with extra arg --rp_serve_api, everything starts as expected and no strange logs are shown.
#3
-Docker base image: runpod/base:0.6.2-cpu
-python sdk: runpod==1.4.2here it is, the exact same container, running perfectly on my end.
Hey
What kind of errors are you experiencing?
Are you maybe seeing pods as serverless? Also which one are you using?
I believe its serverless, please help me with posting the error you're experiencing ( full line )
Hi, thanks for your time, sure, the logs is plenty of errors and tracebacks, please do check the logs i've shared, let my attach it once again. Please open it and you will clearly see what I'm telling you.
In the other hand, when i ran the container locally, there is nothing but the expected messages from pythong indicating FastApi wrapers are up and running.
And this is the serverless config
Hows your handler file
ND dockerfile?
I have good news, it looks like the amount of logs showed can be dismissed, because I've tried my original config and it worked fine in the runpod' serverless.
So let me tweek some configs and will comment back my updates
Alright great
Well, everything is running fine now and the model is present on the container and accessible to the FastAPI. This is something!
Next improvements could be: Dismiss iopaint wrapper and understand how to run Migan jobs directly in a more performant and easy way.
Thanks for all your feedback.
Fast api?
Isn't serverless has it own built in api endpoints
You're using that right?
Hi, I know this is weird (it was one of the points I wanted to ride off), but YES. Inside the container, in serverless endpoint, I have no option (because I dont know how to build an alternative, that was what I was asking help for), to use the iopaint wrapper to run migan model, and this, is served upon a FastAPI instance.
https://github.com/Sanster/IOPaint/discussions/420
Sorry if my comments are unclear, ive tried to explain the best and english is not my main language.
GitHub
IOPaint Beta release · Sanster IOPaint · Discussion #420
IOPaint Beta Release It's been a while since the last release of lama-cleaner(now renamed to IOPaint.), partly due to the fact that during this time I have released my own first macOS applicati...
@nerdylive If you have some recommendations about this, please do share it, would be super helpful. Thanks
Oh yeah i think thats fine