giustiro
RRunPod
•Created by giustiro on 7/12/2024 in #⚡｜serverless
Deploying MIGAN model to Serverless.
@nerdylive If you have some recommendations about this, please do share it, would be super helpful. Thanks
32 replies
RRunPod
•Created by giustiro on 7/12/2024 in #⚡｜serverless
Deploying MIGAN model to Serverless.
Hi, I know this is weird (it was one of the points I wanted to ride off), but YES. Inside the container, in serverless endpoint, I have no option (because I dont know how to build an alternative, that was what I was asking help for), to use the iopaint wrapper to run migan model, and this, is served upon a FastAPI instance.
https://github.com/Sanster/IOPaint/discussions/420
Sorry if my comments are unclear, ive tried to explain the best and english is not my main language.
32 replies
RRunPod
•Created by giustiro on 7/12/2024 in #⚡｜serverless
Deploying MIGAN model to Serverless.
Well, everything is running fine now and the model is present on the container and accessible to the FastAPI. This is something!
Next improvements could be: Dismiss iopaint wrapper and understand how to run Migan jobs directly in a more performant and easy way.
Thanks for all your feedback.
32 replies
RRunPod
•Created by giustiro on 7/12/2024 in #⚡｜serverless
Deploying MIGAN model to Serverless.
So let me tweek some configs and will comment back my updates
32 replies
RRunPod
•Created by giustiro on 7/12/2024 in #⚡｜serverless
Deploying MIGAN model to Serverless.
I have good news, it looks like the amount of logs showed can be dismissed, because I've tried my original config and it worked fine in the runpod' serverless.
32 replies
RRunPod
•Created by giustiro on 7/12/2024 in #⚡｜serverless
Deploying MIGAN model to Serverless.
32 replies
RRunPod
•Created by giustiro on 7/12/2024 in #⚡｜serverless
Deploying MIGAN model to Serverless.
Hi, thanks for your time, sure, the logs is plenty of errors and tracebacks, please do check the logs i've shared, let my attach it once again. Please open it and you will clearly see what I'm telling you.
In the other hand, when i ran the container locally, there is nothing but the expected messages from pythong indicating FastApi wrapers are up and running.
32 replies
RRunPod
•Created by giustiro on 7/12/2024 in #⚡｜serverless
Deploying MIGAN model to Serverless.
32 replies
RRunPod
•Created by giustiro on 7/12/2024 in #⚡｜serverless
Deploying MIGAN model to Serverless.
#3
-Docker base image: runpod/base:0.6.2-cpu
-python sdk: runpod==1.4.2
32 replies
RRunPod
•Created by giustiro on 7/12/2024 in #⚡｜serverless
Deploying MIGAN model to Serverless.
Observations:
#1
HTTPConnectionPool(host='0.0.0.0', port=5000): its expected to be there until the internal FastAPI builds. Then the retry mechanism will hit it with job parameters. I repeat, THIS WAS WORKING around a month ago.
But there are a lot of other errors I haven't seen before, even the 0.0.0.0:5000/v1/api/inapint fails over and over.
#2
When serving the docker locally (with extra arg --rp_serve_api, everything starts as expected and no strange logs are shown.32 replies
RRunPod
•Created by giustiro on 7/12/2024 in #⚡｜serverless
Deploying MIGAN model to Serverless.
Hi. thanks for your inputs, I've deployed a new docker version with updates.
As I mentioned before, I had a working example, sometimes it was filing, but working anyways.
Now I need some help, the exact same endpoint template, with new Docker tag (updated from 0.2.4 to 0.2.7) it is throwing a lot of logs and it always fails!
There were no significant changes, only tried to place the model relative to the
app & src folders, so the model is ready to use.
please @nerdylive @flash-singh can you take a look?
attaching the logs.32 replies
RRunPod
•Created by giustiro on 7/12/2024 in #⚡｜serverless
Deploying MIGAN model to Serverless.
I can share snippets of code if needed of course
32 replies
RRunPod
•Created by giustiro on 7/12/2024 in #⚡｜serverless
Deploying MIGAN model to Serverless.
Hi, thanks for your anwser @nerdylive , would you mind to explain your pov? That's how already i've deployed on serverless, i mean, this is the work flow right now:
-A job is fired.
-The serverless worker starts running.
-The container is up.
-Inside the container, the iopaint API is mounted.
-It looks for the Model weights (can't figure out how to persist them).
-No Model present so it will download it.
-There, the job params are passed to the iopaint API.
-Iopaint process the request.
-The response (in happy path) all way back until container and worker are released.
How it should be done in my opinion:
-A job is fired.
-The serverless worker starts running.
-The container is up.
-MIGAN model is ready (no download required) and it *directly *process the job's req/params.
-The response (in happy path) all way back until container and worker are released.
In other words, I want to avoid to use iopaint wraper. I don't know much about python and the iopaint 'wraper' is a bit complex to me.
If someone can help or advice I would be very gratefull!
32 replies