Img2txt code works locally but not after deploying
I am using a model for Image 2 text , i have made its handler file and tested it locally , for testing i have used a json file with just defining the input
{
"input": {
"image_path": "/content/1700052015451vm8aj9ac.png"
} }
and test it by !python -u rp_handler.py but when i made its dockerfile, pushed it and used it through serverless api, tested it on postman am getting error image not found this is my handler script
} }
and test it by !python -u rp_handler.py but when i made its dockerfile, pushed it and used it through serverless api, tested it on postman am getting error image not found this is my handler script
10 Replies
import runpod
from PIL import Image
from transformers import AutoModel, AutoProcessor
import torch
Load model and processor outside the handler for efficiency
model_name = "unum-cloud/uform-gen2-qwen-500m" # Replace with your model name
model = AutoModel.from_pretrained(model_name, trust_remote_code=True)
processor = AutoProcessor.from_pretrained(model_name, trust_remote_code=True)
Disable gradients and model head for faster inference
model.eval()
model.config.output_hidden_states = False
def process_image(input):
"""
Executes the image description logic with a hardcoded prompt
"""
image_path = input['image_path']
# Hardcoded prompt
prompt = "Image description" # Replace with your desired prompt
try:
image = Image.open(image_path)
except FileNotFoundError:
return {"error": f"Image not found at {image_path}"}
with torch.inference_mode():
inputs = processor(text=[prompt], images=[image], return_tensors="pt")
output = model.generate(
**inputs,
do_sample=False,
use_cache=True,
max_new_tokens=256,
eos_token_id=151645,
pad_token_id=processor.tokenizer.pad_token_id
)
prompt_len = inputs["input_ids"].shape[1]
decoded_text = processor.batch_decode(output[:, prompt_len:])[0]
return {"description": decoded_text}
---------------------------------------------------------------------------- #
RunPod Handler #
---------------------------------------------------------------------------- #
def handler(event):
"""
This is the handler function that will be called by RunPod serverless.
"""
return process_image(event['input'])
if name == 'main':
runpod.serverless.start({'handler': handler})
It’s cause you telling api to load image that is located on container storage
oh, then how can i get the image when i test api from postman?
Upload somewhere where you can get direct link to image
can you please see this, this can get image from any url (image publically accesible)
i am testing on colab, so i dont think i can can images from my local computer to test it out ? but for publically accessiuble urls , internet images it is working
It’s cause your code is not fully implemented you are missing input schema, validation and more
Oh okay, so you are saying even on colab it should work for images present on my local computer, i should chnage the code accordingly
I’m saying that it’s not proper serverless worker
okay , let me check further , but still big thanks atleast you gave me a start and pathway to correct my mistake