RunPod•4mo ago

Continuous Deployment for Pods

Hello, I recently transitioned from using Serverless Endpoints to Pods, but I'm encountering issues with my existing build and deployment workflow. Previously, with Serverless Endpoints, I had a setup in GitHub where I used GitHub Actions workflows to build container images, push them to my registry, and update the template image reference via the GraphQL API. When I updated the template, the endpoint would automatically restart and pull the new image. However, with Pods, this behavior doesn't seem to work the same way. Even after updating the template, the Pod continues running the "old" image and doesn't refresh automatically. Could you suggest a method to trigger a dynamic update or replacement of the Pod? Additionally, are there any other deployment strategies you recommend for my situation? I appreciate your assistance! Thank you!

10 Replies

yhlong00000•4mo ago

Yes, for serverless we handle worker updates for you, for pod you probably have to write more code to stop the pod, trigger our graphql api update template and call our api again to start the pod. Just curious what is the reason for you to switch from serverless to pod?

GuimanOP•4mo ago

Thank you for your response. This approach was pretty much what I had in mind. We had to transition to using Pods because we needed to send specific files, like pictures, to be processed by models. With the input payload, we were restricted to sending them only as Base64, which didn't meet our requirements. Now, by using Pods, we can set up our own Flask server to handle any kind of HTTP requests, files, etc.

yhlong00000•4mo ago

I see. One thing you could try is, instead of sending the entire pictures or files with each request, save them to a cloud storage location. Then, when sending requests, include the file name and path (or cloud storage bucket name) and have the serverless worker download the files from your cloud storage. This way, you’re not limited by input size. Similarly, after processing, you can have the serverless worker upload the results back to your cloud storage and simply return a success or failure status in the response.

GuimanOP•4mo ago

Thanks for the suggestion! If we were the only users, that would be an interesting approach. However, since our business involves providing an API endpoint for customers to upload their pictures and receive processed ones, I don't think this could be accomplished with a Serverless setup without using a proxy gateway to upload the images to Cloud storage, as you mentioned, or convert them to Base64 strings before sending them to the workers.

yhlong00000•3mo ago

@app.route('/process', methods=['POST'])
def process_image():
    # Get the image from the user
    image = request.files['image']

    # Upload to cloud storage and get the file path
    file_path = upload_to_cloud_storage(image)

    # Prepare payload for RunPod serverless
    payload = {
        "file_path": file_path,
        "bucket_name": "your_bucket_name"
    }

    # Call RunPod serverless API
    response = requests.post(RUNPOD_SERVERLESS_URL, json=payload, headers={"Authorization": f"Bearer {RUNPOD_API_KEY}"})
    result = response.json()
    finished_image_path = result.get("processed_image_path")

    # Download processed image
    processed_image = download_from_cloud_storage(finished_image_path)

    # Return the processed image to the user
    return send_file(BytesIO(processed_image), mimetype='image/jpeg')

@app.route('/process', methods=['POST'])
def process_image():
    # Get the image from the user
    image = request.files['image']

    # Upload to cloud storage and get the file path
    file_path = upload_to_cloud_storage(image)

    # Prepare payload for RunPod serverless
    payload = {
        "file_path": file_path,
        "bucket_name": "your_bucket_name"
    }

    # Call RunPod serverless API
    response = requests.post(RUNPOD_SERVERLESS_URL, json=payload, headers={"Authorization": f"Bearer {RUNPOD_API_KEY}"})
    result = response.json()
    finished_image_path = result.get("processed_image_path")

    # Download processed image
    processed_image = download_from_cloud_storage(finished_image_path)

    # Return the processed image to the user
    return send_file(BytesIO(processed_image), mimetype='image/jpeg')

def handler(event):
    # Get image path and bucket name from the event
    file_path = event.get('file_path')
    bucket_name = event.get('bucket_name')

    # Download the image from cloud storage
    image_data = download_from_cloud_storage(file_path)

    # Process the image 
    processed_image = process_image(image_data)

    # Upload the processed image to cloud storage
    finished_image_path = upload_to_cloud_storage(processed_image, bucket_name)

    # Return the path of the processed image
    return {"processed_image_path": finished_image_path}

def handler(event):
    # Get image path and bucket name from the event
    file_path = event.get('file_path')
    bucket_name = event.get('bucket_name')

    # Download the image from cloud storage
    image_data = download_from_cloud_storage(file_path)

    # Process the image 
    processed_image = process_image(image_data)

    # Upload the processed image to cloud storage
    finished_image_path = upload_to_cloud_storage(processed_image, bucket_name)

    # Return the path of the processed image
    return {"processed_image_path": finished_image_path}

GuimanOP•3mo ago

Thanks! If it get it @yhlong00000, there's one app running in a pod (script 1), and the other as a serverless endpoint (script 2), isn't it?

yhlong00000•3mo ago

The first one you can run it on any cloud provider(e.g. aws api gateway + lambda or our cpu pod or our cpu serverless) , since it just need a cpu, second one on runpod gpu serverless

GuimanOP•3mo ago

That's perfect, I'll try that. Thanks again! Could you share the GraphQL schema @yhlong00000 ? Because for some reason there seem to be discrepancies between the examples in the documentation and the GraphQL spec at https://graphql-spec.runpod.io/. I can't see all the mutations in the specs.

yhlong00000•3mo ago

I’m sorry, but I don’t have a schema to share at the moment. The link you have is the official one. I understand it’s not ideal, and we’re currently working on a more user-friendly REST API for developers, complete with better documentation. We’ll announce it as soon as it’s available.

GuimanOP•3mo ago

No problem, thank you! 🙂

Gaming

Programming

Continuous Deployment for Pods