R
RunPod3mo ago
Cai

"Something went wrong!" when trying to sync with Google Cloud Storage

Hello. Just trying to sync my pod's internal volume with google colud storage and getting popup with that error. No matter which data I enter - correct or incorrect, it happens. I'm using community cloud pod with runpod/pytorch:2.0.1-py3.10-rocm5.7-ubuntu22.04 image.
6 Replies
yhlong00000
yhlong000003mo ago
probably wrong credential, path and etc, check this doc: https://docs.runpod.io/pods/configuration/export-data#google-cloud-storage
Export data | RunPod Documentation
Export RunPod data to various cloud providers, including Amazon S3, Google Cloud Storage, Microsoft Azure Blob Storage, Backblaze B2 Cloud Storage, and Dropbox, with secure key and access token management.
Cai
CaiOP3mo ago
Already read this. I think maybe problem is with JSON key; didn't understand clearly which format it should follow (this is my first time integrating google cloud with... anything). Should it be a full JSON value like {"type": ..., "project_id": ..., "private_key": ..., ...} or only "private_key" field's value? If the latter, should I convert newlines to something different?
yhlong00000
yhlong000003mo ago
Should be full JSON
Cai
CaiOP3mo ago
That's was the first thing I tried; getting that error anyway.
yhlong00000
yhlong000003mo ago
Behind the scenes, the system uses rclone for copy file. We will update the UI to provide more detailed error messages. In the meantime, if you are familiar with rclone, you can use it directly to copy files to your bucket.
rclone config create remote "google cloud storage" service_account_credentials "<service_account_credentials>" location "<location>" bucket_policy_only "true"
rclone config create remote "google cloud storage" service_account_credentials "<service_account_credentials>" location "<location>" bucket_policy_only "true"
rclone copy <source> <destination>
rclone copy <source> <destination>
Cai
CaiOP3mo ago
It turned out to be a very stupid issue with my pod ran out of RAM and container was OOM killed, but in UI status was "running", so I've realized what's happening only when looked into logs. Because of "running" state that's was able to click on "cloud sync" button, submit form, etc., but not to actually do sync.

Did you find this page helpful?