CD
Cloudflare Developers•2mo ago
fdc@青き月の魔女

Hello, I want to upload pg data backup to r2 at k8s, and I wrote manifest, but r2 status return Inte

Hello, I want to upload pg data backup to r2 at k8s, and I wrote manifest, but r2 status return InternalError. This is manifest: 
kind: CronJob
metadata:
  name: database-backup
spec:
  schedule: 0 0 * * *
  jobTemplate:
    spec:
      template:
        spec:
          initContainers:
          - name: pgdumper
            image: postgres
            command:
            - "pg_dump"
            - "--dbname=misskey"
            - "--host=database.misskey.svc"
            - "--user=misskey"
            - "--password"
            - "--file=/backup/backup.sql"
            env:
            - name: PGPASSWORD
              value: misskey
            volumeMounts:
            - name: data
              mountPath: /backup
          restartPolicy: OnFailure
          containers:
          - name: upload
            image: amazon/aws-cli
            command:
            - "aws"
            - "s3"
            - "cp"
            - "/backup/backup.sql"
            - "s3://misskey/backup/backup.sql"
            - "--endpoint-url=https://5aaa5369426edda5aff32cc4d275826e.r2.cloudflarestorage.com"
            - "--region=auto"
            - "--debug"
            env:
            - name: AWS_ACCESS_KEY_ID
              valueFrom:
                secretKeyRef:
                  name: database-backup
                  key: access_key_id
            - name: AWS_SECRET_ACCESS_KEY
              valueFrom:
                secretKeyRef:
                  name: database-backup
                  key: secret_access_key
            volumeMounts:
            - name: data
              mountPath: /backup
          volumes:
          - name: data
            emptyDir: {}
kind: CronJob
metadata:
  name: database-backup
spec:
  schedule: 0 0 * * *
  jobTemplate:
    spec:
      template:
        spec:
          initContainers:
          - name: pgdumper
            image: postgres
            command:
            - "pg_dump"
            - "--dbname=misskey"
            - "--host=database.misskey.svc"
            - "--user=misskey"
            - "--password"
            - "--file=/backup/backup.sql"
            env:
            - name: PGPASSWORD
              value: misskey
            volumeMounts:
            - name: data
              mountPath: /backup
          restartPolicy: OnFailure
          containers:
          - name: upload
            image: amazon/aws-cli
            command:
            - "aws"
            - "s3"
            - "cp"
            - "/backup/backup.sql"
            - "s3://misskey/backup/backup.sql"
            - "--endpoint-url=https://5aaa5369426edda5aff32cc4d275826e.r2.cloudflarestorage.com"
            - "--region=auto"
            - "--debug"
            env:
            - name: AWS_ACCESS_KEY_ID
              valueFrom:
                secretKeyRef:
                  name: database-backup
                  key: access_key_id
            - name: AWS_SECRET_ACCESS_KEY
              valueFrom:
                secretKeyRef:
                  name: database-backup
                  key: secret_access_key
            volumeMounts:
            - name: data
              mountPath: /backup
          volumes:
          - name: data
            emptyDir: {}
This is log file:

upload_3.log

14 Replies
sunyanguo
sunyanguo•2mo ago
I’m working on a website that allows users to upload files, and I’m encountering an issue where the file uploads to Cloudflare R2 sometimes fail. Even after retrying up to 5 times, each with a 1-second interval, the upload still doesn’t succeed. I’m unsure where the issue lies and would appreciate any advice on how to troubleshoot this problem effectively.
Unknown User
Unknown User•2mo ago
Message Not Public
Sign In & Join Server To View
Apolikamixitos
Apolikamixitos•2mo ago
Hey folks, I'm currently working on something and it seems like R2 has some instability when it comes to returning messages, It's just throwing We encountered an internal error. Please try again. for a while. I'm currently attempting it from some Github runners and it's failing but it's working fine using the CLI tool from my machine using aws-2.17, are there any further steps to get more verbosity from your side? 
2025-02-12 15:10:29,137 - MainThread - botocore.httpsession - DEBUG - Certificate path: /usr/local/aws-cli/v2/2.24.0/dist/awscli/botocore/cacert.pem
2025-02-12 15:10:29,137 - MainThread - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): ***.r2.cloudflarestorage.com:443
2025-02-12 15:10:29,228 - MainThread - urllib3.connectionpool - DEBUG - ***:443 "PUT /XX/XX/XX.json HTTP/1.1" 500 149
2025-02-12 15:10:29,228 - MainThread - botocore.hooks - DEBUG - Event before-parse.s3.PutObject: calling handler <function _handle_200_error at 0x7fe9f179d1c0>
2025-02-12 15:10:29,228 - MainThread - botocore.hooks - DEBUG - Event before-parse.s3.PutObject: calling handler <function handle_expires_header at 0x7fe9f179cfe0>
2025-02-12 15:10:29,229 - MainThread - botocore.parsers - DEBUG - Response headers: {'Date': 'Wed, 12 Feb 2025 15:10:29 GMT', 'Content-Type': 'application/xml', 'Content-Length': '149', 'Connection': 'keep-alive', 'Vary': 'Accept-Encoding', 'Server': 'cloudflare', 'CF-RAY': '910d81787806f167-ORD'}
2025-02-12 15:10:29,229 - MainThread - botocore.parsers - DEBUG - Response body:
b'<?xml version="1.0" encoding="UTF-8"?><Error><Code>InternalError</Code><Message>We encountered an internal error. Please try again.</Message></Error>'
2025-02-12 15:10:29,231 - MainThread - botocore.hooks - DEBUG - Event needs-retry.s3.PutObject: calling handler <function _update_status_code at 0x7fe9f179d300>
2025-02-12 15:10:29,137 - MainThread - botocore.httpsession - DEBUG - Certificate path: /usr/local/aws-cli/v2/2.24.0/dist/awscli/botocore/cacert.pem
2025-02-12 15:10:29,137 - MainThread - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): ***.r2.cloudflarestorage.com:443
2025-02-12 15:10:29,228 - MainThread - urllib3.connectionpool - DEBUG - ***:443 "PUT /XX/XX/XX.json HTTP/1.1" 500 149
2025-02-12 15:10:29,228 - MainThread - botocore.hooks - DEBUG - Event before-parse.s3.PutObject: calling handler <function _handle_200_error at 0x7fe9f179d1c0>
2025-02-12 15:10:29,228 - MainThread - botocore.hooks - DEBUG - Event before-parse.s3.PutObject: calling handler <function handle_expires_header at 0x7fe9f179cfe0>
2025-02-12 15:10:29,229 - MainThread - botocore.parsers - DEBUG - Response headers: {'Date': 'Wed, 12 Feb 2025 15:10:29 GMT', 'Content-Type': 'application/xml', 'Content-Length': '149', 'Connection': 'keep-alive', 'Vary': 'Accept-Encoding', 'Server': 'cloudflare', 'CF-RAY': '910d81787806f167-ORD'}
2025-02-12 15:10:29,229 - MainThread - botocore.parsers - DEBUG - Response body:
b'<?xml version="1.0" encoding="UTF-8"?><Error><Code>InternalError</Code><Message>We encountered an internal error. Please try again.</Message></Error>'
2025-02-12 15:10:29,231 - MainThread - botocore.hooks - DEBUG - Event needs-retry.s3.PutObject: calling handler <function _update_status_code at 0x7fe9f179d300>
cmaddox
cmaddox•2mo ago
Yes, for temporary access credentials, you must include the session token when making the request https://developers.cloudflare.com/r2/api/s3/tokens/#temporary-access-credentials
Nikos Vlassis
Nikos Vlassis•2mo ago
Hi there, is currently any issues with cloudflare r2 in EU Germany? suddenly we get 403 Forbidden on all links. we didn't change something on Cloudflare or DNS Records. CORS are disabled. Here is a link -> https://assets.gforgadget.gr/wp-content/uploads/2020/08/GforGadget_Logo-1.webp the website page is:https://gforgadget.gr/. with the R2.dev subdomain is working Any help appreciated!
SamTheWich
SamTheWich•2mo ago
Hi! I don't know if this is worth making a thread or help request for, but I've given my domain access to my bucket, but I can't quite figure out how to actually access it through my webpage. Can someone push me in the right direction?
Unknown User
Unknown User•2mo ago
Message Not Public
Sign In & Join Server To View
SamTheWich
SamTheWich•2mo ago
How would I find the key? Sorry I am VERY new to web development, and especially cloudflare Oh that seems simple. So if my bucket was named my_bucket and the folder I want to access was called my_folder: mywebsite.com/my_bucket/my_folder ?
Unknown User
Unknown User•2mo ago
Message Not Public
Sign In & Join Server To View
SamTheWich
SamTheWich•2mo ago
Okay interesting. It wouldn't be possible for me to extend my url would it? If on my website I have a section like mywebsite.com/pictures and I used r2 to host my pictures without a folder called pictures, can I do mywebsite.com/pictures/my_picture.jpg
Unknown User
Unknown User•2mo ago
Message Not Public
Sign In & Join Server To View
SamTheWich
SamTheWich•2mo ago
Okay okay, I'll try what I'm working on. This is why I love cloudflare ong they make everything easy
SamTheWich
SamTheWich•2mo ago
And would it change anything if my folder had a space in it?
No description
SamTheWich
SamTheWich•2mo ago
so %20? I figured, I think I will change it to that, I just want to see it working It seems like that broke my website... Just to be clear, only part of my page has to be in the bucket, right? Like I have most of my page set up with github that my webpage uses no issue, but now it gives me a 404

Did you find this page helpful?