R2 multipart upload failing with FileZilla Pro
This is a followup to https://community.cloudflare.com/t/cloudflare-r2-doesnt-support-listparts-operation/449580
When uploading a file using multipart upload, R2 fails with this error when calling CompleteMultipartUpload
All non-trailing parts must have the same length
Complete logs can be seen at https://community.cloudflare.com/t/r2-multipart-upload-failing-with-filezilla-pro/564712
Cloudflare Community
Cloudflare R2 doesn't support ListParts operation
I’m trying to use FileZilla Pro to upload large files to Cloudflare R2. It works fine for other S3-like storage. However with R2 it fails with ListParts not implemented 09:58:00 Command: GET /file-100M?uploadId=AAH6o24IV4Tcj4u3e%2BPabjQqzrcD4tqxDBRvnjpUK0kLlpCAD0LtYjLRhWI%2BrYY8uiXaGQsl/z6zdyCqV0iqYYEcRvBoV1Xkqys4M20quOgiDEJqaA47iLdFZNj2xkgThY...
Cloudflare Community
R2 multipart upload failing with FileZilla Pro
This is a followup to https://community.cloudflare.com/t/cloudflare-r2-doesnt-support-listparts-operation/449580 When uploading a file using multipart upload, R2 fails with this error when calling CompleteMultipartUpload https://docs.aws.amazon.com/AmazonS3/latest/API/API_CompleteMultipartUpload.html: All non-trailing parts must have the same ...
8 Replies
From your post:
The AWS S3 API documentation does not specify that the parts must have equal sizes.For R2 they do
Object part sizes must be at least 5MiB but no larger than 5GiB. All parts except the last one must be the same size. The last part has no minimum size, but must be the same or smaller than the other parts. The maximum number of parts is 10,000. Most S3 clients conform to these expectations.https://developers.cloudflare.com/r2/objects/multipart-objects/ If FileZilla has no option to disable that behavior then they'd need to add it or add R2 support directly, I don't imagine R2 is going to change on that anytime soon
Is there a way to get an official statement from Cloudflare team?
What would you want that statement to say that isn't in the docs?
That limitation has been that way ever since beta / 1.5 years, I don't really think that's going to change soon
but I noticed something, you're only uploading 2 parts when that happens? Perhaps the last part is bigger then the first?
Thank you for your reply.
you're only uploading 2 parts when that happens? Perhaps the last part is bigger then the first?FileZilla Pro dynamically changes the chunk size depending on the network performance.
well that's just not going to work with r2
maybe check if there's some way to disable that, or if you can increase the multipart upload size minimum. You can use PutObject up to 5 GB with R2, if you can raise FileZilla to use that and you only handle files less then that, could be a workaround
Indeed. However, this is not the first time we faced issues with R2. ListParts was not implemented until recently.
Anyway, thank you for the clarification.
Yea sadly R2 doesn't have full S3 compat. The S3 spec is large with lots of quirks and small things. R2 works with most S3 clients.
for GUI you could try CyberDuck (there's even a guide here: https://discord.com/channels/595317990191398933/940663374377783388/1155472378139267144). For CLI, Rclone works well (https://developers.cloudflare.com/r2/examples/rclone/).