R2 Files loading Blank Page
I've had a bunch of foldered files that make up a training course package stored in R2 for a while (over a year). At the root of the folder is an html file (index.html or story.html) that when accessed would load the training material using the other files within the folder structure. This has worked for over a year until this week it all stopped working.
I've tried:
-deleting the files and re-uploading
-messing with CORS
-creating a fresh bucket
-tried multiple methods of uploading (GUI, API, S3-Browser, CyberDuck)
Nothing seems to work and the only errors i get are 404 for select files that the index.html is trying to call within the same folder.
I suspect this is related to multipart upload issues but I cannot get the files to upload correctly, which i note was not an issue in the past. This also doesn't explain why the previous batch of uploaded folders/files that were working all the sudden stopped.
Looking for guidence on how i should be uploading these files correctly so that it completes with all files in there correct place.
For references the total folder size is 400+MB with about 200+ individual files ranging from bytes/kbs all the way to 1, 5, 10, or even 50MB for some videos and pictures. Hence the need for proper Multipart Uploading.
Outside of that, looking for any insight as to what might have changed on CF's and how they are handling html files stored and served from R2?
24 Replies
Did you get this working? I saw you asked some questions in #r2
rclone is indeed a really helpful and useful tool that should just work, shouldn't even need to mess with its settings for simple transfers
For references the total folder size is 400+MB with about 200+ individual files ranging from bytes/kbs all the way to 1, 5, 10, or even 50MB for some videos and pictures. Hence the need for proper Multipart Uploading.You don't even need to use multipart up to 5 GiB, although some tools will ahead of that limit for performance and stopping entire reuploads.
Outside of that, looking for any insight as to what might have changed on CF's and how they are handling html files stored and served from R2?Nothing recently
Hey, so far I haven't been able to get things working. Or at least working in a way like they were before with R2. I got all the rsync commands going but like you said there probably wasn't a need for that. I've resorted to using Linodes S3 obj storage for now as that seems to work with the same folders and same rsync upload cmd (plus needing to set files to public read). But for now I can't get those files to render and get 404 or 400's for select files in the folder structure when loading the index.html file.
It also seems like the folder or url/i path is not translating correctly from the root R2 domain to where these files are actually sitting with the R2 folder structure. Not sure if others have had something similar?
generally most of the issues I've seen with files/folders in R2 (or object storage in general) is just confusion about them since Object storage doesn't actually have folders, it's all virtual.
If you go to where you think the assets are, do they load fine? If you click on a file name in the R2 bucket, it'll give you the R2 Custom Domain it's reachable at, worth sanity checking "is this asset actually reachable, and if so what is the html page trying to load/is it any different"
ya for sure, using "folder" as a general term. And yes I have tried that, as far as a i can tell the individual files load but it is hard to tell if they are working correct since they are things like .css .js and .wolf files for elements that get referenced in an online training package output that I don't control. Its standard practice to store and deliver these packages via S3 storage and for over a year I had done that via S3 and R2 for the last 6 months or so.
The odd thing is the package (folder) I uploaded 6 months ago that had been working just stopped, which is what caused me to delete and try re-uploading and thats where im at lol. Again Linodes S3 has continued to work the whole time including new uploads so it was my guess that something about the R2 config had changed at least in someway.
could you give the link of a broken page on r2, or look at the console (ctrl+shift+i -> console or right click -> inspect -> console) and look for errors about assets?
Sure, heres the link of the broken folder: https://lmsfiles.aesi-inc.com/lms-test-rise-3%2Fcontent%2Findex.html
Here for reference is the same folders/files on Linodes S3: https://acumen-training.us-east-1.linodeobjects.com/how-to-protect-your-data/content/index.html
Thanks for following up and having a look!
See how the slashes are different between r2 and s3 in your links? That's not R2 doing it (although I think the dashboard might show them wrong), and it looks like when you do that the browser takes them as escapes and thinks the relative directory is
/, so it tries to load the resources with the wrong relative directory, looking for /lib/icomoon.css, but if you do the slashes right: https://lmsfiles.aesi-inc.com/lms-test-rise-3/content/index.html
it works fine, and your relative assets work /lms-test-rise-3/content/lib/icomoon.csshmm, interesting, I still feel like this is a change or bug on R2 side cause like i said i had files uploaded (same type of folder setup) which were working that allt he sudden stopped working. Then all future uploads like this have this same issue. I've tried uploading with multiple S3 clients or cli's and on Mac and windows so i don't think all of those are messing with the slash direction?
only thing i can think of outside of a bug with our R2 tenant is a setting in CloudFlare for the custom domain thats used that is changing those paths? Maybe the URI formatting, could that do it?
as far as I can see, this is just a browser thing/whatever is generating your links to them. You can mess up linode in exactly the same way
https://acumen-training.us-east-1.linodeobjects.com/how-to-protect-your-data%2Fcontent%2Findex.html
guess the real question is how you do generate those links/what is generating those links url encoded?
I copied the link i gave you directly from the R2 dashboard:
ahh yea the dash is messed with that and has been forever
but that wouldn't explain why it only broke now
maybe the better question is: If you fix up all your links to be
/ instead of %2F, is there any that are still broken/don't work?ya, i have had a few weird things happen with links from there but nothing like this and nothing that stays after multiple refreshes, relogs, etc
I only have 2 folders right now for testing and i just tried both and they work with the swap
in prod or even dev I will be grabbing these links via the R2 API or Cf workers so i don't think it will be an issue but this weirdness was stopping me from even starting that lol
idk how your other things are setup, but they wouldn't break like this unless they relied on relative directory links like that specific page does
normally you see relative from base
/index.css or absolute https://r2.example.com/index.css which would both be fineYa i have another bucket for PDF docs so single files and they work fine.
yea single files would work fine too, no relative linking at all in that case
well eitherway sounds like the issue was just the r2 dash showing slashes in the preview urls as
%2F (url escaped) and then the browser breaking on relative asset links in that exact setup, maybe could try to push for it to be fixed now as before it was just a harmless display thing but it breaking webpages in some cases is super annoying