listobjectv2 pagination
Dears,
I hope you are well. I would like to use the https://{{account-id}}.r2.cloudflarestorage.com/{{r2-bucket}}
endpoint with pagination on an arbitrary number of keys per page.
For example, I would like to set the max-keys to 20, having 100 keys in the bucket, with a continuation token that allows me to query the 21-40 keys. My issue is that the continuation token is not being sent for such small amounts.
Also, I have another concern. Suppose that the pagination logic works, how can I know the total amount of items in the bucket without iterating through all of the pages? (To know how many pages are left)
3 Replies
That's just the S3 API, there's tons of docs for it around the place: https://docs.aws.amazon.com/AmazonS3/latest/API/API_ListObjectsV2.html
ListObjectsV2 - Amazon Simple Storage Service
Returns some or all (up to 1,000) of the objects in a bucket with each request. You can use the request parameters as selection criteria to return a subset of the objects in a bucket. A 200 OK response can contain valid or invalid XML. Make sure to design your application to parse the contents of the response and handle it appropriately. For mor...
My understanding is it doesn't give you the total amount of items, you'd have to go through each page. The IsTrunacted will tell you if there's more, and then you can use the NextContinutationToken, and the KeyCount for keys returned in that request
As you correctly point out 💪 the IsTruncated attribute refers to whether there are more records that are not being shown. However, in my case the response object contains the is truncated true without a NextContinuationToken.😅
Any ideas? Thanks in advance