5 Replies
increasing limits is always on our list 🙂 too soon to provide a timeline here though. curious, what are you using Queues & workers for? how would larger batches help?
Great, in this case we it for storing data for model fine tuning, we receive data via API queue and process to store as batchs of jsonl
Those jsonl ends in R2
1000 would be nice for our use case
similar in my case: lots of little requests coming in and getting queued, with the batch queue handler doing aggregate updates/inserts to R2. So increasing the batch limit translates directly to lower R2 put costs, 1000msg/2560kb would be fantastic
@scotto any chance you'd be interested in something new we're building? Quick sneak preview here
If yes, send me your account ID and I can get you setup (safe to share here)
it should make it much easier for you to build larger files in R2!
Would love to have access to that