KV Store max value exceeded for put
I have a durable object worker which uses the KV storage for my application. I noticed the value limit on the put. I was wondering if there are conventions around how I can avoid this limit on a put. Currently, I'm looking to do a try-catch and inspect the error message which I've observed to be something like this:
Would the try-catch + error message check be the conventional way to handle the put value limit?
8 Replies
Do you know anything about the values you are pushing?
they're basic objects but some of the values might be some large base64 string
like
{ b64: "...huge"}
also thank you for your quick response!
i do also have another requirement that kind of forces me to do the byte count bookeeping on the application side and im a bit unsure how to really handle doing this with JSON.stringify because of the nature of strings with typescript
(im basically looking to use the storage to store some edit history of a given document)Maybe go by string.length?
i guess would it be advised to just JSON.stringify() before i do a put
for the value
Yeah, you kind of have to do that, unless you use another serializer
yeah that makes sense. im not too familiar with typescript/javascript but do you happen to know of what string serialization issues there might be with the JSON.stringify and using the KV storage?
this is probably a super noob ts/js question
Try to use zlib to store the data, if it is mostly text. Inflate / Deflate are computationally quick and cheap, storage always have it’s limitations
i can give that a shot. i did go down a path using TextEncoder/TextDecoder in order to introduce some binary encoding for the storage. Thanks @VinerzZ !
on a similar note, this might not be a big deal but i'm sorta just wondering if i'm observing something weird or if i have something weird in my code but i'm trying to intentionally break the storage put by creating a very large string (
"a".repeat(131_072) is what im doing). What's interesting is when I am over the 131_072 limit (i did a 131_072 *1.5) i get a different length for the string i passed and the error message:
"[RangeError] Values cannot be larger than 131072 bytes. A value of size 131178 was provided."
String length: 131172
i am guessing this is some extra padding that might have been added in order to make the storage transaction. however, i just wanted to call this out as it gave me a little less confidence in what i am doing with this api. if you have any insight onto if my intuition is right, that would be great to hear