Migration from KV to D1
Hi,
I am currently running a migration API in an existing pages project which fetches all resources from a certain KV namespace, and inserts those into a specific D1 database. Since there are quite a lot of items in the KV namespace, an internal server error is produced:
Too many API requests by single worker invocation. I understand that internal calls to certain services are limited to 1000. Is there any way to temporarily increase this in order to make the migration process easier?
Worst case I can limit the amount of items interacted with, but this is a little more tedious than just being able to migrate it all at the same time.2 Replies
Unfortunately I don't think there is a way to get this rasied - I've never heard of anyone with a limit above 1k, maybe if you're an ent customer you can ask your CSM
That's okay, thank you anyway! I resorted to creating a simple API for this migration and limiting the amount of records added at once to stay under the 1k limit. By returning the list cursor I have a simple way to iterate over all the records.