Scald
How to mass upload a list of files to blob storage?
I have a large number of files I need to upload to a blob storage, and I'm currently using the
BlobClient.UploadAsync(filePath, options); method to accomplish this. However, it's too slow, probably because I'm uploading files one by one.
I've seen some recommend the Microsoft.Azure.Storage.DataMovement for this use case online, that being a bulk upload of files.
However, my use case requires me to filter the files by matching with a CSV file first, which maps the current file names to the new names, and to avoid unnecessary files.
Has anyone have had to deal with a similar use case before? Bulk uploading files, having to filter them first? I'd rather not have to copy all the files to a new directory if I could avoid it.3 replies