KnightOfRohan
Question about File Upload Security announcement on May 20th regarding file names and mimtypes.
I haven't had a chance to follow-up on this question yet but I'm guessing the way I've asked the question is very convoluted. Apologies.
To restate my question a little more clearly, hopefully:
Based on the #💫┊announcements from May 20th regarding File Upload security and the linked documentation, my understanding is that someone could upload a file with a falsified MIME type, and the acceptedFileTypes() method won't catch this, leading to potential security risks like PHP files being uploaded as other file types.
I assume the falsified MIME type is a surface-level issue and not related to the actual content of the file, as certain hex code patterns carry file type information.
However, if the acceptedFileTypes() method uses Laravel's mimetypes rule, which supposedly reads the file's contents to guess the MIME type, shouldn't it be able to catch falsified MIME types?
Could someone clarify what I'm missing or where the gap in my understanding is regarding how the Validator checks the file's MIME type based on its contents and why this might still be a security concern?
Thank you!
3 replies
Import action freezes browser window when importing 86k rows of data. How to troubleshoot?
I will either submit it, after I've finished the code or put it here so someone who has more experience than me can look it over and submit it themselves.
50 replies
Import action freezes browser window when importing 86k rows of data. How to troubleshoot?
Ah, so if it works similarly to LazyCollection, the difference maker is that the approach you are proposing doesn't read it in one request, but rather queues the reading of the chunks, right?
50 replies
Import action freezes browser window when importing 86k rows of data. How to troubleshoot?
I need to learn how to explore the Filament codebase like you have. I keep resorting to just the docs, which is pretty silly, now that I think of it.
50 replies
Import action freezes browser window when importing 86k rows of data. How to troubleshoot?
Apologies if this is a dumb question but does this mean that I would use a regular file upload action first, and then have that trigger an import action that I can use for the mapping and creation of the read and import queues? Does that make sense?
50 replies
Import action freezes browser window when importing 86k rows of data. How to troubleshoot?
I wonder if I can provide an estimate to the user based on the file size and maybe some historical data of past imports and maybe even the overall number of jobs in queue on the server rather than the number of rows 🤔
50 replies
Import action freezes browser window when importing 86k rows of data. How to troubleshoot?
The job you dispatch can then use a LazyCollection to read the file in in chunks, and dispatch a job to process each of those chunks, or you could even just process the chunks one by one in that job. You can send database notifications from that background job to inform the user about the progress, although I don't think there is any way to determine the number of rows in the file without reading the entire thing into memory.Perfect. This is exactly what I'm hoping for. I will work on this. Thank you so much for your help @Tetracyclic @awcodes !
50 replies
Import action freezes browser window when importing 86k rows of data. How to troubleshoot?
What I'm suggesting is that when you submit the file, it should just save the file to the server and dispatch a queued job with the filename and then immediately send a response to the user.Ahh, yes, that's definitely what I want. I want it to save the file to disk and then over time read it in parts and not load it all to memory.
50 replies
Import action freezes browser window when importing 86k rows of data. How to troubleshoot?
Based on my current understanding and knowledge, I find the LazyCollection approach very attractive, not just because it uses less memory, but because I can generally ensure a relatively consistent use of memory, rather than spikes, which I imagine helps with stability.
50 replies
Import action freezes browser window when importing 86k rows of data. How to troubleshoot?
Sorry I don't understand what you mean here:
So on a large import, if it happens in the initial request, it will still lock up the response until it's read the entire CSV file in.
50 replies
Import action freezes browser window when importing 86k rows of data. How to troubleshoot?
I might be a little too new to submit a PR, but IMHO, this approach with LazyCollections seems to deliver a more consistent experience, so I think it might be useful to have in the main framework.
50 replies
Import action freezes browser window when importing 86k rows of data. How to troubleshoot?
Thank you @Tetracyclic . I am a newb so I'm using ChatGPT to help me understand and implement this and it doesn't seem to propose the trait you mentioned so this is very helpful. I will learn about it and implement the solution.
50 replies