FileUpload BUG
Hello, @everyone!
The.p12 automatically changes to.bin
The.crt .pen to .txt
Same with csv, js, css, and any text files...
All question is abandoned, im tired search the causes....
Why i cant preserve file extension, include for tmp file?!
Works only for getUploadedFileNameForStorageUsing + acceptedFileTypes((['text/javascript', 'text/plain'])
But first - filters all non js files, second filters all non text files
On uploading, in file manager - i can choose any text file, but it will be rejected by text/javascript
If ill remove text/plain - all files will be rejected by "File must be text/javascript type", but no files, event with .js extension - not has text/javascript mimetype. It all text/plain.
"text/javascript" Should be set to "accept" html attribute, on backend should be validated as text/plain and filename should be endswith .js exception!
Why no solves for this...? :squint:
50 Replies
I’m sorry. What? I don’t understand what is happening.
Wait a minute, I already asked a question, I thought this topic was simply being ignored)
I’m just not following what you are asking.
Nothing in filament is going to change the file extension. The acceptedFileTypes are used for validation only.
If I’m understanding your question right.
Example:
https://github.com/filamentphp/filament/issues/12760
If i upload .js file - it will be saved as .txt file
Ok, i can rename it manually with
but the validation problem remains
->acceptedFileTypes(['text/javascript'])
allows you to select only js files in the file manager, but a validation error will be generated when saving the form, since after loading the txt file from the temporary directory is validated, which is renamed by the function above only after passing this validation itself, and in fact the mimetype of the file is text/plain
->acceptedFileTypes(['text/javascript', 'text/plain'])
It works, file will be saved as .js file
but when you click on “upload file”, not only js files will be available, for example csv files, and other text files
css are not available for some reason
And if upload csv file - it will be rejected because it is matched with text/plain, but not matched with text/javascript
Breaking logic )
In short, when uploading csv, css, js or any text file, it is saved with the txt extension
And this spoils the interaction logic a little
I think the problem is that the temporary file is saved with a mimetype extension
And this does not fit in with attempts to validate a specific text file extensionHmm. I haven’t come across anything in filament, livewire or laravel where it renames the file extension. But the validation could be tricky since the program the file is saved out of can have unique mimetypes.
It not all encompassing. Like text/plain won’t cover a csv, for example, if the actual file mimetype is something like text/csv
Hmm... Or is this a Laravel feature that has been discussed for more than five years?
https://stackoverflow.com/questions/53323392/laravel-txt-file-being-converted-to-bin-on-upload
Stack Overflow
Laravel - .TXT file Being Converted to .BIN on Upload
I'm not quite sure where to look, so I can't even tell if this is an issue with the server or with the Laravel project itself, but on one of my models, if I upload a .TXT file attached to a record,...
Could very well be an issue in laravel. Just never experienced it myself.
Said(
I think system detects file mime type as text/plain
For example:
https://stackoverflow.com/questions/12003107/resource-interpreted-as-script-but-transferred-with-mime-type-text-plain-for-l
In windows it fixes via regedit
But macos and linux have same behavior by default
Strangely :squint:
Strangely :squint:
Stack Overflow
Resource interpreted as Script but transferred with MIME type text/...
I'm getting a "Resource interpreted as Script but transferred with MIME type text/plain" warning in Google Chrome when including a local script file.
I know the problem appears when loading a file...
I can’t find the ability to validate a file simply by its extension or name in Filament FIleUpload
Also, the temporary file is saved by mime type, which is also not very cool, but in principle it can be solved if you simply force the required extension and validate it as text/plain
I hear you, but the problem with that is that any file can have any extension and could therefore be an actual malicious file.
Indeed, I found the real reason in the Laravel security report
I’ve been exploring this theme for two days now, and I’ve only just gotten to the bottom of it.
https://securinglaravel.com/laravel-security-file-upload-vulnerability/#1-don%E2%80%99t-trust-user-input
1. Don’t Trust User Input The simple fix here is to not trust the file extension when saving uploaded files. Instead use the extension as reported by the mime type.preserve filenames - rewrites saved files with same name bad idea in all respects)
Yep. But it can still be a problem when mimetypes aren’t evaluated correctly. And multiple applications can save the file with different mimetypes. See this a lot with Microsoft items.
Unfortunately, there’s just no single source of truth for mimetypes.
And things like js and css are just text / plain. But could be diffent depending on which application saved the file.
But those kind of things make sense to be text/plain since, especially with js, it could be executed on the server in a node environment.
If you are excepting the responsibility of the upload though, you should be able to store the file with a .js extension explicitly with the storage facade.
Cool, thanks, very detailed answer
Thank you for your time)
But what is really missing is a separate ability to specify the accept attribute for html input when selecting a file
That’s just a matter of including all the possible mimetypes.
As long as the file is one of the mimetypes in the array it should pass.
Like, text/csv could be different than the minetype for an excel generated csv, but both have the same.csv extension. But have different mimetypes. So the acceptedFileTypes() would need to include all possible types for csv files to pass validation.
CSV alone could potentially be any one of these https://mimeapplication.net/text-csv
I have already taken this into account and realized)
The only UX question left is that when selecting a file, files of the desired type are available in the file manager
For example, to upload a js or csv file, you will also have to add a text/plain type, which can accommodate many other files with a different extension
I think it’s just possible to specify these types separately for the frontend - that would solve the issue
Uploading video
https://youtu.be/iBOoWgxfYfY
Will have to watch this when I’m at the computer on a bigger screen.
I am certainly interested though. Don’t get me wrong. As the maintainer of a media library package it does matter to me. Cheers.
I think it can still be called a full-fledged bug because of the problem of frontend inconsistency with the backend
For example with a js file:
'text/javascript', 'text/plain' is required for successful loading
Then on the frontend the check will pass only because of the presence of 'text/javascript', and 'text/plain' will be redundant
And on the backend the check will be passed only because of the presence of 'text/plain', while 'text/javascript' will not play a role, because on the unix server the file utility will give the mime type 'text/plain'.
To test it, use
file -b --mime index.js.
That is, two validation rules are specified in the common list - one for the frontend and one for the backend
In this case, both rules will hit both frontend and backend together
I think it would be right to separate these rules for backend and frontend.
Otherwise the working case gives the possibility to upload files on the frontend, which will not pass the check on the backend, when saving a form, while moving a file from a temporary directory
In fact, in this case the file on the server gets to the private storage of the temporary directory, but as already said, does not pass validation on form submissionFair, but I wouldn’t say it’s a filament bug.
In all things upload related filament defaults to livewire which defaults to laravel.
Why would frontend and backend validation be different?
But the solution may well lie on the side of the interface for interacting with FileUpload in Filament
If it is possible to separate the frontend and backend validation rules
Unless, of course, livewire imposes restrictions, using mimetypes from the input html attribute of the element on the backend
Frontend checks mimetype only by its extension
And on the backend the verification mechanism is more complicated, I think it’s similar to the command
file -b --mime index.jsThat still doesn’t make sense, the validation is what it is, front end and backend should be the same. Otherwise it’s a security issue.
In fact, by the method of exclusion and logic, I came to exactly this conclusion, since with the mimetype text/javascript - the frontend successfully receives and saves the file to temporary storage
And when submitting a form, the backend already throws an error if the mimetype text/plain is missing
Fair and vice versa
The frontend will not accept a file without text/javascript
But I’m almost sure that the backend will accept a file with text/plain if you force it to be sent via api, for example via postman
Then it’s a conflict of validity between filepond and the server.
Sounds like filepond says it’s valid so the file file gets uploaded to the livewire tmp directory but on save there’s an issue with laravel or livewire’s mimetyoe inference.
Which prevents the persistent storage to the actual disk.
Oh, this is great progress in the issue discussion
I didn’t know that filepond was used, I’ll look into it in more detail
Yea. Filament uses filepond for the file uploads.
Then this is unlikely to be a filament php problem)
And from the filament side, it can only be solved by dividing the validation logic into backend and frontend
But as you noticed, this is not a consistent solution and can open up new vulnerabilities 🤯
Yea. I don’t personally think it it’s a filament issue.
I'm sleepy now too 😄
Get some rest. Feel free to reach out with more details.
Yes, the third day of investigation is coming)
Thanks for helpin
Keep me in the loop.
If you have a reproduction repo I’d be happy to investigate further.
I still don't think this is a filepond problem
The documentation says that it checks the file type in two steps: by setting the basic html accept attribute on the html5 input element
https://pqina.nl/filepond/docs/api/plugins/file-validate-type/
This way the available files are filtered in the file manager
For example, with
['text/javascript'] - only files with the .js extension will be available
And with ['text/javascript', 'text/plain'] - csv files will also be available, since the browser also considers them files of the text/plain type
Further it is written in the documentation that if the browser does not correctly determine the file type, it additionally double-checks the type of the selected file based on its contents
This explains why with ['text/javascript', 'text/plain'] and a csv file selected, an error appears regarding the file type even before it is loadedEasy File Uploading With JavaScript | FilePond
A JavaScript library that can upload anything you throw at it, optimizes images for faster uploads, and offers a great, accessible, silky smooth user experience.
I downloaded the filepond source code, found an implementation for checking the file type based on its contents, and made a small testbed on playcode.io
https://playcode.io/1905242
Using FileReader, it reads the contents of the file in DataURI format, which at the beginning contains the mimetype that was obtained based on the contents of the file
And at this stage, the received type is compared with the passed list for validation, and the resulting text/csv is missing in
['text/javascript', 'text/plain'], as a result an error occurs even before loading into the temporary livewire directory beginsPlayCode.io
TypeScript Playground
Try this online TypeScript Playground playground with instant live preview and console. Easy & Fast. Experiment yourself.
But at the same time, if only
['text/javascript'] is specified in the validation, then the browser will allow you to select a js file, and it will pass content validation, as a result of which the download to the temporary livewire directory begins.
But already on the livewire side the file will be saved with an extension corresponding to its mimetype, which is determined by laravel
The reason for this is described here: https://securinglaravel.com/laravel-security-file-upload-vulnerability/#1-don’t-trust-user-input
The type is defined as text/plain
I assume with 85% confidence that the check occurs using the "file" system utility, using the command file -b --mime index.js
Which produces text/plain
But validation does not occur at this stage, since the file is loaded into a private temporary storage, in which, under normal conditions, it cannot be executed by anyoneSecuring Laravel
Laravel Security: File Upload Vulnerability
Explaining that Laravel Image File Upload Vulnerability...
When the loading is complete, the submit button for the form becomes available.
When you click it, laravel will check the mimetype according to the specified rule
Then
file -b --mime index.js will output text/plain, but it will not match ['text/javascript'], and will display an error from the backend, under the FileUpload field widget
This is where the inconsistency lies not in libraries or plugins, but in the logic of the browser and the operating system.
And it is filementphp that transmits the specified rules to filepond and laravel, which use them during validation
Accordingly, for different logics, it should be possible to differentiate the rules
I believe that by default everything should be as is, and from acceptedFileTypes the rules should be passed to laravel and filepond "as is", without changes
But the acceptedBrowserFileTypes method must be added, which will overwrite the rules passed to filepond
In this case, I will write the following code:
At first, the browser will allow you to select only files with the .js extension
Then filepond will check if the mimetype of the file specified in its extension matches the passed ['text/javascript']
Next, the file will be uploaded to the temporary livewire directory with the extension .txt
Then, when submitting the form, the file will be double-checked by mimetype by Laravel itself using ['text/plain'] and make sure once again that it is a text file
Then it will be renamed, in accordance with the callback I passed to getUploadedFileNameForStorageUsing
And will be saved with its real extension
The same is true for a large mass of other file types, I wrote about them above, and over the past two days I have also seen them in many discussions, there were .crt, .csv, .pen, .p12, .js and many other files
And such files are defined not only in the text/plain direction, but also binary ones, and media files sometimes do not match the mimetype on the frontend and backend
This is a kind of bug report it worked out
I think I was right in inadvertently thinking about the possibility of separating the rules for the frontend and backend
But didn't describe it in enough detail
I'll downloading source code of filemantphp to research posiibility of this changesJeez. At first glance I think you’re still equating the extension to a mimetyoe, when neither are actually connected in any way.
Extensions are just a string read from the file path. Mimetypes are inferred from the actual splinfo for the file. They aren’t comparable.
I understand that these are different things, but still the browser and laravel define it differently
And also the system utility determines the type in the same as laravel
It is for this reason that acceptedFileTypes must contain both text/javascript and text/plain
Because the browser and filepond will detect the file as text/javascript, and laravel as text/plain
Perhaps I made a mistake in my stream of consciousness, somewhere equating them
Still doesn’t sound like a filament issue.
I want to help, but i can’t change larvael or the server.
plus I use a translator, since I don’t know English perfectly, and it would take much more time to express this in English right away)
Maybe the translator contributed to the interpretation😄
Maybe.
Just seems like a problem higher up the chain with either livewire or laravel.
anyway laravel and browser define mime type differently
https://youtu.be/PiQvvIbSUeI
That's right, filamentphp sends validation rules directly to filepond, which are passed to acceptedFileTypes "as is"
And it has the ability to transfer individual rules to it, duplicating the logic of a separate method and saving it in a separate attribute
And pass the rules to filepond in the form
acceptedBrowserFileTypes ?? acceptedFileTypes
but in the filament source code itself, a solution is used that enumerates mime types, among which there is also text/plain for csv
Although for csv in general this is quite understandable, since the file can contain one word or a space delimiter
On the other hand, I’m a little distracted from the question and now I’m thinking that a js file can be defined as text/plain by mimetype, in case of obfuscation
For example, I have already made an obfuscator that cannot be decrypted by available online decryptors, then you can only identify it by extension, blindly trusting
In this case, this could be true, but the browser will consider such a file valid if it has a .js extension
I'll check this hypothesis now
I think I won’t return to this question in the near future, but will simply accept it as a fact and feature
And in my free time I’ll come back and think about whether the ability to separate validation is needed
https://youtu.be/GGwATl3ZSQY
https://youtu.be/XpHHJ2bQHNY
here's the reasons to think that the browser determines the mime type by the file extension
Laravel and
file -b --mime will define it as text/plainWell that seems really inconvenient. 🤔
Lurking this, as I am experiencing the similar if not same issues. Upload to livewire_tmp works, then hit save and bombs out. I've tested on local and Cloudflare S3 bucket with same result. Created another Import CSV function and seems the similar issue, error says that the attachment must be type: text/csv yet I have set
->form([
FileUpload::make('attachment')
->acceptedFileTypes(['text/csv']),
]) I must add, everything works perfect in Herd when using local or Cloudflare S3 bucket. This issue has really thrown me for a loop!