Filament•7mo ago

FileUpload bug - Some files a are missing after uploaded.

When I try to upload multiple files, after the uploading couple of files are missing. this incident is absolutely random. In the below picture I have tried to upload 5 files in two sections. All are same files. In applicationFiles(top one) all 5 files were uploaded, but in the result_files section one file is missing. I have also attache the screen shot of the folder section. The leftover file is in livewire-tmp folder.

47 Replies

danilo•4mo ago

Hi, I've got the same problem, I've an action for the multiple upload, and when I put the debugger in the foreach in the code below, the $data array already is missing some files...

 ->action(function (array $data, $record) {
                            Notification::make()
                                ->title(__('The images are being transformed, soon you will be able to see them'))
                                ->info()
                                ->send();
                            foreach ($data['images'] as $fileName){
                                ImageService::handleImageUpload($fileName, $record);
                            } 
})

 ->action(function (array $data, $record) {
                            Notification::make()
                                ->title(__('The images are being transformed, soon you will be able to see them'))
                                ->info()
                                ->send();
                            foreach ($data['images'] as $fileName){
                                ImageService::handleImageUpload($fileName, $record);
                            } 
})

Also no errors are reported in the storage/log/laravel.log file

Matthew•4mo ago

Anything in the filenames that could be confusing the array into thinking one complete filename is actually 2 or more (incomplete) filenames.? Is it always the same files missing ?

danilo•4mo ago

Hi, the files have all different names, and the missing ones changes from one try to the other, I'm attaching a couple screenshots in this case is the file with name .._06... missing

danilo•4mo ago

*sorry ..._04... is missing

danilo•4mo ago

I've re-ran it and now the ..._00.. is missing

danilo•4mo ago

other times more than one file is missing, sometimes all files are uploaded...

jouni.i•4mo ago

how about if you try to upload about 4 files same time ? still missing files ?

danilo•4mo ago

Hi, I've just tried uploading only 4 files, the first two times it uploaded all of them, the third time one was missing

jouni.i•4mo ago

you use valet or something else ?

danilo•4mo ago

this is with sail on linux, but the same happens to other devs on windows and on the staging server, running linux with PHP-fpm behind nginx (in docker)

jouni.i•4mo ago

php-fpm log not show any error ?

danilo•4mo ago

I haven't checked on the server (but it's in production as far as APP_ENV is concerned), but on my PC it appears the missing files are never received within the array of uploaded $data in the action, I'll expand the code below to have a better context, but with the debugger, in the foreach line $data is already missing the files

return $form
            ->schema([
                ...self::getFormSchema(),
                Actions::make([
                    Action::make('mass-upload-images')
                        ->stickyModalHeader()
                        ->stickyModalFooter()
                        ->icon('heroicon-m-document-arrow-up')
                        ->form([
                            FileUpload::make('images')
                                ->multiple()
                                ->directory(function ($record) {
                                    return  $record->id . '/' . Carbon::now()->format('Y.m.d_H.m.s');
                                })
                                ->preserveFilenames()
                        ])
                        ->action(function (array $data, $record) {
                            Notification::make()
                                ->title(__('The images are being transformed, soon you will be able to see them'))
                                ->info()
                                ->send();
                            foreach ($data['images'] as $fileName){
                                ImageService::handleImageUpload($fileName, $record);
                            }
                            
                        })

return $form
            ->schema([
                ...self::getFormSchema(),
                Actions::make([
                    Action::make('mass-upload-images')
                        ->stickyModalHeader()
                        ->stickyModalFooter()
                        ->icon('heroicon-m-document-arrow-up')
                        ->form([
                            FileUpload::make('images')
                                ->multiple()
                                ->directory(function ($record) {
                                    return  $record->id . '/' . Carbon::now()->format('Y.m.d_H.m.s');
                                })
                                ->preserveFilenames()
                        ])
                        ->action(function (array $data, $record) {
                            Notification::make()
                                ->title(__('The images are being transformed, soon you will be able to see them'))
                                ->info()
                                ->send();
                            foreach ($data['images'] as $fileName){
                                ImageService::handleImageUpload($fileName, $record);
                            }
                            
                        })

I'll try on the server now

jouni.i•4mo ago

if you're doing a four-file transfer, for example, so. are you able to track which file drops out of them and is it the last file whose upload is complete

danilo•4mo ago

As these example files have 00, 01, 02, ... in their filename, in the upload modal it processes two of them at a time, starting from the first two. When they are at 100% it starts with the next two and so on. They are in alphabetical order, so no it's not always the last file to be uploaded that is skipped. Also, with multiple uploads of the same files the missing files changes between tries

jouni.i•4mo ago

that i mean.. file upload size or request time out possible php issue then that is randomized but that issue found in php-fpm log

danilo•4mo ago

(Oh, ok, maybe I need to check in the broser develop tabs then, I'll check it) meanwhile on the server I've run a few tests, the first two uploads worked with all files there, the third one was missing a file but there isn't any log on the php-fpm side, nor in the storage/log/laravel.log file

jouni.i•4mo ago

->action(function (array $data, $record) { Notification::make() ->title(__('The images are being transformed, soon you will be able to see them')) ->info() ->send(); foreach ($data['images'] as $fileName){ ImageService::handleImageUpload($fileName, $record); }
print me dd($data) before notification

danilo•4mo ago

array:1 [▼ 
    0 => "2dModel/1/2024.09.09_06.09.22/Ch00_VIS_00_x4.tif"
    1 => "2dModel/1/2024.09.09_06.09.22/Ch02_VIS_02_x4.tif"
    2 => "2dModel/1/2024.09.09_06.09.22/Ch03_VIS_03_x4.tif"
    3 => "2dModel/1/2024.09.09_06.09.22/Ch04_VIS_04_x4.tif"
    4 => "2dModel/1/2024.09.09_06.09.22/Ch06_VIS_06_x4.tif"
  ]
]

array:1 [▼ 
    0 => "2dModel/1/2024.09.09_06.09.22/Ch00_VIS_00_x4.tif"
    1 => "2dModel/1/2024.09.09_06.09.22/Ch02_VIS_02_x4.tif"
    2 => "2dModel/1/2024.09.09_06.09.22/Ch03_VIS_03_x4.tif"
    3 => "2dModel/1/2024.09.09_06.09.22/Ch04_VIS_04_x4.tif"
    4 => "2dModel/1/2024.09.09_06.09.22/Ch06_VIS_06_x4.tif"
  ]
]

Here it's missing 2 files, the 01 and the 05

jouni.i•4mo ago

ok then try files where size is slow example something test png images

danilo•4mo ago

ok, even though I should have set it so that 3.5 Mb shouldn't be a problem, anyway I'm creating the files and testing it, thankds I've created 9 very small (1000x1000 px) images, around 1Kb each, and tried uploading them twice, each time it was missing some files these are all identical files (I've created an image with GIMP and copied it over to all these new files

jouni.i•4mo ago

images found in livewire-tmp folder ? and you use php8.2 or 8.1 ?

danilo•4mo ago

one time it was missing the 02 and the 05, the other one 02 and 06 I'm using php 8.2 in sail let me check if something remains in the livewire-tmp dir, I'm emptying it now and trying uploading the same files again yes the missing files are still in the livewire-tmp dir

jouni.i•4mo ago

can you show screenshot browser dev tools where show upload request... i think you got error upload-file request

danilo•4mo ago

this is when dropping the file in the fileupload zone

danilo•4mo ago

this was a 9-file upload

jouni.i•4mo ago

yeah and 7 complete

danilo•4mo ago

yes I've split the dev bar screenshot in two files, but it was all together in the browser

jouni.i•4mo ago

yeah but i found only 7 upload-file request that move files to livewire-tmp to correct destination folder

danilo•4mo ago

Actually they are 9, two in the first screenshot, 7 in the second one they do not overlap (the screenshots I mean)

jouni.i•4mo ago

true.. hmm livewire version ?

danilo•4mo ago

I've ran composer update last friday I think, anyway let me check yes livewire/livewire 3.5.6

jouni.i•4mo ago

do it this php artisan livewire:publish --assets and then again and php artisan optimize:clear

danilo•4mo ago

ok... I didn't publish the livewire assets before, I'm trying with the assets published (I'm not sure what's the idea behind it, TBH :)) anyway, two files are missing after publishing the assets and optimize:clear no luck... I've searched across the internet and other people have reported similar problems in the past, it seems something in the FileUpload filament component...

jouni.i•4mo ago

i'm not sure this issue is FileUpload because i just test about 10 times and i have work fine no missing files

danilo•4mo ago

jouni.i•4mo ago

hmm your destination is local or s3 ?

danilo•4mo ago

local

hehehaha•4mo ago

try to use ->moveFiles()?

jouni.i•4mo ago

hmm try to add fileupload ->image() that is only difference my code ah hehehaha that is true... i have this too 😄

danilo•4mo ago

I'm trying the two things now, thanks with just ->image() the problem persists (but once I'm there, I'll leave it in since these must be images anyway :)) even with both moveFiles and image the problem remains... I'll need to try it out in a new project to isolate it at this point... I've found the problem... the resource has a relationManager with the Resources that are created when uploading a file (in the foreach in the code above, we create a new instance of a Model, and connect the file with that Model). If the table method in the relation manager has a ->poll('5s') (for instance) then the problem arises!

danilo•4mo ago

I've created a test project https://github.com/danilogiacomi/filamentFileUploadProblem/

GitHub

GitHub - danilogiacomi/filamentFileUploadProblem

Contribute to danilogiacomi/filamentFileUploadProblem development by creating an account on GitHub.

danilo•4mo ago

if you use it as it is, you will (randomly) find the problem discussed here whereas if (in app/Filament/Resources/OrderResource/RelationManagers/FilesRelationManager.php at line 49) you comment out the ->poll('5s') line, then the problem isn't occurring anymore it's crazy, I'll try to contact the filament devs

jouni.i•4mo ago

ah i'm never use ->pool() a completely useless feature in my opinion, it causes unnecessary queries that strain the server

danilo•4mo ago

you are probably right, but in this case it's perfect as we upload multiple image files which need to be processed (via some jobs) and therefore we can watch them automatically update in the table below our resource. I don't know if there's a better way to do so, in any case I've submitted the issue to the filament github repo (I've actually added a comment, with the link to my test repo, to an existing issue related to missing files from multiple fileUpload)

danilo•4mo ago

https://github.com/filamentphp/filament/issues/13306

GitHub

FileUpload: Some files are missing when uploading multiple files · ...

Package filament/filament Package Version 3 Laravel Version 11 Livewire Version V3 PHP Version php 8 Problem description When I upload multiple files, some of the files are missing. Expected behavi...

jouni.i•4mo ago

can you test if you increase pool example 15sec ? how about then still files lost ?

danilo•4mo ago

Hi I've just tried it and still it was missing a couple of files, out of the 32 I've uploaded, feel free to tinker around with the code in the repo, the README should have enough info for testing the whole thing

Gaming

Programming

FileUpload bug - Some files a are missing after uploaded.