Filament•2mo ago

Problem with Import Action - Import job never completes

I am having a problem with ImportAction. I want to use it to import products from CSV file. I am testing it with CSV file that has two rows in total. But when I run the action, job keep executing endlessly over and over, spamming failed_import_rows table. I am not sure why is it doing so. If it fails to import the records is should end the job in the end. I also created custom App\Jobs\ImportCsv job class that extends Filament\Actions\Imports\Jobs\ImportCsv job class and added property public $tries = 1; but that doesn't work either. If I inspect jobs table in the database I can see that the number of attempts is ridiculesly high, e.g. 15346. Then I need to stop the worker, remove the job from jobs table and clear the failed_import_rows table. This is how I resolve records in my importer class:

public function resolveRecord(): ?Product
    {
        if (! array_key_exists('ID', $this->data) || blank($this->data['ID'])) {
            return new Product();
        }

        return Product::firstOrNew(['id' => $this->data['ID']]);
    }

public function resolveRecord(): ?Product
    {
        if (! array_key_exists('ID', $this->data) || blank($this->data['ID'])) {
            return new Product();
        }

        return Product::firstOrNew(['id' => $this->data['ID']]);
    }

Other then that I just have a bunch of columns defined on the Importer and using afterSave() hook to perform product stock update, as I keep stocks in separate table. In the fail_import_rows table validation_error column is always NULL.

7 Replies

tjodalv•2mo ago

Can anyone help me with why the queued import job never completes?

nathan•2mo ago

Check your laravel log file. The job is dispatched and throwing exceptions. It is then endlessly retried. Mine did the same. I use horizon. I would see the import job pop up and begin running. Then, when I saw it not progressing, I killed the horizon process. I checked the laravel.log file for the exception that was being thrown. Then, I cleared the redis queue php artisan queue:clear {connection} --queue {queue_name}. This removed the pending jobs. I made the needed changes based on the exceptions and reran. This could be done better. We need a way to specify how many attempts to try before the batch fails so we can troubleshoot effectively. However, that's how I worked through my import problems.

tjodalv•2mo ago

thanks for sharing your solution. I did manage to find what was causing import to fail, but still I don't know why the job didn't fail, but instead it kept running over and over again. I had the problem with UniqueConstraint exception.

toeknee•2mo ago

You probably need to try and catch and throw an error

tjodalv•2mo ago

Where should I put try catch exactly? I am using standard ImportAction, I've maped columns like the docs say. The problem was that one of the field had unique constraint and my csv file have record with the value that already exists in the database. But that is not a problem really. The problem is that ImportCsv job keep executing endlessly and never completes of fails. That result millions of records in failed_import_rows table in a few minutes. So the question is why job doesn't fail when the exception is thrown?

lazydog•5w ago

Have you resolved this one? I encounter this issue too. I have 21 rows and only 11rows were processed. No error or error log. In my log I just see the broadcast notification

tjodalv•4w ago

I didn't resolve this problem. The solution for now is to remove unique constraint from that field in the database. Also if you have some foreign key constraints that fails during import the job will keep runing forever until you kill the worker and remove the job from the database.

Gaming

Programming

Problem with Import Action - Import job never completes