Filament•2mo ago

Filamentphp Export Data Integrity

I tried exporting my model with 37k~ records.. unfortunately their values are not the same compared to the database but if I will export them partially(export by type in the model) and try to combine the data the values are correct.. If you are going to ask me what is inside the exporter, well it is a basic exporter.. no fancy query.. so what is the reason for this?

20 Replies

toeknee•2mo ago

Please check your version of Filamentphp

EskieOP•2mo ago

i am currently using v3.2.40

toeknee•2mo ago

Please upgrade

EskieOP•2mo ago

thanks for the advice.. ill take that it didn't work.. here is my exporter

class EarningExporter extends Exporter
{
    protected static ?string $model = Earning::class;

    public static function getColumns(): array
    {
        return [
            ExportColumn::make('id')
                ->label('ID'),
            ExportColumn::make('user_id'),
            ExportColumn::make('user.full_name'),
            ExportColumn::make('earningCutOff.name'),
            ExportColumn::make('earnable_id'),
            ExportColumn::make('earnable.full_name')
                ->label('Reference'),
            ExportColumn::make('reference_package_name'),
            ExportColumn::make('amount'),
            ExportColumn::make('type')
                ->getStateUsing(fn ($record) => $record->type->getLabel()),
            ExportColumn::make('points'),
            ExportColumn::make('status')
                ->getStateUsing(fn ($record) => $record->status->getLabel()),
            ExportColumn::make('level'),
            ExportColumn::make('earned_at'),
            ExportColumn::make('created_at'),
            ExportColumn::make('updated_at'),
        ];
    }

class EarningExporter extends Exporter
{
    protected static ?string $model = Earning::class;

    public static function getColumns(): array
    {
        return [
            ExportColumn::make('id')
                ->label('ID'),
            ExportColumn::make('user_id'),
            ExportColumn::make('user.full_name'),
            ExportColumn::make('earningCutOff.name'),
            ExportColumn::make('earnable_id'),
            ExportColumn::make('earnable.full_name')
                ->label('Reference'),
            ExportColumn::make('reference_package_name'),
            ExportColumn::make('amount'),
            ExportColumn::make('type')
                ->getStateUsing(fn ($record) => $record->type->getLabel()),
            ExportColumn::make('points'),
            ExportColumn::make('status')
                ->getStateUsing(fn ($record) => $record->status->getLabel()),
            ExportColumn::make('level'),
            ExportColumn::make('earned_at'),
            ExportColumn::make('created_at'),
            ExportColumn::make('updated_at'),
        ];
    }

EskieOP•2mo ago

as you can see.. same count but the sum is different.. i already updated my filament to v3.2.124

EskieOP•2mo ago

here is my table

EskieOP•2mo ago

hope you will investigate this matter.. thank you

toeknee•2mo ago

How is amount stored? if you count the records is there a difference? Check the export queries too. you may be excluding trashed on export but not on the resource

Dennis Koch•2mo ago

And you checked that this isn't due to precision? The count is the same.

toeknee•2mo ago

For examount, if you are using decimal/string and not integers you'll end up with precision point issues PHP is known for. As per filamentphp docs store as cents in integers.

EskieOP•2mo ago

not sure because i checked manually the csv like.. example.. I exported two files(one partially exported based on type. ex: Subscription Bonus) and the other one is the whole record... inside the file with the whole record, i removed those data that isn't Subscription Bonus Type and compare to the exported.. counts are same but total amount isn't

toeknee•2mo ago

csv's count differently to the export. how are you storing the amounts... so the export/import is correct your storing of data isn't.

EskieOP•2mo ago

i am using float for the amount

toeknee•2mo ago

https://blog.codeminer42.com/be-cool-dont-use-float-double-for-storing-monetary-values/

Edigleysson Silva

The Miners

Be cool. Don’t use float/double for storing monetary values A tour ...

In this article, we're going to dissect the classical floating point expression confusion 0.1 + 0.2 == 0.3, walking through important concepts and details.

EskieOP•2mo ago

yea but just like what ive said.. if i export them partially, it works...

toeknee•2mo ago

Diff compare the import to the export and find the culprit

Dennis Koch•2mo ago

Can you find which records have different amount compared to DB? Do you have records with a precision higher than 3. Like 12.314

EskieOP•2mo ago

i tried but the more i look it up the more it messes up.. maybe i will send a vid i only store 2 decimals

Dennis Koch•2mo ago

the more it messes up.

What is messed upß

EskieOP•2mo ago

i tried filtering it by type but it seems that the count isn't the same.. ill make a video of these issues in export

Gaming

Programming

Filamentphp Export Data Integrity

Did you find this page helpful?