Filament•5d ago

Filamentphp Export Data Integrity

I tried exporting my model with 37k~ records.. unfortunately their values are not the same compared to the database but if I will export them partially(export by type in the model) and try to combine the data the values are correct.. If you are going to ask me what is inside the exporter, well it is a basic exporter.. no fancy query.. so what is the reason for this?

20 Replies

toeknee•5d ago

Please check your version of Filamentphp

EskieOP•5d ago

i am currently using v3.2.40

toeknee•5d ago

Please upgrade

EskieOP•4d ago

thanks for the advice.. ill take that it didn't work.. here is my exporter

class EarningExporter extends Exporter
{
    protected static ?string $model = Earning::class;

    public static function getColumns(): array
    {
        return [
            ExportColumn::make('id')
                ->label('ID'),
            ExportColumn::make('user_id'),
            ExportColumn::make('user.full_name'),
            ExportColumn::make('earningCutOff.name'),
            ExportColumn::make('earnable_id'),
            ExportColumn::make('earnable.full_name')
                ->label('Reference'),
            ExportColumn::make('reference_package_name'),
            ExportColumn::make('amount'),
            ExportColumn::make('type')
                ->getStateUsing(fn ($record) => $record->type->getLabel()),
            ExportColumn::make('points'),
            ExportColumn::make('status')
                ->getStateUsing(fn ($record) => $record->status->getLabel()),
            ExportColumn::make('level'),
            ExportColumn::make('earned_at'),
            ExportColumn::make('created_at'),
            ExportColumn::make('updated_at'),
        ];
    }

class EarningExporter extends Exporter
{
    protected static ?string $model = Earning::class;

    public static function getColumns(): array
    {
        return [
            ExportColumn::make('id')
                ->label('ID'),
            ExportColumn::make('user_id'),
            ExportColumn::make('user.full_name'),
            ExportColumn::make('earningCutOff.name'),
            ExportColumn::make('earnable_id'),
            ExportColumn::make('earnable.full_name')
                ->label('Reference'),
            ExportColumn::make('reference_package_name'),
            ExportColumn::make('amount'),
            ExportColumn::make('type')
                ->getStateUsing(fn ($record) => $record->type->getLabel()),
            ExportColumn::make('points'),
            ExportColumn::make('status')
                ->getStateUsing(fn ($record) => $record->status->getLabel()),
            ExportColumn::make('level'),
            ExportColumn::make('earned_at'),
            ExportColumn::make('created_at'),
            ExportColumn::make('updated_at'),
        ];
    }

EskieOP•4d ago

as you can see.. same count but the sum is different.. i already updated my filament to v3.2.124

EskieOP•4d ago

here is my table

EskieOP•4d ago

hope you will investigate this matter.. thank you

toeknee•4d ago

How is amount stored? if you count the records is there a difference? Check the export queries too. you may be excluding trashed on export but not on the resource

Dennis Koch•4d ago

And you checked that this isn't due to precision? The count is the same.

toeknee•4d ago

For examount, if you are using decimal/string and not integers you'll end up with precision point issues PHP is known for. As per filamentphp docs store as cents in integers.

EskieOP•4d ago

not sure because i checked manually the csv like.. example.. I exported two files(one partially exported based on type. ex: Subscription Bonus) and the other one is the whole record... inside the file with the whole record, i removed those data that isn't Subscription Bonus Type and compare to the exported.. counts are same but total amount isn't

toeknee•4d ago

csv's count differently to the export. how are you storing the amounts... so the export/import is correct your storing of data isn't.

EskieOP•4d ago

i am using float for the amount

toeknee•4d ago

https://blog.codeminer42.com/be-cool-dont-use-float-double-for-storing-monetary-values/

Edigleysson Silva

The Miners

Be cool. Don’t use float/double for storing monetary values A tour ...

In this article, we're going to dissect the classical floating point expression confusion 0.1 + 0.2 == 0.3, walking through important concepts and details.

EskieOP•4d ago

yea but just like what ive said.. if i export them partially, it works...

toeknee•4d ago

Diff compare the import to the export and find the culprit

Dennis Koch•4d ago

Can you find which records have different amount compared to DB? Do you have records with a precision higher than 3. Like 12.314

EskieOP•4d ago

i tried but the more i look it up the more it messes up.. maybe i will send a vid i only store 2 decimals

Dennis Koch•4d ago

the more it messes up.

What is messed upß

EskieOP•4d ago

i tried filtering it by type but it seems that the count isn't the same.. ill make a video of these issues in export

Gaming

Programming

Filamentphp Export Data Integrity