Filament•12mo ago

Non comma Separated CSV Import

I'm having issues with CSV imports, when the csv data is separated by semicolons. The columns are not recognised correctly. When i save the CSV with comma separators it works perfectly fine. Can I somehow change the separator settings in Import?

16 Replies

igorclaussOP•12mo ago

Still searching for a solution squint

nobidium•12mo ago

Have you tried casting the data before passing it to rules?

igorclaussOP•12mo ago

I did. My issue is that the importer don't even recognizes the headers. It shows all of them in a list with semicolons.

DrByte•12mo ago

I don't think Filament currently has any interface for passing the delimiter. https://csv.thephpleague.com/9.0/connections/controls/#the-delimiter-character https://github.com/filamentphp/filament/blob/ff6e64e09bde4b79426fff2da092f540fdd8e8b9/packages/actions/src/Concerns/CanImportRecords.php#L85-L88

Csv character controls - CSV

Working with CSV can often be much more difficult than you expect, with different types of delimiters and complicated structures. This library makes it easy to read and write almost anything.

GitHub

filament/packages/actions/src/Concerns/CanImportRecords.php at ff6e...

A collection of beautiful full-stack components for Laravel. The perfect starting point for your next app. Using Livewire, Alpine.js and Tailwind CSS. - filamentphp/filament

igorclaussOP•12mo ago

Then it's time to create a pull request.

DrByte•12mo ago

Excellent. I noticed that there are 3 places in that class where it would need to be implemented, for the 3 different ways the importer gets instantiated

igorclaussOP•12mo ago

I failed on a dynamic solution. I ended up overwriting the CanImportRecords trait and created my own ImportAction. Within first I changed the code on those three places by adding ->setDelimiter(';') calls. It works for me now. Tough I dislike this solution a little bit.

DrByte•12mo ago

Here's my proposed changes:

<?php
    protected string | Closure | null $csvDelimiter = null;

--------
// 3 instances of this for Readers: 
  
                    if (filled($delimiter = $this->getDelimiter())) {
                        $csvReader->setDelimiter($delimiter);
                    }

// 1 of this, for Writer:

                    if (filled($delimiter = $this->getDelimiter())) {
                        $csv->setDelimiter($delimiter);
                    }

---
// to allow calling ->delimiter(';') for semicolon (for example)

    public function delimiter(string | Closure | null $delimiter): static
    {
        $this->csvDelimiter = $delimiter;

        return $this;
    }

// and internal parser for it:

    public function getDelimiter(): ?string
    {
        return $this->evaluate($this->csvDelimiter);
    }

<?php
    protected string | Closure | null $csvDelimiter = null;

--------
// 3 instances of this for Readers: 
  
                    if (filled($delimiter = $this->getDelimiter())) {
                        $csvReader->setDelimiter($delimiter);
                    }

// 1 of this, for Writer:

                    if (filled($delimiter = $this->getDelimiter())) {
                        $csv->setDelimiter($delimiter);
                    }

---
// to allow calling ->delimiter(';') for semicolon (for example)

    public function delimiter(string | Closure | null $delimiter): static
    {
        $this->csvDelimiter = $delimiter;

        return $this;
    }

// and internal parser for it:

    public function getDelimiter(): ?string
    {
        return $this->evaluate($this->csvDelimiter);
    }

DrByte•12mo ago

Here's a gist to test it with. Curious your feedback @igorclauss https://gist.github.com/drbyte/c944c3acda2bafb364a857ac6e2c8ce6

Gist

draft: filament csv import delimiter

draft: filament csv import delimiter. GitHub Gist: instantly share code, notes, and snippets.

DrByte•12mo ago

To use: pass ->delimiter(';') to your importer

igorclaussOP•12mo ago

Awesome, 👍

DrByte•12mo ago

same as you would for maxRows or other params

igorclaussOP•12mo ago

That's how I fixed it for me. But I did it in the trait itself I thought there must be a way to guess the delimiter. Because we have the csv file...

DrByte•12mo ago

Ya I read somewhere that it may try to guess it on its own, but I couldn't see "where" it did that. So, figure, if someone "needs" to specify it, this will let them. ... without having to clone the entire trait and edit it 🙂

DrByte•12mo ago

PR submitted. Feel free to add your test results in comments: https://github.com/filamentphp/filament/pull/10176

GitHub

Add ability to specify ->delimiter(char) for CSV separator by drbyt...

Ref: League CSV docs Ref: Filament Discord discussion Changes have been thoroughly tested to not break existing functionality. New functionality has been documented or existing documentation has...

igorclaussOP•12mo ago

I haven't found the guessing either. But League csv definitely has a method for that, haven't they? I found this solution:

$csvReader = CsvReader::createFromStream($csvStream);
$delimiter = $this->guessDelimiter($csvReader);
$csvReader->setDelimiter($delimiter);



/**
* @return string
*/
public function guessDelimiter(CsvReader $csvReader, array $set = [], int $limit = 20): string
{
    if (empty($set)) {
        $set = [';', ',', '|', "\t"];
    }

   $delimiterCounts = Info::getDelimiterStats($csvReader, $set, $limit);

   // get the delimiter by the highest count of appearances
   return array_search(max($delimiterCounts), $delimiterCounts);
}

$csvReader = CsvReader::createFromStream($csvStream);
$delimiter = $this->guessDelimiter($csvReader);
$csvReader->setDelimiter($delimiter);



/**
* @return string
*/
public function guessDelimiter(CsvReader $csvReader, array $set = [], int $limit = 20): string
{
    if (empty($set)) {
        $set = [';', ',', '|', "\t"];
    }

   $delimiterCounts = Info::getDelimiterStats($csvReader, $set, $limit);

   // get the delimiter by the highest count of appearances
   return array_search(max($delimiterCounts), $delimiterCounts);
}

It was merged right next to your solution. Great work and thanks for the inspiration.

Gaming

Programming

Non comma Separated CSV Import