Java Community | Help. Code. Learn.•3w ago

Populate Spring Data JPA before each @Test

I want to test my Products repository (or other tests that requires database to be populated first). I have tried using ApplicationListener<ApplicationReadyEvent> previously but even though (in the testing environment), the populating methods seem to have been run, the .save methods called by them returns null for each data creation when populating. My goal is to have the database bootstrapped with the data I have parsed from my CSV parser so it can be used as a Test resource. I have tried so many things, but yet none of them automatic populators work, for it to work, I have to manually populate the data explicitly before tests in the test environment. The DataJpaTest:

@Slf4j
@DataJpaTest
public class ProductRepositoryTest {

    @Autowired
    private ProductRepository productRepository;

    @MockBean
    private ProductService productService;

    @MockBean
    private ProductCsvImportServiceImpl productCsvImportService;

    @BeforeEach
    void setUp() {
        productRepository.deleteAll();
        productRepository.flush();

        CSVParser csvParser = new CSVParser(productService, productCsvImportService);
        csvParser.saveProductsFromCsv("seeders/ProductsSeeder.csv");
    }

    @Test
    void testFindAll() {
        List<Product> productList = productRepository.findAll();

        // Assertions
        assertThat(productList.size()).isEqualTo(45);
    }

@Slf4j
@DataJpaTest
public class ProductRepositoryTest {

    @Autowired
    private ProductRepository productRepository;

    @MockBean
    private ProductService productService;

    @MockBean
    private ProductCsvImportServiceImpl productCsvImportService;

    @BeforeEach
    void setUp() {
        productRepository.deleteAll();
        productRepository.flush();

        CSVParser csvParser = new CSVParser(productService, productCsvImportService);
        csvParser.saveProductsFromCsv("seeders/ProductsSeeder.csv");
    }

    @Test
    void testFindAll() {
        List<Product> productList = productRepository.findAll();

        // Assertions
        assertThat(productList.size()).isEqualTo(45);
    }

21 Replies

JavaBot•3w ago

⌛ This post has been reserved for your question.

Hey @circle! Please use /close or the Close Post button above when your problem is solved. Please remember to follow the help guidelines. This post will be automatically marked as dormant after 300 minutes of inactivity.

TIP: Narrow down your issue to simple and precise questions to maximize the chance that others will reply in here.

circleOP•3w ago

CSVParser:

@Component
@Slf4j
@RequiredArgsConstructor
public class CSVParser {

    private final ProductService productService;
    private final ProductCsvImportService productCsvImportService;

    public void saveProductsFromCsv(String path) {
        try {
            ClassPathResource resource = new ClassPathResource(path);
            File file = resource.getFile();

            try (FileInputStream fis = new FileInputStream(file)) {
                MultipartFile multipartFile = new MockMultipartFile(
                    "file",
                    file.getName(),
                    "text/csv",
                    fis
                );
                productCsvImportService.importProductsFromCsv(multipartFile);
            }
        } catch (IOException e) {
            throw new RuntimeException(e);
        }

        log.info("Injected {} products", productService.getAllProducts(null).size());
    }
}

@Component
@Slf4j
@RequiredArgsConstructor
public class CSVParser {

    private final ProductService productService;
    private final ProductCsvImportService productCsvImportService;

    public void saveProductsFromCsv(String path) {
        try {
            ClassPathResource resource = new ClassPathResource(path);
            File file = resource.getFile();

            try (FileInputStream fis = new FileInputStream(file)) {
                MultipartFile multipartFile = new MockMultipartFile(
                    "file",
                    file.getName(),
                    "text/csv",
                    fis
                );
                productCsvImportService.importProductsFromCsv(multipartFile);
            }
        } catch (IOException e) {
            throw new RuntimeException(e);
        }

        log.info("Injected {} products", productService.getAllProducts(null).size());
    }
}

Service layer being called to import:

@Service
@RequiredArgsConstructor
@Slf4j
public class ProductCsvImportServiceImpl implements ProductCsvImportService {

    private final int CSV_VARIANT_COLUMNS_COUNT = 3;
    private final int CSV_VARIANT_STARTS_AT = 4;

    private final ProductService productService;
    private final ApplicationProperties applicationProperties;

    @Override
    public void importProductsFromCsv(MultipartFile file) {
        if (!Objects.requireNonNull(file.getOriginalFilename()).substring(file.getOriginalFilename().lastIndexOf(".") + 1).equals("csv")) {
            throw new InvalidRequestBodyValue("Must be a valid CSV file");
        }

        try (BufferedReader br = new BufferedReader(new InputStreamReader(file.getInputStream()))) {
            // Skip the header on row 1
            String line = br.readLine();

            while ((line = br.readLine()) != null) {
                try {
                    ProductPostDto productPostDto = parseLineToProductPostDto(line, null);
                    if (productPostDto != null && !productPostDto.getVariants().isEmpty()) {
                        productService.createProduct(productPostDto);
                    }
                } catch (InvalidRequestBodyValue | ResourceExistsException e) {
                    // Log the error and skip the current line
                    log.warn("Error importing product: " + e.getMessage());
                }
            }
        } catch (IOException e) {
            throw new ResourceUploadException("Error reading the file");
        }
    }

@Service
@RequiredArgsConstructor
@Slf4j
public class ProductCsvImportServiceImpl implements ProductCsvImportService {

    private final int CSV_VARIANT_COLUMNS_COUNT = 3;
    private final int CSV_VARIANT_STARTS_AT = 4;

    private final ProductService productService;
    private final ApplicationProperties applicationProperties;

    @Override
    public void importProductsFromCsv(MultipartFile file) {
        if (!Objects.requireNonNull(file.getOriginalFilename()).substring(file.getOriginalFilename().lastIndexOf(".") + 1).equals("csv")) {
            throw new InvalidRequestBodyValue("Must be a valid CSV file");
        }

        try (BufferedReader br = new BufferedReader(new InputStreamReader(file.getInputStream()))) {
            // Skip the header on row 1
            String line = br.readLine();

            while ((line = br.readLine()) != null) {
                try {
                    ProductPostDto productPostDto = parseLineToProductPostDto(line, null);
                    if (productPostDto != null && !productPostDto.getVariants().isEmpty()) {
                        productService.createProduct(productPostDto);
                    }
                } catch (InvalidRequestBodyValue | ResourceExistsException e) {
                    // Log the error and skip the current line
                    log.warn("Error importing product: " + e.getMessage());
                }
            }
        } catch (IOException e) {
            throw new ResourceUploadException("Error reading the file");
        }
    }

The Service layer used to create the resource:

@Service
@Primary
@RequiredArgsConstructor
@Slf4j
public class ProductServiceImpl implements ProductService {

    private final ProductRepository productRepository;
    private final ProductMapper productMapper;
    private final ImageStorageService imageStorageService;

    @Override
    public ProductDto createProduct(ProductPostDto productPostDto) {
        validateProductPostDto(productPostDto);

        Product product = buildProductFromDto(productPostDto);

        // Convert and return
        Product savedProduct = productRepository.save(product);
        ProductDto productDto = productMapper.fromEntity(savedProduct);
        return productDto;
    }

@Service
@Primary
@RequiredArgsConstructor
@Slf4j
public class ProductServiceImpl implements ProductService {

    private final ProductRepository productRepository;
    private final ProductMapper productMapper;
    private final ImageStorageService imageStorageService;

    @Override
    public ProductDto createProduct(ProductPostDto productPostDto) {
        validateProductPostDto(productPostDto);

        Product product = buildProductFromDto(productPostDto);

        // Convert and return
        Product savedProduct = productRepository.save(product);
        ProductDto productDto = productMapper.fromEntity(savedProduct);
        return productDto;
    }

circleOP•3w ago

The roductCsvImportServiceImpl doesn't seem to be constructed properly? The CSV_VARIANT_COLUMNS_COUNT and CSV_VARIANT_STARTS_AT is even initialized wtih 0 instead of their default values

circleOP•3w ago

There seem to be content inside the passed multipartfile before being passed to importProductsFromCsv.

circleOP•3w ago

Seem to be skipped by the first if else checker.

circleOP•3w ago

Not sure why the whole try catch was skipped I have no light on this

circleOP•3w ago

Here's the branch it's working on https://github.com/vianneynara/kuenyawz-api/blob/feature/89-paginate-products/src/test/java/dev/realtards/kuenyawz/repositories/ProductRepositoryTest.java

GitHub

kuenyawz-api/src/test/java/dev/realtards/kuenyawz/repositories/Prod...

RESTful API server using Spring Boot framework. Contribute to vianneynara/kuenyawz-api development by creating an account on GitHub.

circleOP•3w ago

Originally, I wanted to just @Import the DatabaseBooststrapper like this. To make it clear, you can see that the account service returns null (null came on the account repository's .save method):

    @Override
    public Account createAccount(AccountRegistrationDto accountRegistrationDto) {
        if (accountRepository.existsByEmail(accountRegistrationDto.getEmail())) {
            throw new AccountExistsException();
        }

        Account account = Account.builder()
            .fullName(accountRegistrationDto.getFullName())
            .email(accountRegistrationDto.getEmail().toLowerCase())
            .password(passwordEncoder.encode(accountRegistrationDto.getPassword()))
            .privilege(Account.Privilege.USER)
            .build();

        account = accountRepository.save(account);

        return account;
    }

    @Override
    public Account createAccount(AccountRegistrationDto accountRegistrationDto) {
        if (accountRepository.existsByEmail(accountRegistrationDto.getEmail())) {
            throw new AccountExistsException();
        }

        Account account = Account.builder()
            .fullName(accountRegistrationDto.getFullName())
            .email(accountRegistrationDto.getEmail().toLowerCase())
            .password(passwordEncoder.encode(accountRegistrationDto.getPassword()))
            .privilege(Account.Privilege.USER)
            .build();

        account = accountRepository.save(account);

        return account;
    }

I have made sure that the Account builder above contains information about the current passed AccountRegistrationDto.

circleOP•3w ago

Aside from that, I'm currently modifying my paring method to use the safer option: OpenCSV Urgh

circleOP•3w ago

Now somewhy my OpenCSV breaks 😭

circleOP•3w ago

OOT but might be related to this: https://stackoverflow.com/questions/42328597/opencsv-populating-first-column-as-null, imma save it again

Stack Overflow

OPENCSV populating first column as null

I am trying to read a csv file using opencsv using HeaderColumnNameMappingStrategy. Opencsv populates my first column of csv as null everytime. Here is my code : Entity Class : @Entity @Table(na...

circleOP•3w ago

Oh lol this helped, was exporting from excel, seems like i have to save it without BOM I gave up using DataJpaTest, I'll only use SpringBootTest with H2 from now on literally goign to smash my head

JavaBot•3w ago

💤 Post marked as dormant

This post has been inactive for over 300 minutes, thus, it has been archived. If your question was not answered yet, feel free to re-open this post or create a new one. In case your post is not getting any attention, you can try to use /help ping. Warning: abusing this will result in moderative actions taken against you.

Tomasm21•2w ago

And I have made that it would be possible to test using just @DataJpaTest. Interested? Of course you can launch whole Spring application context by using @SpringBootTest May I send a pull request in github? I created new CSVParser similar to yours and and T21_CSVproductsImporter similar to your ProductCsvImportServiceImpl. I have not made these classes to be beans by adding @Service annotation. And have not used ProductService, ProductCsvImportService interfaces and ApplicationProperties configuration properties class. Instead I have investigated how well your classes work and copied only the necessary methods to read CSV file and to convert each read line to the ProductPostDto class. Using T21_CSVproductsImporter class one of methods I return a list of ProductPostDto and in test class in the @BeforeEach method I save the list to the ProductRepository. Your classes that are used to read CSV and to properly convert data use various beans. Those beans cannot be injected when you just run the test class using just @DataJpaTest. It runs properly only when you run whole application context with @SpringBootTest. That's why I created two new classes that are free of such dependencies and have only the necessary functionality to read csv and to make it ProductPostDto. But not to save to repository using productService.createProduct(productDto); because productService needs to be injected like a bean. And the only class that needs to be injected is DotenvConfig:

@Slf4j
@DataJpaTest
@Import({DotenvConfig.class})
public class ProductRepositoryTest {
//....
}

@Slf4j
@DataJpaTest
@Import({DotenvConfig.class})
public class ProductRepositoryTest {
//....
}

Not sure why though.

JavaBot•2w ago

💤 Post marked as dormant

This post has been inactive for over 300 minutes, thus, it has been archived. If your question was not answered yet, feel free to re-open this post or create a new one. In case your post is not getting any attention, you can try to use /help ping. Warning: abusing this will result in moderative actions taken against you.

circleOP•2w ago

Yes I did that, this one worked. It's just sad that I can't utilize the stuff If you'd like to, I can see I'm sorry, I already refactored that to use OpenCSV instead for consistency between columns and DTOs Interesting, I might have to look it up. It probably had to do with a missing dependency, though I didn't even think it would be a problem

Tomasm21•2w ago

Now at our timezone UTC+2 hours 14:16. I'll be at home at 23:00 And will send To you directly? Or github pull request?

circleOP•2w ago

That is interesting, I'd love to see your implementations of it as I have not explored a lot more It might be a problem since it will conflict, but what matters is I could see the proper implementation of it. Sorry if I couldn't merge it

Tomasm21•2w ago

I will send you all classes directly here on discord OK? Making friend request to be able to send direct message.

circleOP•2w ago

Sure thing!

JavaBot•2w ago

💤 Post marked as dormant

This post has been inactive for over 300 minutes, thus, it has been archived. If your question was not answered yet, feel free to re-open this post or create a new one. In case your post is not getting any attention, you can try to use /help ping. Warning: abusing this will result in moderative actions taken against you.

Gaming

Programming

Populate Spring Data JPA before each @Test