C#•2w ago

Optimizing Filtering and Pagination: Reducing Backend Requests Efficiently

I want to implement filtering and pagination while minimizing backend requests as much as possible. Here’s the challenge:

    1)When the page loads, I send the first N records from the backend.
    2)The user applies filters, and if matching data exists within those N records, I display it instantly.
    3)If no matching data is found in the current N records, I need to check the backend for more data.
    4)This means that for every filter change, I might have to query the backend, which increases requests.

    1)When the page loads, I send the first N records from the backend.
    2)The user applies filters, and if matching data exists within those N records, I display it instantly.
    3)If no matching data is found in the current N records, I need to check the backend for more data.
    4)This means that for every filter change, I might have to query the backend, which increases requests.

-I know caching can help, but if the filtering logic is complex, managing cache efficiently can also become challenging. -How can I reduce backend requests while handling filtering and pagination efficiently?

18 Replies

Salman•2w ago

this is how I did it at least:

 public async Task<ActionResult<IEnumerable<ProductDTO>>> GetProducts(string categoryName = null, string searchQuery = null, decimal? minPrice = null,
        decimal? maxPrice = null,
        int? pageNumber = null,
        int? pageSize = null)
    {
        IQueryable<Product> productsQuery = _context.Products.Include(p => p.Category);

        if (!string.IsNullOrWhiteSpace(categoryName))
            productsQuery = productsQuery.Where(p => p.Category.Name.ToLower() == categoryName.ToLower());

        if (!string.IsNullOrEmpty(searchQuery))
        {
            productsQuery = productsQuery.Where(p =>
                p.Name.ToLower().Contains(searchQuery.ToLower()) ||
                p.Description.ToLower().Contains(searchQuery.ToLower())
            );
        }

        if (minPrice.HasValue)
            productsQuery = productsQuery.Where(p => p.Price >= minPrice);

        if (maxPrice.HasValue)
            productsQuery = productsQuery.Where(p => p.Price <= maxPrice);

        if (pageNumber.HasValue && pageSize.HasValue)
            productsQuery = productsQuery.Skip((pageNumber.Value - 1) * pageSize.Value).Take(pageSize.Value);

        var products = await productsQuery.ToListAsync();
        var productDTOs = products.Select(p => new ProductDTO(p.Id, p.Name, p.Description, p.Price, p.imageUrl, p.CategoryId)).ToList();

        return productDTOs;
        
    }

 public async Task<ActionResult<IEnumerable<ProductDTO>>> GetProducts(string categoryName = null, string searchQuery = null, decimal? minPrice = null,
        decimal? maxPrice = null,
        int? pageNumber = null,
        int? pageSize = null)
    {
        IQueryable<Product> productsQuery = _context.Products.Include(p => p.Category);

        if (!string.IsNullOrWhiteSpace(categoryName))
            productsQuery = productsQuery.Where(p => p.Category.Name.ToLower() == categoryName.ToLower());

        if (!string.IsNullOrEmpty(searchQuery))
        {
            productsQuery = productsQuery.Where(p =>
                p.Name.ToLower().Contains(searchQuery.ToLower()) ||
                p.Description.ToLower().Contains(searchQuery.ToLower())
            );
        }

        if (minPrice.HasValue)
            productsQuery = productsQuery.Where(p => p.Price >= minPrice);

        if (maxPrice.HasValue)
            productsQuery = productsQuery.Where(p => p.Price <= maxPrice);

        if (pageNumber.HasValue && pageSize.HasValue)
            productsQuery = productsQuery.Skip((pageNumber.Value - 1) * pageSize.Value).Take(pageSize.Value);

        var products = await productsQuery.ToListAsync();
        var productDTOs = products.Select(p => new ProductDTO(p.Id, p.Name, p.Description, p.Price, p.imageUrl, p.CategoryId)).ToList();

        return productDTOs;
        
    }

It could've been better or refined but it might give you a rought idea about it

YujiOP•2w ago

I have one more question, when i apply filter i am approaching this way Whatever data i have sented at frontend i use js and apply filters there should i change my approach?

Salman•2w ago

I didnt understand that but if you meant JS is okay to apply filters at the frontend then yes totally fine In the above code as you can see I'm receiving various filters from the frontend , I check if a filter is present I apply it on the EF Query, if it doesnt I move to the next step, eventually I return the retrieved data from the database.

YujiOP•2w ago

My current approach involves fetching data from the backend and sending it to the frontend in a structured format, making it easy to filter and display to the user. While most filtering happens on the frontend, the backend also applies filtering based on the structured view I control. So basically you just build the result progressively based on user request and at last apply pagination and sent the data to user got it!

Salman•2w ago

exactly

YujiOP•2w ago

So I am confused right now should i change my approach or not :/

Salman•2w ago

your approach sounds fine as long as it works

YujiOP•2w ago

Issue comes in when i want to reduce backend call, how i can update strucutre effectively

Salman•2w ago

You should call your backend when searching a value , applying a filter or updating the page , make sure you dont call it unnecessarily. For searching a value you can't rely on the items you have on the frontend cuz maybe the item exists in the other sections since the frontend only contains a section of the whole data. So you might always need to call backend in order to make sure that you search the whole data

YujiOP•2w ago

So there is uncertainity and we cant rely on data that i sent on frontend got it, so I might as well change filtering part on backend, because I am doing 2 way filters right now So instead of this i can rely on backend So removing filtering logic from frontend is my next step thanks

Salman•2w ago

Ofc you cant, you only have a portion of data on the frontend Another approach is that just fetch the whole data on the frontend and then apply the filters on the frontend, no need to call the backend. But the obvious problem is that what if the database has thousands of records, fetching them all would be unnecessary and too slow . So the best option imo is pagination and filtering

YujiOP•2w ago

I was high few moment ago I only thought about first condition

data is not presnt at front end -> get from backend

data is not presnt at front end -> get from backend

If data is present at frontend -> there can be two case -> there is still some missing data based on filter at backend 
or 
there is no missing data and frontend have evything

If data is present at frontend -> there can be two case -> there is still some missing data based on filter at backend 
or 
there is no missing data and frontend have evything

Yeah this will cause backend issue, i thought about this but seems bad idea I think its good if your db have like few hundred or thousand record?

Salman•2w ago

Nah I still dont prefer it

YujiOP•2w ago

Reason for that?

Salman•2w ago

Because it'll still be too slow The pagination approach is efficient for long term

YujiOP•2w ago

Even for few hundred records? Got it Thanks alot for helping brother appreciate your time and efforts me closing this request now

Salman•2w ago

Welcome $close

MODiX•2w ago

If you have no further questions, please use /close to mark the forum thread as answered

Gaming

Programming

Optimizing Filtering and Pagination: Reducing Backend Requests Efficiently

Did you find this page helpful?