Pagination per page takes longer to reflect the changes

I have the fastPaginate() for my tables but changing from diffrent records per pasge takes long time ..Above one minute.How can I optimize this cause it's giving a very bad experience.