Nextjs: Reducing outbound bandwidth when google indexes all site URLs
I have noticed that when google indexes all 10k+ site URLs, the outbound bandwidth in fly.io goes very high (8gb+). As my site will continue to have thousands of more URLs as more users use the site, I am wondering how to get around this before I get a very large bill.
For context, the project is using Supabase as backend.
15 Replies
Disable it for paths that you don’t want to
Reduce the build output to the maximum
And a lot of hope that shit goes well
All public routes (currently 10k+) need to be indexed unfortunately.
The private routes are already disallowed in the robots file
You need to remove js usage from where you can
With server components is easier, but “classic” components, next sucks
Hmmm. With js usage, do you mean client components are the main cause for this? I only have client components where absolutely necessary (forms, interactive components, etc).
You should use the output analyzer to see what’s bundled
Or even if you run “next build” you should have some ideas about it
Other than that, idk how you can make things cheaper without leaving nextjs or without a contract with specific prices for bandwidth
The irony is that I was suggested nextjs because it is better for seo over vanilla react.
It’s better for SEO
Not for bandwidth
:deranged:
A blank nextjs page has like 100?kb worth of js
Just out of curiosity, given the bandwidth issue, what framework would you suggest?
It’s too late to change now but for future projects, I would prefer to try something else.
For websites (content but not necessarily needs js, like a blog, or newsletter), Astro
For apps, either solid/svelte 5 (after is released)
Solid is more react esque in terms of syntax, but with better performance and overall better user facing output
Svelte5 will be super close to solid in terms of output, but their own dsl
I have heard a lot about astro. Might give it a try in my next project.
I'll try svelte too once the next version drops.
For this project, I'll try to bring down the JS even more but I suspect we will just have to eat the bandwidth costs for now.
In theory you can use the react components on Astro, but they add heavy react code to the client
How well Astro scales for big projects idk
Wouldn't the new astro islands help with that?
Astro islands are akin to vercel ppr
The problem still react deps bundled for Astro
Astro allows close to 0 js, but adding client side react adds a lot back into it
I will have to investigate it. I think astro will be better but I would still need some client side rendering for some interactive stuff. Lets see. Thanks for the help.
Glad to help