Getting 404 using Googlebot, but everything seems fine in production.
Hey all!
I have an issue where if I open up my Google Search Console most of my pages are not indexed (this seems a bit random, but most doesnt work), since its returning 404 for the content of the page. The robots tag seems fine, and it is set to follow,index (so it looks like the head looks somewhat good), but the body doesn't.
The live website works fine, however the code is very poorly written, and it is quite slow aswell (going to improve it, but I don't think that relates to the issue).
I'm hosting it in DigitalOcean using app platform. Build Command: npm run build ; Run Command: node .output/server/index.mjs
The live site is here: https://whitelab.hu
Please let me know if you need more info to troubleshoot. Thank you in advance! 🙏
my Nuxt config:
Laborvizsgálatok és szűrővizsgálatok | Whitelab
Általános laborvizsgálatok és szűrővizsgálatok, koronavírus PCR és antigén tesztek, valamint védettségi vizsgálatok a Whitelab kínálatában.
3 Replies
Hi @smoothie. I'm kapa.ai, your friendly support bot. I will try to answer your question by searching through the knowledge base. :wave:
----------------------
Based on the information provided, it seems you're experiencing an issue with Google Search Console reporting 404 errors for most of your pages, despite the live website working fine. This could be related to how your Nuxt application is handling server-side rendering (SSR) or how it's deployed on DigitalOcean. Let's address this issue step by step:
1. SSR Configuration:
Your Nuxt config shows that SSR is enabled (
ssr: true
), which is correct for SEO purposes. However, there might be issues with how the server is handling requests from Googlebot.
2. DigitalOcean App Platform:
Your deployment setup on DigitalOcean App Platform seems correct. The build command npm run build
and run command node .output/server/index.mjs
are appropriate for a Nuxt 3 application DigitalOcean Deploy (https://nitro.unjs.io/deploy/providers/digitalocean).
3. Sitemap and Robots:
You're using nuxt-simple-sitemap
and nuxt-simple-robots
, which is good for SEO. Make sure your sitemap is accessible and correctly lists all your pages.
4. Route Rules:
Your routeRules
configuration looks fine, with caching set up for the home page.
To troubleshoot this issue, consider the following steps:
1. Check Server Logs:
Review your DigitalOcean App Platform logs for any errors when Googlebot is crawling your site.
2. Test with Google Search Console's URL Inspection Tool:
Use this tool to see how Googlebot views your pages. This can help identify if the issue is specific to Googlebot.
3. Verify Dynamic Rendering:
Ensure that your server is correctly handling requests from Googlebot. You might need to implement dynamic rendering if your content relies heavily on client-side JavaScript Universal Rendering (https://nuxt.com/docs/guide/concepts/rendering#universal-rendering).
4. Check for Redirect Issues:
Verify that your redirects
configuration isn't causing unexpected behavior for Googlebot.
5. Implement Error Handling:
Add proper error handling to your Nuxt app, including a custom 404 page PageError Component (https://ui.nuxt.com/pro/components/page-error).
6. Optimize Performance:
While you mentioned the site is slow, improving performance can sometimes resolve crawling issues. Consider implementing lazy loading and optimizing your build.
7. Verify API Routes:
If you're using API routes for dynamic content, ensure they're accessible to Googlebot.
8. Check for Client-Side Only Content:
Ensure critical content isn't rendered only on the client-side, as Googlebot might not execute all JavaScript.
If the issue persists after trying these steps, you might need to dig deeper into your application's server-side rendering process or consider reaching out to DigitalOcean support to ensure there are no platform-specific issues affecting Googlebot's access to your site.
Remember, SEO optimization is an ongoing process, and it may take some time for Google to recrawl and reindex your site after making changes.
----------------------
Note: you can ask me a follow up question by @ mentioning me again :speech_balloon:
----------------------<@353981160847573013> Kapa.ai is still learning and improving, please let me know how I did by reacting below.
Your home page is returning a 304
seems like some proxy misconfiguration
also please mgirate to @nuxtjs/robots and @nuxtjs/sitemap those modules are deprecated