Nuxt•2w ago

Enabling indexing with X-Robots-Tag

Hello! I have encountered a strange problem - my Nuxt site using nuxtjs/seo is blocking indexing on development/production with X-Robots-Tag header sent when loading the page. Could anyone help? This is a HTTP response in browser

Header     Value
connection     keep-alive
content-length     22225
content-type     text/html;charset=utf-8
date     Fri, 28 Mar 2025 16:02:47 GMT
keep-alive     timeout=5
vary     Accept-Encoding
x-powered-by     Nuxt
x-robots-tag     noindex

Header     Value
connection     keep-alive
content-length     22225
content-type     text/html;charset=utf-8
date     Fri, 28 Mar 2025 16:02:47 GMT
keep-alive     timeout=5
vary     Accept-Encoding
x-powered-by     Nuxt
x-robots-tag     noindex

Project: https://github.com/not-byte/nuxt-template For anyone seeking into this problem - thanks a lot!

GitHub

GitHub - not-byte/nuxt-template

Contribute to not-byte/nuxt-template development by creating an account on GitHub.

23 Replies

kapa.ai•2w ago

Hi @botprzemek. I'm kapa.ai, your friendly support bot. I will try to answer your question by searching through the knowledge base. :wave: ---------------------- I understand you're encountering an issue with your Nuxt site where the X-Robots-Tag: noindex header is being sent in both development and production environments, potentially preventing search engines from indexing your site. Looking at the knowledge sources provided, I don't see specific information about the nuxtjs/seo module and its interaction with the X-Robots-Tag header. However, I can provide some insights based on the available information. From the documentation, I can see that Nuxt has built-in SEO capabilities and also supports robots meta tags. The issue you're experiencing might be related to one of these configurations: 1. Check your SEO meta settings: You might have a robots meta tag set to noindex somewhere in your code. This could be in a useSeoMeta or useServerSeoMeta call.

   useSeoMeta({
     robots: 'noindex, follow' // This would prevent indexing
   })

   useSeoMeta({
     robots: 'noindex, follow' // This would prevent indexing
   })

Look for such configurations in your app.vue or layout files Docs > Api > Composables > useSeoMeta. 2. Check your public/robots.txt file: If you have a robots.txt file in your public directory, make sure it's not blocking indexing Docs > Guide > Directory-structure > public. 3. Check your Nuxt configuration: In your nuxt.config.ts file, look for any head meta tags related to robots or any experimental features that might affect this behavior. 4. Check for route rules: You might have route rules that set robots to noindex for certain routes or globally. If you're using the nuxtjs/seo module specifically (which isn't covered in the provided knowledge sources), you might want to check its configuration in your nuxt.config.ts file. For a temporary workaround, you could try using the experimental respectNoSSRHeader feature to control SSR responses Docs > Guide > Going-further > Experimental Features, though this is more related to SSR than SEO specifically. Without more specific information about your configuration, it's difficult to pinpoint the exact cause. I recommend reviewing all SEO-related configurations in your project and ensuring none of them are setting noindex directives. ---------------------- Note: you can ask me a follow up question by @ mentioning me again :speech_balloon: ----------------------

kapa.ai•2w ago

<@307600657345675265> Kapa.ai is still learning and improving, please let me know how I did by reacting below.

botprzemekOP•2w ago

@kapa.ai My useSeoMeta are set like this const robots = (): string => "index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1";

kapa.ai•2w ago

Let me check the knowledge base for you again. One moment! :books: ---------------------- I see you're using the useSeoMeta composable with a computed getter syntax for the robots meta tag. The way you've set it up should actually be allowing indexing with index, follow directives, which is the opposite of what you're experiencing with the X-Robots-Tag: noindex header. However, there's a small issue with your implementation. According to the documentation, when using the computed getter syntax with useSeoMeta, you should omit the property name in the function declaration. The correct format would be:

useSeoMeta({
  robots: () => "index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1"
})

useSeoMeta({
  robots: () => "index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1"
})

Notice that in your code, you're defining a separate function with const robots = (): string => which isn't directly connected to the useSeoMeta call. Make sure you're actually passing this function to the useSeoMeta composable. Additionally, the X-Robots-Tag HTTP header might be set elsewhere in your application. You should check: 1. Any server middleware that might be setting response headers 2. Nitro plugins that might be modifying the HTML response 3. Your nuxt.config.ts file for any head configurations with robots directives You could also try explicitly setting the response header to allow indexing using the useResponseHeader composable in your app.vue or a middleware file:

const header = useResponseHeader('X-Robots-Tag');
header.value = 'index, follow';

const header = useResponseHeader('X-Robots-Tag');
header.value = 'index, follow';

This should override any existing X-Robots-Tag header Docs > Api > Composables > useResponseHeader. If you're still facing issues, you might want to check if there are any Nitro plugins that are hooking into the render:html or render:response hooks and modifying the response headers Docs > Getting-started > Views. ---------------------- Note: you can ask me a follow up question by @ mentioning me again :speech_balloon: ----------------------

kapa.ai•2w ago

<@307600657345675265> Kapa.ai is still learning and improving, please let me know how I did by reacting below.

botprzemekOP•2w ago

botprzemekOP•2w ago

useReponseHeader correctly says noindex, nofollow

harlan•2w ago

most likely nuxt seo thinks the environment is not a production environment do you have a live example?

botprzemekOP•2w ago

Yeah, deployed it right now https://nuxt.notbyte.com/

notbyte

NuxtJS template · notbyte

Discover our template for NuxtJS - an open source framework that makes web development intuitive and powerful. Create performant and production-grade full-stack web apps and websites with confidence.

harlan•2w ago

Can you share the site config Or nuxt config You're doing something quite wrong sitemap path is pointed at local host So likely the entire app thinks it's development

botprzemekOP•2w ago

export default defineNuxtConfig({
    compatibilityDate: "2024-11-01",
    devtools: {
        enabled: true,
        timeline: {
            enabled: true
        }
    },
    modules: [
        "@nuxtjs/seo",
        "@nuxtjs/i18n",
        "@nuxtjs/tailwindcss",
        "@nuxt/eslint",
        "@nuxt/test-utils",
        "@nuxt/image"
    ],
    runtimeConfig: {
        public: {
            name: "notbyte",
            url: "https://nuxt.notbyte.com/",
            prefix: "/"
        }
    },
    app: {
        baseURL: process.env.NUXT_PUBLIC_PREFIX || "/",
        buildAssetsDir: "/",
        rootAttrs: {
            id: "notbyte-root"
        },
        rootTag: "body",
        teleportAttrs: {
            id: "notbyte-teleports"
        },
        teleportTag: "aside"
    },
    ogImage: {
        defaults: {
            cacheMaxAgeSeconds: 60 * 60 * 24 * 7,
            fonts: ["Chakra Petch:400", "Chakra Petch:700"],
            component: "Default"
        },
        fonts: [
            {
                name: "Chakra Petch",
                weight: 400,
                path: "/fonts/chakra-petch-v11-latin_latin-ext-regular.woff"
            },
            {
                name: "Chakra Petch",
                weight: 700,
                path: "/fonts/chakra-petch-v11-latin_latin-ext-700.woff"
            }
        ]
    },
    i18n: {
        baseUrl: process.env.NUXT_PUBLIC_URL,
        bundle: {
            optimizeTranslationDirective: false
        },
        customRoutes: "page",
        defaultLocale: "en",
        detectBrowserLanguage: {
            useCookie: true,
            cookieKey: "notbyte-lang",
            redirectOn: "root"
        },
        locales: [
            {
                code: "en",
                name: "English",
                language: "en-US",
                file: "en-US.ts"
            },
            {
                code: "pl",
                name: "Polski",
                language: "pl-PL",
                file: "pl-PL.ts"
            }
        ],
        strategy: "prefix_and_default"
    }
});

export default defineNuxtConfig({
    compatibilityDate: "2024-11-01",
    devtools: {
        enabled: true,
        timeline: {
            enabled: true
        }
    },
    modules: [
        "@nuxtjs/seo",
        "@nuxtjs/i18n",
        "@nuxtjs/tailwindcss",
        "@nuxt/eslint",
        "@nuxt/test-utils",
        "@nuxt/image"
    ],
    runtimeConfig: {
        public: {
            name: "notbyte",
            url: "https://nuxt.notbyte.com/",
            prefix: "/"
        }
    },
    app: {
        baseURL: process.env.NUXT_PUBLIC_PREFIX || "/",
        buildAssetsDir: "/",
        rootAttrs: {
            id: "notbyte-root"
        },
        rootTag: "body",
        teleportAttrs: {
            id: "notbyte-teleports"
        },
        teleportTag: "aside"
    },
    ogImage: {
        defaults: {
            cacheMaxAgeSeconds: 60 * 60 * 24 * 7,
            fonts: ["Chakra Petch:400", "Chakra Petch:700"],
            component: "Default"
        },
        fonts: [
            {
                name: "Chakra Petch",
                weight: 400,
                path: "/fonts/chakra-petch-v11-latin_latin-ext-regular.woff"
            },
            {
                name: "Chakra Petch",
                weight: 700,
                path: "/fonts/chakra-petch-v11-latin_latin-ext-700.woff"
            }
        ]
    },
    i18n: {
        baseUrl: process.env.NUXT_PUBLIC_URL,
        bundle: {
            optimizeTranslationDirective: false
        },
        customRoutes: "page",
        defaultLocale: "en",
        detectBrowserLanguage: {
            useCookie: true,
            cookieKey: "notbyte-lang",
            redirectOn: "root"
        },
        locales: [
            {
                code: "en",
                name: "English",
                language: "en-US",
                file: "en-US.ts"
            },
            {
                code: "pl",
                name: "Polski",
                language: "pl-PL",
                file: "pl-PL.ts"
            }
        ],
        strategy: "prefix_and_default"
    }
});

# Build a production distribution

FROM cgr.dev/chainguard/node:latest AS builder

WORKDIR /app

COPY --chown=node:node package*.json .

RUN npm install --clean

COPY --chown=node:node . .

RUN npm run build

# Run output in a clean environment

FROM alpine AS runner

WORKDIR /app

LABEL authors="botprzemek"

RUN apk add --update nodejs

COPY --from=builder --chown=node:node /app/.output ./.output

ENV NODE_ENV="production"

CMD ["node", ".output/server/index.mjs"]

EXPOSE 3000

# Build a production distribution

FROM cgr.dev/chainguard/node:latest AS builder

WORKDIR /app

COPY --chown=node:node package*.json .

RUN npm install --clean

COPY --chown=node:node . .

RUN npm run build

# Run output in a clean environment

FROM alpine AS runner

WORKDIR /app

LABEL authors="botprzemek"

RUN apk add --update nodejs

COPY --from=builder --chown=node:node /app/.output ./.output

ENV NODE_ENV="production"

CMD ["node", ".output/server/index.mjs"]

EXPOSE 3000

harlan•2w ago

What's your build command in package.json I'd suggest setting site config per the docs also

botprzemekOP•2w ago

Just wondering, do env variables load in production ex. in docker-compose environment?

harlan•2w ago

Possibly the node env is development when you're building Depends how it's distributed

botprzemekOP•2w ago

Following documentation https://nuxtseo.com/docs/site-config/guides/i18n I had created this, is it enough? Keys name, description, baseUrl and currentLocale

Nuxt SEO

Nuxt I18n · Nuxt Site Config · Nuxt SEO

How to use the Nuxt Site Config module with Nuxt I18n.

harlan•2w ago

Oh ye this should be fine

botprzemekOP•2w ago

For better understanding, I need to provide environment variables within building stage It should be fine as default values are correct, absolute url pointing to the domain Sitemap and robots are pointing to localhost https://nuxt.notbyte.com/robots.txt https://nuxt.notbyte.com/sitemap_index.xml

harlan•2w ago

yes i think the issue is that these files may be getting generated at build-time? although the server response time is super slow for them being static it points to localhost when it detects a development environment

botprzemekOP•2w ago

I have managed to reproduce the error, It seems that after setting buildAssetsDir to something other than default value SEO engine creates wrong paths. Should I create issue on GitHub?

defineNuxtConfig({
  app: {
    buildAssetsDir: string,
  }
})

defineNuxtConfig({
  app: {
    buildAssetsDir: string,
  }
})

@harlan

harlan•2w ago

that would be helpful, thanks

botprzemekOP•2w ago

Thanks for helping, left the issue on gh :nuxt: @kapa.ai close the discussion

kapa.ai•2w ago

Let me check the knowledge base for you again. One moment! :books: ---------------------- I'll close this discussion about the X-Robots-Tag indexing issue in your Nuxt application. To summarize what we've covered: 1. You're experiencing an issue where your Nuxt site is sending an X-Robots-Tag: noindex header despite having configured robots: "index, follow..." in your SEO settings 2. I suggested checking your implementation of useSeoMeta to ensure it's properly connected to your application 3. I recommended using the useResponseHeader composable to explicitly override the header 4. I advised checking for any server middleware or Nitro plugins that might be modifying response headers If you need further assistance with this or any other Nuxt-related issues in the future, feel free to start a new discussion. Thank you for your questions! ---------------------- Note: you can ask me a follow up question by @ mentioning me again :speech_balloon: ----------------------

kapa.ai•2w ago

<@307600657345675265> Kapa.ai is still learning and improving, please let me know how I did by reacting below.

Gaming

Programming

Enabling indexing with X-Robots-Tag

Did you find this page helpful?