Nuxt•2w ago

Failed to fetch urls with sitemap.xml

I am getting the following error when trying to get all valid urls for my webapp. [@nuxtjs/sitemap] Failed to fetch source. { url: 'https://api.kingsburyandpartners.ae/sitemap/urls.json', error: { FetchError: [GET] "https://api.kingsburyandpartners.ae/sitemap/urls.json": <no response> fetch failed at process.processTicksAndRejections (node:internal/process/task_queues:95:5)

23 Replies

kapa.ai•2w ago

Hi @nathanr. I'm kapa.ai, your friendly support bot. I will try to answer your question by searching through the knowledge base. :wave: ---------------------- Based on the error message you're encountering, it appears you're having an issue with the Nuxt sitemap module when it tries to fetch URLs from an external source. The error Failed to fetch source indicates that the sitemap module is unable to retrieve data from the specified URL (<https://api.kingsburyandpartners.ae/sitemap/urls.json>). This could be due to several reasons: 1. The API endpoint might be unavailable or not responding 2. There might be network connectivity issues 3. The endpoint might require authentication 4. CORS issues might be preventing the request For Nuxt sitemap configuration, you need to ensure your API endpoint is accessible. According to a similar issue on Stack Overflow, you might need to: 1. Check if your API is properly configured to return the sitemap URLs 2. Verify the API endpoint is accessible from your build environment If you're using the newer @nuxtjs/sitemap module (v5+), you should follow the dynamic URLs guide as mentioned in this Stack Overflow answer. For dynamic pages, you need to properly configure an API source for the sitemap module. If you're using Nuxt 2 with the sitemap module, make sure you're using the correct version (2.x) as noted in another Stack Overflow post. You might also want to check if there are any CORS issues when fetching from your API, as similar fetch errors can occur due to CORS restrictions as mentioned in this Stack Overflow post. If the issue persists, you could try: 1. Verifying the API endpoint manually (using a tool like Postman) 2. Adding proper error handling in your sitemap configuration 3. Checking your server logs for more detailed error information ---------------------- Note: you can ask me a follow up question by @ mentioning me again :speech_balloon: ----------------------

kapa.ai•2w ago

<@243051127904534528> Kapa.ai is still learning and improving, please let me know how I did by reacting below.

nathanrOP•2w ago

@kapa.ai When I visit or curl the url I can see all the valid urls from my database

kapa.ai•2w ago

Let me check the knowledge base for you again. One moment! :books: ---------------------- If you can successfully access the URL directly through your browser or curl, but the sitemap module is still failing to fetch it, this suggests there might be an issue with how the request is being made during the build process. Here are some potential causes and solutions: 1. SSR vs Client-side fetching: The error <no response> fetch failed often appears when there are issues with server-side requests. As seen in this GitHub issue, there can be differences in how API routes work in development versus production builds. 2. CORS issues: Even if you can access the URL directly, the sitemap module might be encountering CORS restrictions when making the request from your Nuxt application. This is a common issue as mentioned in this Stack Overflow post. 3. Request timeout: The API might be taking too long to respond during the build process. Here are some potential solutions: 1. Try prerendering the route:

   nitro: {
     prerender: {
       routes: ['/sitemap.xml'],
     },
   }

   nitro: {
     prerender: {
       routes: ['/sitemap.xml'],
     },
   }

2. Use a server proxy: Instead of fetching directly from the external URL, create a server route in your Nuxt app that fetches the data and returns it. 3. Add proper headers: Make sure your request includes appropriate headers, especially if authentication is required. 4. Increase timeout: If the request is timing out, you might need to increase the timeout setting for the fetch operation. 5. Check for HTTPS issues: If you're developing locally on HTTP but your API uses HTTPS, this could cause issues as noted in this Stack Overflow answer. 6. Verify environment variables: Make sure your API URL is correctly set in your environment configuration as discussed in this Stack Overflow post. If the issue persists, you might want to check the Nuxt server logs for more detailed error information or consider using a different approach for generating your sitemap, such as manually creating it or using a different module. ---------------------- Note: you can ask me a follow up question by @ mentioning me again :speech_balloon: ----------------------

kapa.ai•2w ago

<@243051127904534528> Kapa.ai is still learning and improving, please let me know how I did by reacting below.

nathanrOP•2w ago

@kapa.ai even when building the app it's returning fetch failed

kapa.ai•2w ago

Let me check the knowledge base for you again. One moment! :books: ---------------------- Based on your issue with fetch failing during the build process, this is a common problem in Nuxt applications when making external API requests. Let me explain what might be happening and suggest some solutions. When Nuxt builds your application, it attempts to fetch data from your external API endpoint, but it's failing with a "fetch failed" error. This can happen for several reasons: 1. Server-side fetch issues: During build time, Nuxt is making server-side requests which behave differently than client-side requests. As seen in several issues, there can be problems with how headers and cookies are forwarded during server-side fetching. 2. Environment differences: The build environment might have different network configurations than your development environment. Here are some potential solutions: 1. Use useRequestFetch instead of regular $fetch: This composable is designed to forward request context and headers properly during server-side requests.

// Instead of using $fetch directly
const requestFetch = useRequestFetch()
const { data } = await useAsyncData(() => requestFetch('/api/your-endpoint'))

// Instead of using $fetch directly
const requestFetch = useRequestFetch()
const { data } = await useAsyncData(() => requestFetch('/api/your-endpoint'))

As mentioned in the useRequestFetch documentation, this helps forward the request context and headers when making server-side fetch requests. 2. Try using useAsyncData with $fetch instead of useFetch: There appears to be a bug with useFetch in some Nuxt versions that causes hydration issues in production builds but works fine in development.

// Instead of useFetch
const { data } = await useAsyncData(
  'your-key',
  () => $fetch('/your-endpoint')
)

// Instead of useFetch
const { data } = await useAsyncData(
  'your-key',
  () => $fetch('/your-endpoint')
)

This approach has been reported to work in both development and production environments according to this GitHub issue. 3. Add a custom key to your useFetch calls: If you're using useFetch, try adding a manual key:

const { data } = await useFetch('/your-endpoint', {
  key: 'define-custom-key'
})

const { data } = await useFetch('/your-endpoint', {
  key: 'define-custom-key'
})

4. Create a server proxy: Instead of fetching directly from your external API, create a server route in your Nuxt app that fetches the data and returns it:

// server/api/proxy.js
export default defineEventHandler(async (event) => {
  return await $fetch('<https://api.kingsburyandpartners.ae/sitemap/urls.json'>)
})

// server/api/proxy.js
export default defineEventHandler(async (event) => {
  return await $fetch('<https://api.kingsburyandpartners.ae/sitemap/urls.json'>)
})

Then fetch from this internal endpoint instead. 5. Check for CORS issues: Even though you can access the URL directly, the server-side fetch might be encountering CORS restrictions as mentioned in this Stack Overflow post. If you continue to face issues, consider examining the detailed build logs with debug mode enabled in your Nuxt config:

// nuxt.config.js
export default defineNuxtConfig({
  debug: true,
  // other config
})

// nuxt.config.js
export default defineNuxtConfig({
  debug: true,
  // other config
})

This will provide more verbose logs that might help identify the specific cause of the fetch failure. ---------------------- Note: you can ask me a follow up question by @ mentioning me again :speech_balloon: ----------------------

kapa.ai•2w ago

<@243051127904534528> Kapa.ai is still learning and improving, please let me know how I did by reacting below.

nathanrOP•2w ago

@kapa.ai what is the correct way to do seo for a page

kapa.ai•2w ago

Let me check the knowledge base for you again. One moment! :books: ---------------------- SEO for a Page in Nuxt In Nuxt, there are several approaches to implement SEO for your pages. Here are the recommended methods: Using useSeoMeta Composable The most modern and type-safe approach in Nuxt 3 is using the useSeoMeta composable:

<script setup lang="ts">
useSeoMeta({
  title: 'My Amazing Site',
  ogTitle: 'My Amazing Site',
  description: 'This is my amazing site, let me tell you all about it.',
  ogDescription: 'This is my amazing site, let me tell you all about it.',
  ogImage: '<https://example.com/image.png',>
  twitterCard: 'summary_large_image',
})
</script>

<script setup lang="ts">
useSeoMeta({
  title: 'My Amazing Site',
  ogTitle: 'My Amazing Site',
  description: 'This is my amazing site, let me tell you all about it.',
  ogDescription: 'This is my amazing site, let me tell you all about it.',
  ogImage: '<https://example.com/image.png',>
  twitterCard: 'summary_large_image',
})
</script>

For reactive content, use the computed getter syntax:

<script setup lang="ts">
const title = ref('My title')

useSeoMeta({
  title,
  description: () => `This is a description for the ${title.value} page`
})
</script>

<script setup lang="ts">
const title = ref('My title')

useSeoMeta({
  title,
  description: () => `This is a description for the ${title.value} page`
})
</script>

This approach is recommended because it's XSS safe and has full TypeScript support with over 100+ meta tags fully typed Nuxt Docs: useSeoMeta. Performance Optimization For better performance, you can use a server-only condition when meta tags don't need to be reactive:

<script setup lang="ts">
if (import.meta.server) {
  // These meta tags will only be added during server-side rendering
  useSeoMeta({
    robots: 'index, follow',
    description: 'Static description that does not need reactivity',
    ogImage: '<https://example.com/image.png',>
  })
}

// Only use reactive meta tags when necessary
const dynamicTitle = ref('My title')
useSeoMeta({
  title: () => dynamicTitle.value,
  ogTitle: () => dynamicTitle.value,
})
</script>

<script setup lang="ts">
if (import.meta.server) {
  // These meta tags will only be added during server-side rendering
  useSeoMeta({
    robots: 'index, follow',
    description: 'Static description that does not need reactivity',
    ogImage: '<https://example.com/image.png',>
  })
}

// Only use reactive meta tags when necessary
const dynamicTitle = ref('My title')
useSeoMeta({
  title: () => dynamicTitle.value,
  ogTitle: () => dynamicTitle.value,
})
</script>

Using Meta Components Nuxt 3 also provides meta components for a more declarative approach:

<template>
  <div>
    <Head>
      <Title>My App</Title>
      <Meta name="description" content="My app description"/>
    </Head>
    <!-- page content -->
  </div>
</template>

<template>
  <div>
    <Head>
      <Title>My App</Title>
      <Meta name="description" content="My app description"/>
    </Head>
    <!-- page content -->
  </div>
</template>

Make sure to use capital letters for these component names to distinguish them from native HTML elements Nuxt Docs: Migration Meta. Using definePageMeta for Page-Specific Metadata For pages in the pages/ directory, you can use definePageMeta:

<script setup lang="ts">
definePageMeta({
  layout: 'default',
  // You can add custom metadata here
})
</script>

<script setup lang="ts">
definePageMeta({
  layout: 'default',
  // You can add custom metadata here
})
</script>

This is useful for page-specific configuration Nuxt Docs: definePageMeta. Options API (if not using Composition API) If you prefer the Options API:

<script>
// Must use defineNuxtComponent
export default defineNuxtComponent({
  head (nuxtApp) {
    return {
      meta: [{
        name: 'description',
        content: 'This is my page description.'
      }]
    }
  }
})
</script>

<script>
// Must use defineNuxtComponent
export default defineNuxtComponent({
  head (nuxtApp) {
    return {
      meta: [{
        name: 'description',
        content: 'This is my page description.'
      }]
    }
  }
})
</script>

Note that the head method receives the nuxt app but cannot access the component instance Nuxt Docs: Bridge Meta. Choose the approach that best fits your project's needs and coding style. The useSeoMeta composable is generally recommended for most modern Nuxt 3 applications. ---------------------- Note: you can ask me a follow up question by @ mentioning me again :speech_balloon: ----------------------

kapa.ai•2w ago

<@243051127904534528> Kapa.ai is still learning and improving, please let me know how I did by reacting below.

nathanrOP•2w ago

It's failing to create the meta tag

nathanrOP•2w ago

@kapa.ai it's failing to create the meta tag // Set SEO meta tags reactively if (import.meta.server) { useSeoMeta({ title: metaTitle, description: metaDescription, robots: 'index, follow', 'og:title': metaTitle, 'og:description': metaDescription, 'og:image': 'https://hub.kingsburyandpartners.ae/private-office.jpg', 'og:url': canonicalUrl, 'og:type': 'website', 'twitter:card': 'summary_large_image', }); }

Your Online Private Office in the UAE | Access Over 300 Trusted Services

Private Office Jpg', | Your Online Private Office in the UAE | Acce...

The Kingsbury Private Office — created for individuals, families, and business owners who value access, time, and discretion in how their affairs are managed. Our Private Office brings together over 300 services across more than 60 trusted partners, delivering clear, confident support across your financial, personal, and corporate world — fr...

kapa.ai•2w ago

Let me check the knowledge base for you again. One moment! :books: ---------------------- I see you're having issues with creating meta tags using the useSeoMeta composable in Nuxt. There appears to be a syntax error in your implementation. The issue is with how you're specifying the Open Graph properties. According to the Nuxt useSeoMeta documentation, Open Graph properties should use camelCase format rather than the hyphenated format you're using. Here's the corrected version:

if (import.meta.server) {
    useSeoMeta({
        title: metaTitle,
        description: metaDescription,
        robots: 'index, follow',
        ogTitle: metaTitle,           // Changed from 'og:title'
        ogDescription: metaDescription, // Changed from 'og:description'
        ogImage: '<https://hub.kingsburyandpartners.ae/private-office.jpg',> // Changed from 'og:image'
        ogUrl: canonicalUrl,          // Changed from 'og:url'
        ogType: 'website',            // Changed from 'og:type'
        twitterCard: 'summary_large_image', // Changed from 'twitter:card'
    });
}

if (import.meta.server) {
    useSeoMeta({
        title: metaTitle,
        description: metaDescription,
        robots: 'index, follow',
        ogTitle: metaTitle,           // Changed from 'og:title'
        ogDescription: metaDescription, // Changed from 'og:description'
        ogImage: '<https://hub.kingsburyandpartners.ae/private-office.jpg',> // Changed from 'og:image'
        ogUrl: canonicalUrl,          // Changed from 'og:url'
        ogType: 'website',            // Changed from 'og:type'
        twitterCard: 'summary_large_image', // Changed from 'twitter:card'
    });
}

The useSeoMeta composable is designed to use a flat object structure with specific property names that follow camelCase conventions for different meta tag types. This helps avoid common mistakes and provides TypeScript support. If you're working with dynamic content, make sure your metaTitle, metaDescription, and canonicalUrl variables are properly defined before the useSeoMeta call. For reactive content, you can also use the computed getter syntax as mentioned in the useSeoMeta documentation:

useSeoMeta({
  title: () => metaTitle.value,
  ogTitle: () => metaTitle.value
})

useSeoMeta({
  title: () => metaTitle.value,
  ogTitle: () => metaTitle.value
})

---------------------- Note: you can ask me a follow up question by @ mentioning me again :speech_balloon: ----------------------

kapa.ai•2w ago

<@243051127904534528> Kapa.ai is still learning and improving, please let me know how I did by reacting below.

nathanrOP•2w ago

@kapa.ai this is my source sitemap: { debug: true, sources: [ ['https://api.kingsburyandpartners.ae/sitemap/urls', { headers: { 'User-Agent': 'Nuxt-Sitemap', 'Content-Type': 'application/json' }, }], ], }, but I am getting this? ├─ /sitemap/debug.json (468ms) nitro 9:09:28 PM ERROR [@nuxtjs/sitemap] Failed to fetch source. { url: 'https://api.kingsburyandpartners.ae/sitemap/urls',
is this a nitro issue??

kapa.ai•2w ago

Let me check the knowledge base for you again. One moment! :books: ---------------------- Based on the error message you're seeing, this appears to be an issue with the Nuxt sitemap module failing to fetch your external API source. Let me explain what might be happening and suggest some solutions. Understanding the Issue The error [js/sitemap] Failed to fetch source indicates that the sitemap module is unable to retrieve data from your specified URL. This is likely related to how Nitro (Nuxt's server engine) handles external requests during the build or generation process. Several GitHub issues show similar problems with external fetches in Nitro: 1. In this GitHub issue, users experienced issues with the crypto.createHash implementation in Nitro's unenv, which can affect network requests. 2. Another GitHub issue shows problems with URL formatting in Nitro, causing 403 Forbidden errors. Possible Solutions Here are some approaches to fix this issue: 1. Add Compatibility Flags Try adding compatibility flags to your Nuxt configuration to enable Node.js compatibility:

// nuxt.config.ts
export default defineNuxtConfig({
  nitro: {
    // Add Node.js compatibility
    preset: 'node',
    // Or if using Cloudflare
    compatibility: {
      flags: ["nodejs_compat"]
    }
  },
  // Your sitemap config...
})

// nuxt.config.ts
export default defineNuxtConfig({
  nitro: {
    // Add Node.js compatibility
    preset: 'node',
    // Or if using Cloudflare
    compatibility: {
      flags: ["nodejs_compat"]
    }
  },
  // Your sitemap config...
})

2. Create a Server Proxy Instead of fetching directly from the external API, create a server route in your Nuxt app:

// server/api/sitemap-urls.js
export default defineEventHandler(async (event) => {
  return await $fetch('<https://api.kingsburyandpartners.ae/sitemap/urls',> {
    headers: { 
      'User-Agent': 'Nuxt-Sitemap', 
      'Content-Type': 'application/json' 
    }
  })
})

// server/api/sitemap-urls.js
export default defineEventHandler(async (event) => {
  return await $fetch('<https://api.kingsburyandpartners.ae/sitemap/urls',> {
    headers: { 
      'User-Agent': 'Nuxt-Sitemap', 
      'Content-Type': 'application/json' 
    }
  })
})

Then update your sitemap configuration to use this local endpoint:

sitemap: {
  debug: true,
  sources: [
    ['/api/sitemap-urls'],
  ],
}

sitemap: {
  debug: true,
  sources: [
    ['/api/sitemap-urls'],
  ],
}

3. Inline External Dependencies If you're using custom modules that might be causing issues with Nitro's bundling, try inlining them:

// nuxt.config.ts
export default defineNuxtConfig({
  nitro: {
    externals: {
      inline: [/your-problematic-dependency/],
    }
  },
  // Your sitemap config...
})

// nuxt.config.ts
export default defineNuxtConfig({
  nitro: {
    externals: {
      inline: [/your-problematic-dependency/],
    }
  },
  // Your sitemap config...
})

4. Add Prerender Routes Manually You can also try manually adding your sitemap routes using addPrerenderRoutes:

// In your module or plugin
import { addPrerenderRoutes } from '@nuxt/kit'

// Add your routes manually
addPrerenderRoutes(['/sitemap.xml'])

// In your module or plugin
import { addPrerenderRoutes } from '@nuxt/kit'

// Add your routes manually
addPrerenderRoutes(['/sitemap.xml'])

As mentioned in the Nuxt Kit documentation, this can help ensure your sitemap is properly generated. Is This a Nitro Issue? Yes, this appears to be related to how Nitro handles external requests during the build process. The error suggests that Nitro is failing to properly fetch from your external API, which is a known limitation in certain environments. If you're deploying to a serverless or edge environment, these environments often have restrictions on outbound network requests during build time, which can cause these types of fetch failures. Try the solutions above, particularly the server proxy approach, as it's often the most reliable way to handle external API requests in Nuxt applications. Let me know if you need more specific guidance based on your deployment environment! ---------------------- Note: you can ask me a follow up question by @ mentioning me again :speech_balloon: ----------------------

kapa.ai•2w ago

<@243051127904534528> Kapa.ai is still learning and improving, please let me know how I did by reacting below.

nathanrOP•2w ago

@kapa.ai using a proxy server worked

kapa.ai•2w ago

Let me check the knowledge base for you again. One moment! :books: ---------------------- I'm glad to hear that using a proxy server worked for your sitemap issue! This is a common solution for handling external API requests in Nuxt applications. The proxy approach is effective because it bypasses the CORS restrictions and network limitations that can occur during server-side rendering or build time. By creating a server route within your Nuxt application that fetches the external data, you're able to avoid the "Failed to fetch source" errors. This solution works well because: 1. Server-side requests in Nuxt (via Nitro) can sometimes have issues with external APIs as mentioned in this GitHub issue 2. The proxy approach allows you to control headers, error handling, and request configuration in one place 3. It's particularly useful when deploying to environments with restrictions on outbound network requests If you encounter similar issues with other external API calls in your Nuxt application, remember that the same proxy pattern can be applied to those as well. Great job implementing the solution! ---------------------- Note: you can ask me a follow up question by @ mentioning me again :speech_balloon: ----------------------

kapa.ai•2w ago

<@243051127904534528> Kapa.ai is still learning and improving, please let me know how I did by reacting below.

harlan•2w ago

🤔 what was the issue?

nathanrOP•2w ago

I was trying to generate a site map on all my dynamic urls, but using

sitemap: {
  sources: ['https://api.kingsburyandpartners.ae/sitemap/urls'],
},

sitemap: {
  sources: ['https://api.kingsburyandpartners.ae/sitemap/urls'],
},

was throwing a fetch error, - Thought it could have been CORS so tried to add headers to the sitemap source. - Thought it could have been timeout on the request as the data is pulled from the database at the time - Added a static file on the webserver (Django) - Added a static file on the frontend system But what solved it was a proxy server, and this worked with my original Django view which pulled the data from the database and returned it as a json I was actually get a CORS error to be honest, so not sure if that was cause by Nitro making the request during the prerender, and how that was fixed was because it was going through my Nuxt app

Gaming

Programming

Failed to fetch urls with sitemap.xml

Did you find this page helpful?