Nuxt•5mo ago

Dynamic Sitemap Generation not working in production

In my nuxt.config.ts I have the following, where the URL endpoint returns an array of correctly formatted sitemap objects. I know this because when running the site locally with npm run dev it works great. I know this because in localhost I can see 3615 URL's pulled from the production API.

    sitemap: {
      sources : [
          'https://api.mcmodels.net/urls'
      ],
        cacheMaxAgeSeconds: 3600,
        xslColumns: [
            { label: 'URL', width: '50%' },
            { label: 'Last Modified', select: 'sitemap:lastmod', width: '25%' },
            { label: 'Priority', select: 'sitemap:priority', width: '12.5%' },
            { label: 'Change Frequency', select: 'sitemap:changefreq', width: '12.5%' },
        ],
    },

    sitemap: {
      sources : [
          'https://api.mcmodels.net/urls'
      ],
        cacheMaxAgeSeconds: 3600,
        xslColumns: [
            { label: 'URL', width: '50%' },
            { label: 'Last Modified', select: 'sitemap:lastmod', width: '25%' },
            { label: 'Priority', select: 'sitemap:priority', width: '12.5%' },
            { label: 'Change Frequency', select: 'sitemap:changefreq', width: '12.5%' },
        ],
    },

That same code is being ran full SSR on the production site at https://mcmodels.net/sitemap.xml yet it's not pulling any of the dynamic URLs.

36 Replies

JoshuaOP•5mo ago

And as you can see the sources URL is working well, especially known since it works great in localhost. https://api.mcmodels.net/urls An initial suspicion is that the sitemap module won't hit the "remote" api at the same domain, but I'm not sure.

oneeach•5mo ago

what are you using to generate your sitemap?

JoshuaOP•5mo ago

https://api.mcmodels.net/urls This is coming from a Laravel backend on the same server.

class SitemapController
{
    public function index(): JsonResponse
    {
        $sitemapUrls = [];

        // Fetch all products
        $products = Product::all();
        foreach ($products as $product) {
            $sitemapUrls[] = [
                'loc' => "https://mcmodels.net/products/{$product->id}/{$product->slug}",
                'lastmod' => $product->last_updated,
                'changefreq' => 'weekly',
                'priority' => 1,
            ];
        }

        // Fetch all stores
        $stores = Store::all();
        foreach ($stores as $store) {
            $sitemapUrls[] = [
                'loc' => "https://mcmodels.net/vendors/{$store->id}/{$store->slug}",
                'lastmod' => $store->last_updated,
                'changefreq' => 'weekly',
                'priority' => 1,
            ];
        }

        return response()->json($sitemapUrls);
    }
}

class SitemapController
{
    public function index(): JsonResponse
    {
        $sitemapUrls = [];

        // Fetch all products
        $products = Product::all();
        foreach ($products as $product) {
            $sitemapUrls[] = [
                'loc' => "https://mcmodels.net/products/{$product->id}/{$product->slug}",
                'lastmod' => $product->last_updated,
                'changefreq' => 'weekly',
                'priority' => 1,
            ];
        }

        // Fetch all stores
        $stores = Store::all();
        foreach ($stores as $store) {
            $sitemapUrls[] = [
                'loc' => "https://mcmodels.net/vendors/{$store->id}/{$store->slug}",
                'lastmod' => $store->last_updated,
                'changefreq' => 'weekly',
                'priority' => 1,
            ];
        }

        return response()->json($sitemapUrls);
    }
}

I don't reckon the problem is there, because it generates fine and returns at that endpoint, and works at localhost.

oneeach•5mo ago

sitemap is not a default nuxt config value, are you using a sitemap plugin or module to generate the file and use the config setting? https://nuxt.com/docs/api/nuxt-config

JoshuaOP•5mo ago

Yes sorry I should have clarified https://nuxtseo.com/sitemap/api/config

Nuxt SEO

Config · Nuxt SEO

Configure the sitemap module.

JoshuaOP•5mo ago

The official nuxt sitemap module

JoshuaOP•5mo ago

Followed the docs and this here: https://nuxtseo.com/sitemap/getting-started/data-sources#_2-runtime-provide-a-sources-array

Nuxt SEO

Data Sources · Nuxt SEO

Learn how the Nuxt Sitemap sources work.

oneeach•5mo ago

I have not used that one, I've used the one from nuxt: https://nuxt.com/modules/sitemap

Nuxt

Sitemap · Nuxt Modules

Powerfully flexible XML Sitemaps that integrate seamlessly.

JoshuaOP•5mo ago

Click the documentation linked here 😛 https://nuxt.com/modules/sitemap#documentation

Nuxt

Sitemap · Nuxt Modules

Powerfully flexible XML Sitemaps that integrate seamlessly.

oneeach•5mo ago

oh I guess that's the same one lol

JoshuaOP•5mo ago

They are the same module I don't use the nuxt server api, much prefer doing it with laravel. But I don't see why that would make a difference since https://api.mcmodels.net/urls is clearly accessible and localhost hits it and constructs a 4000 url sitemap at the same endpoint that production is hitting.

oneeach•5mo ago

have you tired to use an async function to get your urls? https://nuxtseo.com/sitemap/getting-started/data-sources#_1-build-time-provide-a-urls-function

JoshuaOP•5mo ago

I can't get the sitemap data concurrent when I build, I need it to be updated during runtime. I may not build the site for a few days/weeks, but products and stores change daily.

oneeach•5mo ago

do you see anything in your server logs? or do you see a query on your laravel log?

JoshuaOP•5mo ago

Laravel logs are error-free (from anything related to this) xD I did just update the API as well to fix the last modified column being null.

JoshuaOP•5mo ago

again, reflects immediately when running locally

oneeach•5mo ago

I mean on your mcmodels.net server, do you have error in your log there?

JoshuaOP•5mo ago

No I am ssh'ed into my server looking at laravel.log right now, there is no related errors

oneeach•5mo ago

not laravel, your nuxt app

JoshuaOP•5mo ago

Also clean Just a bunch of bots trying to hit wordpress endpoints that don't exist :heh:

oneeach•5mo ago

I'm not sure, maybe create an issue on the sitemap gitlab page?

JoshuaOP•5mo ago

Yeah I will be creating an issue if this thread doesn't work out. It's such a basic thing I presumed it to be a config issue or something. Or some rule that the sitemap won't pull from the same domain or something silly I put in a github issue, I doubt its a bug with a module but I don't know what else to do.

Broken•5mo ago

Hey! Have you found a workaround? Facing the same issue rn

JoshuaOP•5mo ago

I did find a workaround last night @Broken not super happy with it but it works fine

    sources: [
        '/api/__sitemap__/urls'
    ],

    sources: [
        '/api/__sitemap__/urls'
    ],

I changed my nuxt.config.ts to have this as the source, and then inside that file:

export default defineSitemapEventHandler(async () => {
    try {
        const response = await fetch('https://backend.api.url/urls');
        const data = await response.json();

        if (Array.isArray(data)) {
            return data;
        } else {
            console.error('Unexpected data format from api.mcmodels.net/urls');
            return [];
        }
    } catch (error) {
        console.error('Error fetching sitemap data:', error);
        return [];
    }
})

export default defineSitemapEventHandler(async () => {
    try {
        const response = await fetch('https://backend.api.url/urls');
        const data = await response.json();

        if (Array.isArray(data)) {
            return data;
        } else {
            console.error('Unexpected data format from api.mcmodels.net/urls');
            return [];
        }
    } catch (error) {
        console.error('Error fetching sitemap data:', error);
        return [];
    }
})

Basically just a wrapper makes it work for some reason.

Broken•5mo ago

Nice! Thank you @Joshua Trying it

Broken•5mo ago

So I guess you've created a file named urls in the __sitemap__ folder? @Joshua

JoshuaOP•5mo ago

yes

Broken•5mo ago

ok nice trying in production

JoshuaOP•5mo ago

the data coming back from my endpoint is already formatted for a sitemap though keep in mind if you just return url strings you may need to do some transforming

Broken•5mo ago

Before I also had a wrapper but the route wasn't __sitemap__/urls yep me too

JoshuaOP•5mo ago

kk cool

Broken•5mo ago

trying this

JoshuaOP•5mo ago

Yeah looks right, if urls.ts has what I sent.

Broken•5mo ago

Still has no urls in it 😦 https://app.radardrop.ai/airdrops-sitemap.xml Will dig more tomorrow Thank you for your help @Joshua

// /server/api/__sitemap__/urls.ts
export default defineSitemapEventHandler(async () => {
  const runtimeConfig = useRuntimeConfig();
  const baseUrl = runtimeConfig.public.apiBaseURL;

  return await $fetch<{ loc: string; lastmod: Date }[]>(
    `${baseUrl}/v1/airdrops/sitemap/urls`,
  );
});

// /server/api/__sitemap__/urls.ts
export default defineSitemapEventHandler(async () => {
  const runtimeConfig = useRuntimeConfig();
  const baseUrl = runtimeConfig.public.apiBaseURL;

  return await $fetch<{ loc: string; lastmod: Date }[]>(
    `${baseUrl}/v1/airdrops/sitemap/urls`,
  );
});

// service called
  public async getUrls() {
    const appOrigin = this.configService.get('APP_ORIGIN');
    const airdrops = await this.repo.find({ order: { temperature: 'DESC' } });

    return airdrops.map((airdrop) => ({
      loc: `${appOrigin}/airdrops/${airdrop.slug}`,
      lastmod: airdrop.updatedAt,
    }));
  }

// service called
  public async getUrls() {
    const appOrigin = this.configService.get('APP_ORIGIN');
    const airdrops = await this.repo.find({ order: { temperature: 'DESC' } });

    return airdrops.map((airdrop) => ({
      loc: `${appOrigin}/airdrops/${airdrop.slug}`,
      lastmod: airdrop.updatedAt,
    }));
  }

Digging more tomorrow It’s weird, the api is working just fine https://app.radardrop.ai/api/__sitemap__/urls

JoshuaOP•5mo ago

Yeah I really do think there’s an oddity with the module

Broken•5mo ago

ok I have this very (verbose....?) error

Gaming

Programming

Dynamic Sitemap Generation not working in production