How to generate sitemap for NextJs Projects

3rd Aug 2020

Nextjs is the best framework out there to create server side rendered react apps. Now with the support of pre-rendered pages and static export, its on the way to become ultimate tool for both dynamic and static content generation.

Sitemaps are important requirement for generating good SEO. It provides the information to search engines about your website and what urls to crawl.

NextJS doesn't provide native support for sitemap generation. So our best option is to write a custom solution or use an existing packages like next-sitemap. (PS: I'm the author of this package)

Source Code

Documentation

Example Project

Summary

In this tutorial we will create a nextjs app that generates 10000+ pre-rendered pages and create sitemaps and robots.txt for the generated contents.

Project Setup

Let's start by creating a sample next.js project. You can skip this setup, if you already have an existing project.

yarn add next react react-dom

Now lets create an index page and pre-rendered page.

import React from 'react'

const HelloWorld: React.FC = () => {
  return (
    <div>
      <h1>HelloWorld Component</h1>
    </div>
  )
}

export default HelloWorld

Add the NextJs build script

{
  "scripts": {
    "dev": "next",
    "build": "next build"
  }
}

Let's also add a pre-rendered page which will generate 10000 static pages.

import React from 'react'

const DynamicPage = () => {
  return (
    <div>
      <h1>DynamicPage Component</h1>
    </div>
  )
}

export const getStaticProps = async () => {
  return {
    props: {
      dynamic: 'hello',
    },
  }
}

export const getStaticPaths = async () => {
  return {
    paths: [...Array(10000)].map((_, index) => ({
      params: {
        dynamic: `page-${index}`,
      },
    })),
    fallback: false,
  }
}

export default DynamicPage

We are now ready to add the sitemap generator package to our project.

Add next-sitemap package

Add next-sitemap with

yarn add next-sitemap -D

next-sitemap requires a basic configuration file under your project root. Create next-sitemap.js with following contents.

module.exports = {
  siteUrl: 'https://www.example.com',
  generateRobotsTxt: true,
}

Add postbuild script on package.json for triggering sitemap generate every time our project gets build.

{
  "scripts": {
    "dev": "next",
    "build": "next build",
    "postbuild": "next-sitemap"
  }
}

Congrats!! Now you are ready to build your project and dynamically create sitemaps and robots.txt. Let's move on to building the project.

Build the project

yarn build

Now there is a public directory with robots.txt, sitemap.xml & sitemap-*.xml. These sitemaps contains absolute links to all static and pre-rendered pages and they are being split into chunks of 5000 links per file.

Generated robots.txt will look like this.

User-agent: *
Allow: /
Host: https://www.example.com
Sitemap: https://www.example.com/sitemap.xml
Sitemap: https://www.example.com/sitemap-1.xml
Sitemap: https://www.example.com/sitemap-2.xml

For additional configurations such as user-agent policies, custom-sitemaps, chunk-size etc. Please check the next-sitemap Documentation