Sitemap

Learn how to create a sitemap in your Indie Kit application

Sitemap

Great news! Indie Kit automatically handles sitemap generation for your application! πŸŽ‰

Your sitemap is automatically created and accessible at /sitemap.xml. It includes all your static pages, blog posts, policy pages, and documentation pages without any setup required.

Why Sitemaps Matter 🎯

Having a sitemap is crucial for SEO because it:

  • πŸ” Helps Search Engines Discover Content: Ensures all your pages are found and indexed by search engines like Google, Bing, and others
  • ⚑ Speeds Up Indexing: New pages and updates are discovered faster, improving your site's visibility
  • πŸ“Š Improves Crawl Efficiency: Tells search engines which pages are most important and how often they change
  • πŸ—ΊοΈ Provides Structure: Gives search engines a clear map of your site's architecture and content hierarchy
  • πŸ“ˆ Boosts Rankings: Better indexing means better chances of ranking for relevant searches
  • πŸ†• Prioritizes Fresh Content: Helps search engines know when content is updated, especially for blogs and dynamic pages

Without a sitemap, search engines might miss important pages, leading to lower visibility and reduced organic traffic.

What's Included by Default πŸ“¦

Indie Kit's sitemap automatically includes:

  • βœ… Static pages (home, about, contact, pricing, etc.)
  • βœ… Policy pages (privacy, terms, refund, cookie)
  • βœ… Blog posts (with creation dates)
  • βœ… Documentation pages
  • βœ… Proper metadata (lastModified, changeFrequency, priority)

How It Works πŸ”§

Indie Kit's sitemap generation is handled automatically through src/app/sitemap.ts. The system:

  • πŸ“„ Collects all static pages (home, about, contact, etc.) with appropriate priority levels
  • πŸ“‹ Gathers policy pages (privacy, terms, refund, cookie) with standard metadata
  • πŸ“ Fetches all blog posts with their creation dates
  • πŸ“š Includes all documentation pages from your docs directory
  • βš™οΈ Combines everything with proper metadata (lastModified, changeFrequency, priority)
  • πŸš€ Automatically regenerates when you deploy or add new content

Zero Configuration Required

Everything works out of the box! Your sitemap is automatically generated and kept up-to-date without any manual intervention.

Customizing Your Sitemap 🎨

Want to add your own pages? Simply update src/app/sitemap.ts:

// Add your custom pages to the staticPages array
const staticPages = [
  "",
  "/about",
  "/contact",
  "/join-waitlist",
  "/blog",
  "/pricing",
  "/features",      // ✨ Your new page
  "/testimonials",  // ✨ Your new page
  "/case-studies",  // ✨ Your new page
].map((route) => ({
  url: `${baseUrl}${route}`,
  lastModified: new Date(),
  changeFrequency: "monthly" as const,
  priority: route === "" ? 1 : 0.8,
}));

Programmatic SEO (pSEO) πŸš€

Want to include database items in your sitemap? This is perfect for programmatic SEO! Here's an example with a "projects" database:

import { MetadataRoute } from "next";
import { getAllBlogs } from "@/lib/mdx/blogs";
import { db } from "@/db"; // Your database instance
import { projects } from "@/db/schema"; // Your projects table

export default async function sitemap(): Promise<MetadataRoute.Sitemap> {
  const baseUrl = process.env.NEXT_PUBLIC_APP_URL!;
  const blogs = await getAllBlogs();
  
  // Fetch all projects from database
  const allProjects = await db.select().from(projects);

  // Static pages
  const staticPages = [
    "",
    "/about",
    "/contact",
    "/projects", // Projects listing page
  ].map((route) => ({
    url: `${baseUrl}${route}`,
    lastModified: new Date(),
    changeFrequency: "monthly" as const,
    priority: route === "" ? 1 : 0.8,
  }));

  // Blog pages
  const blogPages = blogs.map((blog) => ({
    url: `${baseUrl}/blog/${blog.slug}`,
    lastModified: new Date(blog.frontmatter.createdDate),
    changeFrequency: "weekly" as const,
    priority: 0.6,
  }));

  // 🎯 Project pages from database
  const projectPages = allProjects.map((project) => ({
    url: `${baseUrl}/projects/${project.slug}`,
    lastModified: project.updatedAt || project.createdAt,
    changeFrequency: "weekly" as const,
    priority: 0.7,
  }));

  return [...staticPages, ...blogPages, ...projectPages];
}

Generating Multiple Sitemaps πŸ“š

For large sites with 50,000+ URLs, you can generate multiple sitemaps:

import type { MetadataRoute } from 'next'
import { BASE_URL } from '@/app/lib/constants'

export async function generateSitemaps() {
  // Fetch the total number of products and calculate the number of sitemaps needed
  return [{ id: 0 }, { id: 1 }, { id: 2 }, { id: 3 }]
}

export default async function sitemap({
  id,
}: {
  id: number
}): Promise<MetadataRoute.Sitemap> {
  // Google's limit is 50,000 URLs per sitemap
  const start = id * 50000
  const end = start + 50000
  const products = await getProducts(
    `SELECT id, date FROM products WHERE id BETWEEN ${start} AND ${end}`
  )
  return products.map((product) => ({
    url: `${BASE_URL}/product/${product.id}`,
    lastModified: product.date,
  }))
}

This generates multiple sitemaps like:

  • /sitemap/0.xml
  • /sitemap/1.xml
  • /sitemap/2.xml
  • etc.

Customizing Robots.txt πŸ€–

To control search engine crawlers, update src/app/robots.ts:

import { MetadataRoute } from 'next'

export default function robots(): MetadataRoute.Robots {
  return {
    rules: [
      {
        userAgent: '*',
        allow: '/',
        disallow: ['/admin/', '/api/'],
      },
      {
        userAgent: 'Googlebot',
        allow: '/',
        crawlDelay: 2,
      },
    ],
    sitemap: `${process.env.NEXT_PUBLIC_APP_URL}/sitemap.xml`,
  }
}

Pro Tip

You can customize rules for specific user agents. This is useful for controlling how different search engines crawl your site!

Best Practices πŸ’‘

  1. Priority Values

    • Homepage: 1.0 (highest priority)
    • Main pages: 0.8
    • Blog posts: 0.6-0.7
    • Policy pages: 0.5
  2. Change Frequency

    • Static pages: monthly
    • Blog posts: weekly
    • Dynamic content: daily or weekly
  3. Last Modified

    • Use actual creation/update dates for blog posts
    • Use new Date() for static pages
    • Keep database items updated with their modification dates
  4. Testing

    • Visit /sitemap.xml to verify your sitemap
    • Check that all expected URLs are included
    • Validate with Google Search Console

Resources πŸ“š

Learn more about sitemaps in Next.js:

All Set!

Your sitemap is automatically generated and ready to help search engines discover your content! πŸŽ‰