discourse-indexnow — Automatically notifies Bing and Yandex after topic creation or editing

Development and Operations
1
:information_source: Overview Automatically submits URLs of new, edited, or deleted public topics to the IndexNow protocol, allowing Bing, Yandex, and other compatible search engines to discover or de-index your content within minutes rather than waiting for crawlers to slowly scrape it.
:hammer_and_wrench: Repository github.com/imlotso/discourse-indexnow
:open_book: Installation Guide How to install plugins in Discourse

discourse indexnow - sitetalk
discourse indexnow - sitetalk1063×1382 75.2 KB

Features

  • Automatically submits a new topic when published; automatically resubmits when edits modify the first post or key topic attributes.
  • Reply-triggered submission: Automatically submits the paginated URL (e.g., ?page=3) when new replies are generated, with support for per-URL cooldown to prevent excessive submissions.
  • Automatically submits the corresponding URL when a topic is deleted, helping search engines detect 404/410 errors faster and accelerating de-indexing.
  • When the site has Content Localization enabled with crawler locale parameters, localized topic URLs are automatically submitted using Discourse’s actual configured locale query parameters—no hardcoded values.
  • Merges the main URL and any existing localized URLs into a single IndexNow urlList batch request instead of submitting them individually.
  • Historical post backfill: Filter topics by category and date range, preview matching results first, then submit in a single batch—reusing the same batch submission engine. If a batch exceeds the protocol’s 10,000-URL per-request limit, it will automatically chunk. Batches still in progress (e.g., from accidentally selecting too large a range) can be cancelled directly from the Logs tab.
  • Manual submission: Paste one URL per line to submit specific links on demand; external links and topics that don’t meet conditions will be automatically filtered out.
  • Smart rate limiting and quota management: Supports hourly/daily submission limits based on a sliding window. Automatically backs off on IndexNow 429 responses (prioritizing the Retry-After header). The admin dashboard provides an intuitive quota progress bar and a countdown to quota replenishment.
  • Supports excluding specified categories and tags, adding a business-layer exclusion rule on top of automatic privacy filtering.
  • Supports key rotation (with a confirmation dialog to prevent accidental triggers); generating a new key immediately invalidates the old one.
  • Admin dashboard includes a built-in check for public accessibility of /<key>.txt.
  • When topics are moved between categories, category visibility changes, or tags are updated, the system automatically resubmits or excludes in the background without blocking the page during bulk operations.
  • Logs record batch ID, locale, trigger reason (create/edit/delete/category change/backfill/manual), success/failure trends over the past 7 days, and failure reason categories (rate limit / key error / domain mismatch / other).
  • All submissions run asynchronously and share the same privacy filtering logic as automatic submissions—private messages, restricted categories, unlisted/deleted topics, and sites requiring login are never submitted.
  • The admin interface is available in English and Simplified Chinese.

Setup Steps

  1. Install the plugin following the guide above, then rebuild the container.
  2. Navigate to Admin > Plugins > discourse-indexnow.
  3. On the Settings tab, click Generate key (or enter a 32-character hexadecimal key you’ve already registered elsewhere).
  4. Check Enable IndexNow submissions.
  5. Verify the key is publicly accessible:
https://your-forum-domain/<key>.txt

You should receive HTTP 200 with the key itself as the response body. The admin panel also performs this check automatically, displaying a cached green/red status next to “key.txt accessible.”
6. (Optional) Use the “Backfill Submissions” panel on the Logs tab: select a category and date range, click Preview to see the match count, then click Submit. If a batch is still processing, you can cancel it on the same page.
7. (Optional) Use the manual submission input box to paste one URL per line and submit specific links immediately, outside the regular automatic flow.

The Logs page also displays a quota usage progress bar, a 7-day success/failure trend chart, and failure reason breakdowns, making it easy to see at a glance whether submissions failed due to rate limiting, key mismatch, or another cause.

Configuration Options

Name Description
indexnow_enabled Master plugin toggle. Cannot be enabled when the site has login_required set; if that setting is enabled afterward, the plugin will automatically disable itself.
indexnow_api_key The current 32-character hexadecimal key used for submission verification.
indexnow_submit_on_create Automatically submits when a new public topic is published.
indexnow_submit_on_edit Automatically resubmits after edits to the first post or changes to topic attributes.
indexnow_submit_on_reply Automatically submits the paginated URL where the new reply appears.
indexnow_url_cooldown_minutes Cooldown period (in minutes) per individual URL to prevent spammy re-submissions.
indexnow_excluded_category_ids Additional categories to exclude—even if the category itself is public, topics within it won’t be submitted.
indexnow_excluded_tag_names Additional tags to exclude—even if the topic’s category meets submission criteria, topics with these tags won’t be submitted.
indexnow_hourly_limit Maximum number of URLs submitted per hour (based on a sliding window).
indexnow_daily_limit Maximum number of URLs submitted per day (based on a sliding window).

Known Limitations

  • Google does not currently participate in the IndexNow protocol.
  • Localized URLs are only submitted when the crawler locale parameter is enabled and the topic actually has localized content available.
  • Edits to the first post and subsequent reply pagination changes are supported, but individual edits to middle posts do not currently trigger a separate submission.

This is an independently developed plugin and is not officially maintained by Discourse. It is already running in production on my own forum, with RSpec tests and CI running on every push/PR. Please feel free to report issues and feature requests via GitHub issues or replies to this thread.