Skip to content

RDoc-4138 Search results link to random language, RDoc-4148 Sync Algolia config automatically & place it inside git - #2710

Open
poissoncorp wants to merge 2 commits into
ravendb:mainfrom
poissoncorp:RDoc-4138-search-results-language
Open

poissoncorp wants to merge 2 commits into
ravendb:mainfrom
poissoncorp:RDoc-4138-search-results-language

Conversation

@poissoncorp

@poissoncorp poissoncorp commented Oct 22, 2026 •

Copy link
Copy Markdown
Contributor

Issue link

RDoc-4138 Search results link to random language

RDoc-4148 Sync Algolia config automatically & place it inside git

Additional description

The crawler indexed each page's C# HTML under a random ?lang= URL, so search results opened a random language and saved it as the reader's preference.

  • Crawler config and index settings in scripts/lib/algolia-*, as live, plus Quill results grouped under "Quill" with its landing page excluded, and initialIndexSettings fixed (was keyed ProductioDocsCrawler)
  • ignoreQueryParams: ["lang"], so each page is indexed under its plain URL
  • scripts/sync-algolia.ps1 pushes them on a diff and recrawls after a crawler change; deploy.ps1 runs it after the CloudFront invalidation, skipped without ALGOLIA_* credentials

Verified: -DryRun against the live crawler and index shows only the intended diff. The write path first runs on deploy.

Type of change

  • Content - docs
  • Content - cloud
  • Content - Quill
  • Content - guides
  • Content - start pages/other
  • New docs feature (consider updating /templates or readme)
  • Bug fix
  • Optimization
  • Other

Changes in docs URLs

  • No changes in docs URLs
  • Articles are restructured, URLs will change, mapping is required (update /scripts/redirects.json file, set Documents Moved PR label)

Changes in UX/UI

  • No changes in UX/UI
  • Changes in UX/UI (include screenshots and description)

@poissoncorp
poissoncorp force-pushed the RDoc-4138-search-results-language branch 2 times, most recently from f3c87ed to 9aa02c6 Compare October 23, 2026 00:02
@poissoncorp
poissoncorp requested a review from Lwiel October 23, 2026 00:54
@poissoncorp
poissoncorp force-pushed the RDoc-4138-search-results-language branch from 9aa02c6 to ccb9bf2 Compare October 23, 2026 01:13
Comment thread scripts/deploy.ps1
Comment on lines +280 to +287
# After the invalidation, so a recrawl reads this release's HTML; errors are held like the CloudFront sync.
Write-Host 'Syncing Algolia configuration' -ForegroundColor Cyan
try {
& $SyncAlgoliaPath
} catch {
if (-not $configError) { $configError = $_ }
Write-Host " Algolia config sync failed: $_" -ForegroundColor Red
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We don't wait for the invalidation to complete before executing this

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should also check $LASTEXITCODE here

Comment thread scripts/sync-algolia.ps1
. (Join-Path $PSScriptRoot 'lib/cloudfront-common.ps1')

# node prints non-ASCII (the breadcrumb separator).
[Console]::OutputEncoding = [Text.UTF8Encoding]::new()

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we restore the default encoding at the end?


const indexSettings = require("./algolia-index-settings.json");

const INDEX_NAME = "ProductionDocsCrawler";

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We already have it in docusaurus.config.ts, I'd rather keep this in some common place

};

const CRAWLER_CONFIG = {
appId: "GYTCYX561T",

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We already have it in docusaurus.config.ts, I'd rather keep this in some common place

],
safetyChecks: {
beforeIndexPublishing: {
maxLostRecordsPercentage: 50,

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we should increase this value for a single run, we'll drop a lot of index entries with this PR

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants