Skip to content

Fix libtiff license download blocked by GitLab Cloudflare challenge - #102

Open
Silvio Angelo (silvioangels) wants to merge 1 commit into
microsoft:mainfrom
silvioangels:fix/gitlab-libtiff-cloudflare
Open

Silvio Angelo (silvioangels) wants to merge 1 commit into
microsoft:mainfrom
silvioangels:fix/gitlab-libtiff-cloudflare

Conversation

@silvioangels

Copy link
Copy Markdown

Summary

This PR fixes a failure in the Skill Recorder compliance preparation flow when retrieving the reviewed libtiff COPYRIGHT file from GitLab.

The current implementation retrieves the file through GitLab's web raw endpoint:

https://gitlab.com/libtiff/libtiff/-/raw//COPYRIGHT

In the environment where the issue was reproduced, automated requests to this endpoint consistently returned:

HTTP 403 Forbidden
Cf-Mitigated: challenge
Server: cloudflare

This caused npm run compliance:licenses to fail during installation, even though the same resource was accessible through a browser.

Root cause

The failure was isolated to GitLab's /-/raw/ web endpoint.

During troubleshooting, the following behavior was reproduced:

  • The GitLab project page was accessible.
  • GitHub raw resources were accessible.
  • The libtiff COPYRIGHT file was accessible through a browser.
  • Automated Node.js fetch() requests to the GitLab /-/raw/ endpoint returned HTTP 403.
  • curl requests to the same endpoint also returned HTTP 403 with Cf-Mitigated: challenge.
  • Changing the User-Agent did not resolve the challenge.
  • The issue was reproduced across different network connections.
  • GitLab's Repository Files API successfully returned the same COPYRIGHT file.

The GitLab API endpoint used is:

https://gitlab.com/api/v4/projects/libtiff%2Flibtiff/repository/files/COPYRIGHT/raw?ref=

Most importantly, the file returned by the GitLab API produced the exact SHA-256 already expected by Skill Recorder's reviewed compliance policy.

No reviewed hashes or compliance policy values were changed.

Solution

This change adds an optional rawApiBase to the libtiff source definition.

When rawApiBase is available, buildTesseractNoticeSpecs() constructs the legal-material URL using GitLab's Repository Files API:

https://gitlab.com/api/v4/projects/libtiff%2Flibtiff/repository/files//raw?ref=

Sources without rawApiBase continue using the existing rawBase behavior.

This intentionally scopes the behavior change to libtiff while preserving the existing URL construction behavior for the other Tesseract dependencies.

Compliance and integrity

This change does not:

  • change the reviewed libtiff revision;
  • change the expected SHA-256;
  • bypass legal-material validation;
  • disable compliance checks;
  • relax archive or legal-material validation.

During troubleshooting, the COPYRIGHT file retrieved through the GitLab Repository Files API was verified against the existing reviewed SHA-256 and produced an exact match.

The transport endpoint changes, but the reviewed material and its integrity validation remain unchanged.

Regression test

A regression test was added:

libtiff notice uses the GitLab repository files API

The test verifies that:

  • the generated libtiff legal-material URL uses the GitLab Repository Files API;
  • the generated URL does not fall back to the problematic gitlab.com/libtiff/libtiff/-/raw/ endpoint.

Compliance test results:

  • 34 tests
  • 34 passed
  • 0 failed
  • 0 skipped

Validation performed

The change was validated using Node.js 24.22.1.

The following validation was performed:

Lockfile validation

npm run check:lockfile

Completed successfully.

Dependency installation

npm ci

Completed successfully.

Reviewed Electron setup

npm run electron:install-reviewed

Completed successfully.

Compliance tests

node --test scripts/compliance.test.mjs

Result:

  • 34 tests
  • 34 passed
  • 0 failed
  • 0 skipped

The new regression test passed successfully:

libtiff notice uses the GitLab repository files API

Compliance preparation

npm run compliance:licenses

Completed successfully and prepared the compliance material without reproducing the previous GitLab HTTP 403 failure.

Build

npm run build

Completed successfully.

Runtime validation

npm start

Successfully launched Skill Recorder from the modified source tree.

The application reached the recording UI successfully.

Why use the GitLab Repository Files API?

The GitLab Repository Files API provides an official programmatic endpoint for retrieving repository files and, in this case, does not trigger the Cloudflare challenge encountered by automated clients using the web /-/raw/ endpoint.

Because Skill Recorder continues to verify the retrieved legal material against the existing reviewed SHA-256, changing the retrieval endpoint does not change the trusted compliance artifact.

Scope

The change is intentionally limited to libtiff.

Other Tesseract source definitions continue using their existing rawBase URLs unchanged.

No reviewed hashes or compliance-policy values are modified by this PR.

@silvioangels

Copy link
Copy Markdown
Author

@microsoft-github-policy-service agree

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant