Repository navigation
Add URL shortener resolution expansion - #1012
Merged
Merged
Conversation
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: aca7f766f1
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Resolve known URL-shortener attributes into their final HTTP(S) destination with a new
url_shortenerexpansion module. Return the original attribute and a standardshortened-linkobject containing the original URL, final URL, shortener hostname, and ordered redirect history. The existing object template covers this case, so no new template is required.Follow HTTP 301/302/303/307/308 redirects using HEAD, with GET fallback on 405/501 and no response-body downloads. Support additional exact-match shortener hostnames, configurable request timeouts and redirect limits, and relative redirect locations. Normalize Unicode hostnames using non-transitional IDNA2008/UTS #46 with strict label validation, preserving the same hostname for allowlist matching, DNS, HTTP Host, and TLS. Validate every hop against all DNS answers, pin connections to a validated public address, verify TLS hostnames, and reject loops, credentials, unsafe destinations, and HTTP failures.
Includes module documentation, installation/configuration guidance, generated documentation entries, and 112 module tests.
Closes #260.
Validation:
python -m pytest -q tests/test_url_shortener.py tests/test_import_external_reference.py tests/test_webarchive.py— 203 passed, including regression coverage for sharp s, final sigma, UTS Make misp-modules really asynchronous #46 mapping, invalid joiners, and invalid A-labels.git diff --checkpassed.poetry check --lockand wheel build passed; the wheel explicitly requiresidna>=3.10,<4, and existing locked package versions are unchanged.