Repository navigation
new: [expansion] NetAPI Lookup: compromised feed listing and Top 1M rank - #1014
Open
slavaolesik wants to merge 1 commit into
Open
slavaolesik wants to merge 1 commit into
slavaolesik wants to merge 1 commit into
Conversation
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
New expansion + hover module: NetAPI Lookup
Adds
misp_modules/modules/expansion/netapi.py, enrichingdomain,hostname,url,ip-src,ip-dstanddomain|ipattributes from two free methods of the NetAPI JSON API, returningmisp_standardresults built with PyMISP.No API key. The lookups are anonymous (30 requests a minute and 1,000 a day per IP). An optional token only raises those limits, so
moduleconfigis["api_token", "api_url"]andrequirementsis empty.Behaviour
compromised-checksays whether the value is in the NetAPI feed of hosts that public threat feeds (URLhaus, CERT Polska, blackbook and others) report for malware or phishing: listed now (seen within 24 hours), listed in the past only, or never listed, with first and last seen, the reporting feed and the registrar.matched entry: example.com (the registered domain)) instead of presenting it as a listing of the hostname itself.domain-rankadds the position of the registered domain in the NetAPI Top 1M. A top-ranked domain in the feed is more likely a compromised legitimate site than attacker infrastructure. If the rank call fails, that is reported next to the listing instead of dropping the listing.to_ids: falsewith correlation disabled. Only the queried indicator leaves the MISP instance.Measured output
Tests
tests/test_netapi.py, 13 tests, no network access: current, past and clean listings, the registered-domain match, URL anddomain|iphandling, the token header only when configured, API errors and rate limits passed through with their message, an unreachable API degrading to an error, and a failed rank lookup keeping the listing. Clean under the repository'sblack,isortandflake8settings, and loaded and queried through a localmisp-modulesserver.Like the ScanMalware PR, I have not committed regenerated
documentation/output, sincedocumentation/generate.pyrun locally drops modules whose optional dependencies are missing. The logo is included becausemoduleinforeferences it.Disclosure: I run NetAPI, so this is a vendor-submitted module.