Skip to content

new: [expansion] NetAPI Lookup: compromised feed listing and Top 1M rank - #1014

Open
slavaolesik wants to merge 1 commit into
MISP:mainfrom
slavaolesik:netapi-expansion
Open

slavaolesik wants to merge 1 commit into
MISP:mainfrom
slavaolesik:netapi-expansion

Conversation

@slavaolesik

Copy link
Copy Markdown

New expansion + hover module: NetAPI Lookup

Adds misp_modules/modules/expansion/netapi.py, enriching domain, hostname, url, ip-src, ip-dst and domain|ip attributes from two free methods of the NetAPI JSON API, returning misp_standard results built with PyMISP.

No API key. The lookups are anonymous (30 requests a minute and 1,000 a day per IP). An optional token only raises those limits, so moduleconfig is ["api_token", "api_url"] and requirements is empty.

Behaviour

  • compromised-check says whether the value is in the NetAPI feed of hosts that public threat feeds (URLhaus, CERT Polska, blackbook and others) report for malware or phishing: listed now (seen within 24 hours), listed in the past only, or never listed, with first and last seen, the reporting feed and the registrar.
  • A hostname is matched through its registered domain, and the result says so (matched entry: example.com (the registered domain)) instead of presenting it as a listing of the hostname itself.
  • A past listing is labelled as past. Most of those are cleaned-up sites, so it is not reported as current.
  • A value that was never listed returns a text attribute saying so, not an error: for a reputation lookup that is an answer.
  • For domains, hostnames and URLs, domain-rank adds the position of the registered domain in the NetAPI Top 1M. A top-ranked domain in the feed is more likely a compromised legitimate site than attacker infrastructure. If the rank call fails, that is reported next to the listing instead of dropping the listing.
  • A URL is looked up by its host. All attributes are to_ids: false with correlation disabled. Only the queried indicator leaves the MISP instance.

Measured output

domain     000359.xyz          -> listed now, blackbook, registrar Alibaba Cloud | not in the Top 1M
hostname   www.000359.xyz      -> listed now, matched entry 000359.xyz (the registered domain)
domain     github.com          -> not listed | Top 1M #11
ip-dst     1.1.1.1             -> listed in the past (last seen 2026-06-17), urlhaus.abuse.ch
domain     not_a_host          -> error: Not a valid domain name or IP address.

Tests

tests/test_netapi.py, 13 tests, no network access: current, past and clean listings, the registered-domain match, URL and domain|ip handling, the token header only when configured, API errors and rate limits passed through with their message, an unreachable API degrading to an error, and a failed rank lookup keeping the listing. Clean under the repository's black, isort and flake8 settings, and loaded and queried through a local misp-modules server.

Like the ScanMalware PR, I have not committed regenerated documentation/ output, since documentation/generate.py run locally drops modules whose optional dependencies are missing. The logo is included because moduleinfo references it.

Disclosure: I run NetAPI, so this is a vendor-submitted module.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant