Masscan: TCP port scanner, scanning entire Internet in under 5 minutes

[wfn]

A very nice and well-built tool. Excellent use of sequence numbers for stateless send/receive (SYN cookies): https://github.com/robertdavidgraham/masscan/blob/master/src... (very well-documented source too, a pleasure to read).

I once (2016) used it to scan port 22 on the whole ipv4 (had to experiment with rate limits to not trigger alarms and get complaints forwarded by my VPS provider; the clever ip+port randomization technique helps a lot with that). Then took the ~22m (iirc) IPs which responded and ran ssh-keyscan on them to extract and analyze some ~15m ssh banners and public keys (a bunch of them broken, through debianized PRNG etc.) I think most of the scanning + extraction was done overnight, via ~13 VPS rented hourly (whole thing cost < $10, and very few complaints). Fun times :) I should write it up some time, and do it again.

[DLA]

"A very nice and well-built tool. .... (very well-documented source too, a pleasure to read)" Could not agree more. This code is really nicely organized and commented. Such a pleasure to read and learn about how this tool works. I wish more codebases had comments like this! Great work and thank you @ErrataRob.

[dosshell]

What is "22m" ?

22 meter?

22 millies of something.

I do not follow?

[mike_d]

A quick plug for my friends over at GreyNoise... they have honeypots all over the internet and identify various scanners as well as their observed intentions.

Here is everyone running masscan against the internet: https://www.greynoise.io/viz/query/?gnql=tags%3A%22Masscan%2...

[mod]

All the results on the first page (as much as it would show me) are just VPS providers.

[unixhero]

Warning. Masscan is the best scanner I have ever seen.

We have also managed to take down the entire corporate network by using it with a too high rate limit. So tread lightly around massscan and its power. Our pentesters did the same 6 months later. The managed service provider is not able to solve the routing table loop which causes the firewall to DOS when a rapid masscan is triggered.

[Socketier]

Yeah I managed to do the same, it filled up the firewall state table and nobody was able to initiate new connections, only existing ones continued to work. And that's how I learned about stateful firewalls!

[LeonidBugaev]

Question: how do you actually run it without getting banned?

I've got banned pretty hard by both my local home ISP and using Linode servers, when tried such scanners. Mass port scanning is easy to track, and it usually forbidden all ISP ToC.

[jonaslejon]

You need to have a really good relationship with your ISP and get their acknowledge prior to the scanning

[Ms-J]

Use a no-logs, anonymous VPN. I do it all the time.

[jihadjihad]

If masscan is of interest to you, be sure to check out zmap [0] as well. It can scan the entire IPv4 address space in around 45 minutes.

0: https://github.com/zmap/zmap

[ABraidotti]

I enjoy Rob Graham on Twitter too: https://twitter.com/ErrataRob

He often discusses current events in infosec if you're into that.

[amatecha]

Yeah, I appreciate his willingness to share "inconvenient truths" that may run counter to the running narrative of whatever tech/security story of the moment. Lots of interesting perspectives, even if maybe I didn't "want" to hear it ;)

[danilonc]

Old discussing about the tool:

https://news.ycombinator.com/item?id=8803498

[dang]

Thanks! Expanded:

Some idiot is using your tool to mass scan our network - https://news.ycombinator.com/item?id=24728123 - Oct 2020 (182 comments)

MASSCAN: Mass IP port scanner - https://news.ycombinator.com/item?id=12260809 - Aug 2016 (33 comments)

Masscan: Scan the entire Internet in under 5 minutes - https://news.ycombinator.com/item?id=8803498 - Dec 2014 (29 comments)

Masscan: scan the entire Internet in under 6 minutes, 10 million packets/second - https://news.ycombinator.com/item?id=6391266 - Sept 2013 (30 comments)

Masscan: The entire internet in 3 minutes - https://news.ycombinator.com/item?id=6388222 - Sept 2013 (12 comments)

[Aissen]

Awesome tool. Rob added IPv6 support last year, which can be really useful if you know what you're doing (ex: want to scan a single subnet for given OUI without privacy extensions).

[sneak]

FWIW I think the late Dan Kaminsky is the person who originally invented this two process userspace TCP scan technique 2001-2002ish in his tool Paketto Keiretsu.

It's a good one.

I sometimes wonder if it inspired some of the TCP-in-userspace stuff that is done in go (gvisor lib, I think it was).

I'm glad Rob is continuing the development of this idea and continues to scan the whole internet. Scanning the whole internet is cool. It's a shame it's de facto illegal these days (and will get most internet connections in the USA terminated near-instantly).

[tptacek]

Decoupled sender/receiver scanners date back to the 1990s (we did one in CASL, our misbegotten packet programming language from 1997, the script for which I think shipped with Ballista --- and we stole the scanner design from someone else, probably from CORE SDI), as do userland TCP libraries (LWIP is a famous one, or at least a library that got used for that purpose, that predates all this stuff).

[xinniethepooh]

Source on it being de-facto illegal?

I've not seen anything that says simply scanning for open ports is illegal, doing vulnerability scans may be though.

[samstave]

>the late Dan Kaminsky

Ehhh... I forgot he was the "the late Dan Kaminsky"

:-(

[_wldu]

I wrote netscan years ago and still use it. It's pretty fast too: https://github.com/62726164/netscan

[bobek]

Massacan is awesome. One of the usecases is to periodically scan your own servers to see if you have not accidentally opened some new ports in firewalls.

https://github.com/bobek/masscan_as_a_service

[chaz6]

I am not sure any computer is capable of sending and processing 2^126 packets (assuming global unicast, 1 packet out, 1 packet in) in under 5 minutes.

[jerf]

Well, 5 minutes is about 2^8 seconds, you can get about 2^32 cycles per core per second nowadays, and you can have about 2^6 cores maximum, so with an incredibly, unspeakably generous "scan one IP in one cycle" a 64-core computer should be able to scan about 2^(8+32+6) = 2^46 addresses in a little under five minutes.

We seem to be a wee bit short of 2^126, yes.

[helge9210]

You can do it in parallel from several computers. Masscan has capability of dividing workload across multiple processes.

[birdyrooster]

Definitely but then click bait wouldn’t be click bait without some misleading information

[zamadatix]

"Only" about a /15 worth of v6 is actively advertised on the internet. Still, good luck sending that many packets.

[dmw_ng]

The v6 space has structure and is much smaller than that

[OnlyMortal]

Amateur!

[londons_explore]

Impressive that the bitcoin donation address has received $20k!

[sva_]

Most of it was sent when Bitcoin was worth a lot less.

[dmos62]

I like that you checked it!

[trutannus]

Funny this would show up here. I keep seeing this on a bot profiling project of mine. Infrequent requests, but persistent.

[SavantIdiot]

Rather than have everyone who can run this (without getting banned) actually run it, couldn't someone just post the results to a file for us once per day or week? Seems inefficient to have millions of people run this when it results in (mostly) the same data.

[nbk_2000]

Not as a file but as a searchable index, yes. Here's a few: shodan.io spyse.com zoomeye.org

[palebluedot]

try https://search.censys.io

[johnnyApplePRNG]

Was curious the other day and couldn't find a real answer... does AWS allow portscanning from their infrastructure?

All I could find was statements from them that they do not allow port scanning OF their infrastructure.

[Cantinflas]

They don't, I got a company acc banned scanning my own local infra from an aws instance

[samstave]

Sort-of.

You can port scan your own infra/vpcs etc -- but YOU MUST tell them you are doing so and why you are doing it, else they will block it.

again, as with anything AWS (and other providers) have a good rapport with your rep, and SEs in AWS and you have a lot more freedom than you expect just from boilerplate ULA TOS stuff.

You may not scan anything other than your own infra. And you can get your external monitors whitelisted as well...

Just talk to your rep.

[mindcrime]

And if Amazon doesn't, are there any hosting providers that do allow port-scanning from their infrastructure? I'm assuming the big providers like AWS, Google and Azure all have these limits in place, but I wonder if you can do it from some of the smaller providers? Even if means going down to some fly-by-night outfit in Belize or something...

[helge9210]

Practically, no. Unsure about actual implementation, but looks like packets disappear right after leaving the interface.

[dadrian]

If you bring your own IPs they don't care. However, the bandwidth fees are egregious.

[nunez]

Perfect for finding publicly-accessible Kubernetes API servers.

[1-6]

I created a honeypot once and they started crypto mining on mine.

[bogomipz]

The author states:

>"A mutex on the fast path of a program severely limits scalability. Instead, Masscan uses "rings" to synchronize things, such as when the user-mode TCP stack in the receive thread needs to transmit a packet without interfering with the transmit thread."

Is "rings" here referring to PF_RING mentioned in the preceding paragraph or is it referring to a specific synchronization primitive?

[emerged]

I would have fun optimizing this. The performance discussion toward the end of the readme was pretty basic, no assembly language or multi threading optimizations, etc.

But I have told myself to stop spending time optimizing things for no reason..

[dheera]

> This increases the rate to 100,000 packets/second, which will scan the entire Internet (minus excludes) in about 10 hours per port (or 655,360 hours if scanning all ports).

So 655360 hours, not 5 minutes

[wfn]

Well, keep reading the README, then :) see PF_RING: https://github.com/robertdavidgraham/masscan#pf_ring