The Flip Side of memcrashed

Rapid7 Labs keeps a keen eye on research and findings from other savvy security and technology organizations and noticed Cloudflare’s report on new distributed denial of service (DDoS) amplification attacks using memcached. If you haven’t read Cloudflare’s (excellent) analysis yet, the TLDR is, memcached over UDP makes for an ideal amplifier — the spoofed source requests from an attacker are tiny, and the resulting replies to the spoofed source can be enormous.

Rapid7’s Project Sonar sees well over 100,000 exposed memcached servers at any given time

Global memcached Node View

Hilbert map memcached Node View

That’s quite a spread of potential DDoS soldiers just sitting and waiting to be brought into the amplification army.

Since we perform both active and passive internet information and intelligence gathering, we also took a look at the data from our Heisenberg Cloud honeypot agent network thinking we’d see somewhat similar activity to that of Cloudflare. What we found was far more interesting (and inspired this post).

On February 20th (about four days before Cloudflare’s reported attack), we saw a spike in memcached probes:

When we correlated the source IPv4s with our Sonar data we noticed that none of the IPv4s talking to Heisenberg were in the memcached data set.

Our source lists are also very different:

Country Number of nodes
United States 257
China 108
Russia 8
Romania 7
Seychelles 6
United Kingdom 6
France 4
Germany 3
Iran 3
Netherlands 3
Other 10
ASO AS # Unique IPs
Hurricane Electric, Inc. AS6939 189
No.31,Jin-rong Street AS4134 51
CNCGROUP China169 Backbone AS4837 39
LeaseWeb Netherlands B.V. AS60781 36
Quasi Networks LTD. AS29073 8
Flokinet Ltd AS200651 7
China Unicom Shanghai network AS17621 5
Digital Ocean, Inc. AS14061 5
B2 Net Solutions Inc. AS55286 4
Steadfast AS32748 4
Other Other 54

Rapid7’s early warning system caught the protocol probes for active/exposed memcached servers just a few days before the amplification attacks started. Since we just track payloads and connections to 11211 and do not try to emulate a full memcached server, the bot herders mostly left us alone, though we are still tracking more elevated probe counts than we were seeing before the DDoS campaign began.

We have a better picture of what infrastructure is going into this novel DDoS campaign and must echo Cloudflare’s advice: double check your use of memcached and secure your configurations.

We have a Metasploit module in the works that will scan for and identify memcached instances that are vulnerable to amplification attacks, so keep an eye out!

Banner image "Summer Time on Loop" by Dane Deaner