Busting Blocklist Myths

1. Blocklists are old technology

The delivery mechanism for DNSBLs has been around for years. However, the threat intelligence within blocklists is continually evolving to reflect current dangers, as are the means by how this data is researched and published.

Tried and tested

There is a vast difference between “old and dated” and “tried and tested.” One can’t deny that blocklist data’s delivery mechanism, i.e. via DNS zone, is decades old.  In fact, the first DNSBL was created in 1997 by Eric Ziegast, who was working at MAPS with its founder, Paul Vixie.

But, and this is a big BUT, DNSBLs are a light, robust, and convenient real-time delivery mechanism for reputation data relating to IPs, domains, and content.  They are continually developing to counter nefarious online activity.

New research technologies

The early days of blocklist creation involved hours of tireless manual research and the use of heuristics.  Today, machine learning enables us to process an ever-increasing amount of data, with our researchers still undertaking manual investigations and applying heuristics.

The type of datasets that are produced is also advancing.  In addition to IP and domain reputation data, hash blocklists are compiled, enabling specific content such as email addresses, malware files, and cryptowallet addresses to be listed.

By leveraging new tools and technologies, we continue to identify bad actors and bad behavior while limiting the number of false positives.  We spend considerable resources advancing our threat hunting capabilities, allowing our customers to place their focus elsewhere.

So, while DNSBLs’ delivery mechanism might be considered “old,” the actual data within is carefully researched using up to the minute technology and techniques to deliver IP, domain, and content reputation data that protects against current threats.

2. Blocklists only stop spam

Er, no.  DNSBLs can filter all manner of email-borne threats, including phishing links, malware files, bots, and URLs to malware dropper sites. Spam is just the tip of the iceberg.

Clarification 

Firstly, let’s define spam…. Obviously, we don’t mean the trademark name of the famous processed meat product.  We are talking about unsolicited emails that are sent in bulk to a large number of recipients.

Most people consider spam to be the umbrella term for the plethora of scummy emails distributed, selling pills, sex, and lists, among other undesirable things.

Email, though, is also a vector for more sinister issues like malware files (94% of which are delivered via email), ransomware, phishing sites….and, so the list goes on. Blocklists can mitigate the risks from these and more.

Blocking malware files 

Emails that have malware files attached can be blocked with the malware component of the afore mentioned Hash Blocklist. This lists cryptographic hashes of malware files. The Spamhaus researchers can list malware files within 20 seconds of initially observing them. 

Blocking domains

Domain blocklists (DBL) contain lists of domains that are not only associated with sending spam but also those associated with phishing, malware, and botnet command and controllers (C&Cs).

Blocking compromised machines

Our eXploits Blocklist (XBL) only lists IP addresses (approximately 13 million of them) that are showing signs of compromise, i.e., machines infected with malware, worms, and Trojans, or third-party exploits such as open proxies or devices controlled by botnets.

You’ll note that there isn’t a single mention of the word ‘spam’ in that litany of internet baddies.  Where a machine is compromised, it’s highly likely that further criminal activity will follow, including spewing out emails with malware attachments to all available contacts to further propagate and increase its reach or the download of its log-in and financial credentials.

Enough said?

We could continue, but we figure that we’ve made the point that blocklists filter out a whole lot more than just spam!

3. By the time entries are in a blocklist; it’s too late

The belief that blocklists are purely reactive, i.e., an IP, domain, or hash is only listed once observed to be behaving maliciously, is simply not accurate.

Spamhaus spends a significant amount of time investigating internet environments and resources.  Our researchers are continually looking for indicators that suggest a property is about to participate in malicious activity.  Here are some examples of where blocklists are proactive in their listings:

Hijacked IP ranges

Where we observe a hijacked IP range, it is (almost) a dead cert that nefarious activity involving that range will follow. Therefore, we will list the respective IP range(s) to block any potential attack vectors they may emit.

Poor domain reputation 

Spamhaus is provided with lists of domains, as and when they are registered, and we proactively investigate their various proprieties.  From this analysis, our researchers can score the domain’s reputation, evaluating its likelihood of being used with malicious intent. These are then listed before the domains are used in the wild.

Newly registered domains

Our Zero Reputation Blocklist (ZRD) lists all newly registered domains for 24 hours.  When a new domain is initially registered, it’s improbable that emails will be sent from it immediately, unless it’s being used by bad actors, who register and burn through hundreds and thousands of domains.

IP ranges that shouldn’t be sending email

The Policy Blocklist (PBL) that Spamhaus supports enables Internet Service Providers (ISPs) to list ranges of IP addresses that should never be sending email directly to the internet.  This is an entirely pro-active blocklist.

Reactive intelligence can still protect

Spamhaus has reduced latency, so when attacks occur, it takes less than 20 seconds from detection for an IP, domain, or hash to be listed.  For large, vicious campaigns, our data may not protect users hit at the initial start of the attack, but in the case of a hailstorm campaign, which runs for up to 5 minutes, 90% of those affected will have protection.

4. All blocklists are created equal

This is far from true in many ways. Having read this far, you will quickly be realizing (if you hadn’t already) that no blocklist is the same.  Each has separate listing policies, i.e., the criteria used to decide as to whether something should be listed or not.  Similarly, the organizations that research and compile blocklists are not the same. While starting a blocklist is relatively easy, maintaining its consistency is difficult (and crucial).

Different threats and different places to apply the data

Given that every blocklist has a different focus, they need to be applied to the right part of an email transaction. For example, using the PBL in deep header parsing would be a recipe for disaster. Follow our best practice to ensure you are implementing them correctly.

Different risk profiles call for different blocklists

Always make sure you understand the functionality of each blocklist you use.  Spamhaus, for example, has an “Abused legit” section to the DBL.  Here hacked legitimate resources are listed.  You can choose whether to ignore or act on the query response relating to this part of the DBL. Be aware that it can potentially increase your risk of false positives.  You need to weigh-up your organization’s appetite for risk versus the operational issues relating to an increased number of false positives.

Consistency is king

When it comes to blocklists; providers need to be consistent.  Each listing needs to adhere to its particular policies, day in, and day out, which isn’t easy to do.

5. Blocklists should be used only during the SMTP transaction

Incorrect! Suppose you use blocklists only during the SMTP transaction. In that case, you miss the opportunity to scrutinize many elements, such as URLs, Reply-To email addresses, files sent as attachments, etc., against reputation data.

While it is true that the SMTP stage alone can block 95% of the spam or more, a fraction of nasty messages carrying threats could still pass through.

6. Blocklists should be used only during content analysis

No!  If you are only using blocklists during content analysis, you are missing a huge opportunity to immediately drop a massive percentage (see that 95% figure above?) of unwanted email at the initial IP connection and during the SMTP transaction before transferring data.

Dropping malicious email before it reaches the expensive content-filtering stage enables you to save on bandwidth, significantly reduce the number of open connections, conserve storage, and benefit from a general reduction in memory and CPU requirements.

7. Only IP blocklists should be used to filter email during the SMTP transaction

Absolutely not!  As discussed, Spamhaus lists a number of domains before they’re seen in the wild.  Therefore, querying the DBL and ZRD at the appropriate stages of the SMTP transaction provides additional proactive protection.

What myth would you like busted?

That’s enough myth-busting for one day.  If you’ve any other myths relating to blocklists and Spamhaus, let us know via Twitter or LinkedIn – we’d be delighted to bust those too.