Tag Archives: Spam

Spam This Post – It’s Protected By Cleantalk

Spam is one of those things on the web that no matter what, you can’t really escape from it.  I’ve been posting stuff on the web for years, helped moderate a number of forums etc and always Spam is a problem.

Chicken Spam

I have this WordPress site you’re reading this post on, and two Drupal sites that I maintain.  One of those Drupal sites I do my very best to keep it out of Google, it’s just a personal blog with information only I and maybe my family care about.  So it’s got a robots.txt on it and a number of HTTP Headers that are designed to stop it getting any traffic.

Still it gets Spam.  I’ve been writing and posting in that Diary for over 20 years now, so it’s been around a while.  For ages I just deleted Spam as it came in, there was only one or two posts a month if even that.  Then just as it started to get really bad, along came Mollom.  Mollom was a great web service that you just plugged in your Drupal instance and it would flag any new content as either Good, Bad or “Maybe”.  Anything Maybe you had to manually review to ensure you weren’t hiding good content.  Mollom was an excellent free service, but after many years Acquia decided that it wasn’t a Core Product and terminated it.  There’s a great blog post by Dries talking about it and its eventual demise.

The sad, dead Mollom Drupal Plugin

Once it died, neither of my Drupal sites were Spam protected anymore and of course the Spam rolled in.  I tried a few different things in that timeframe, mostly either “Anti-Bot” type plugins, or Captchas.  Captcha seemed like the best solution, but I hated that on this very wordpress blog, whenever I went to login to the Admin panel I would be greeted with a picture where I had to click traffic lights and all that nonsense.  And so I moved to hCaptcha and it seemd to be a bit better, but then once again the Spam started rolling in.  Not really a surprise when you realise there’s plenty of Captcha Solving Services available.  I guess it’s profitable to pay the small fee to be able to post Spam.

Ugh! So again I was back to square one, and even for a time turned off comments on the popular posts on this site that were getting spam.  Then I got annoyed and went hunting for a solution and that’s when I discovered Cleantalk.

The first thing I was impressed with is there’s a plugin for both WordPress AND Drupal.  With WordPress being the primary blogging platform a lot of people use, I was glad to see Drupal (Which I think is superior myself, but they serve different audiences) hadn’t been forgotten about.

The WordPress Plugin is super easy:

Cleantalk WordPress Plugin

You install it, paste in your key provided by the cleantalk website and off you go.  You’re protected from Spam.  You can click on “advanced settings” if really need to tweak some settings when you’re using a WordPress Caching plugin, or some of the e-Commerce modules.  The Cleantalk WordPress module has settings to make sure all different types of WordPress sites are supported out of the box.

Drupal is a similar story:

Drupal Cleantalk Plugin

Again it’s very easy to setup, you put in your API key and tweak a few settings to how you need them for your setup, and away you go.  Your site is protected against Spam!

As I said, I’ve got 3 sites.  2 Drupal and 1 WordPress.  To see the status of your Spam Protection you just log into the Cleantalk Portal and you get a nice simple overview of your sites:

Cleantalk Admin Panel

From there you can see how many Spams were blocked before they were even attempted (SpamFirewall), for those spammers lucky enough to get through the SpamFirewall, how many Approved or Spam comments were processed and dealt with.

The SpamFirewall Logs

The SpamFirewall Logs

I’ve only been using Cleantalk for a couple of weeks thus far, so I don’t have a lot of Approved comments (my blogs aren’t that popular I guess!) but I sure have a lot of blocked spam.  Handbag sales, do I want a video etc.  All blocked.  No longer do I have to keep comments disabled, no longer do I have to delete comments as soon as they’re posted etc.  You can see here the content that Spammers tried to post being denied:

Spam being blocked by Cleantalk

Overall I’m very happy with Cleantalk.  It’s fitted in where Mollom left off – automated, hassle free Spam prevention.  I think the cost of the service is reasonable, even for a hobbiest like me.  I’ve asked a few support questions and both times the support people were very quick to help and offer solutions, and that’s always a good thing.

One final note: I signed up before my trial period (7 days) was over.  In doing so, I was granted an extra 6 months free, on top of the 3 years I purchased.  In writing this review, I will submit it to Cleantalk and they’ll give me a bonus 12 months of Spam protection.  So this post is, in a way, sponsored.  But the reason I’d like a bonus 12 months is because I’m very happy with the service they’re providing!  So much so I wrote this post.

Thanks for reading.  Now go and protect your website from Spam the easy way with Cleantalk!

Cleantalk Logo

Tim

Running Your Own Mailserver(s)

This post is now out of date! Running your own mailserver is even easier these days thanks to rspamd. You literally plug rspamd into your mailserver using a milter, it’s a single line in postfix, and rspamd rolls up everything below in the smtpd_recipient_restrictions section and then some more, plus it’s got a nice webGUI. 

Rspamd: Zero spam, Rapid delivery.

Running your own mailserver isn’t that hard.  I always have a chuckle when I read people say “Why would you do it yourself, there’s so much management?”  That’s crap, they just don’t know how to do it.

A mailserver basically runs itself, there’s plenty of online tools to verify that you’re not an open relay, that you’ve configured your TLS settings correctly etc.  Plenty of configuration guides (another is included below) to show you how to lock it down so that it’s not a spam wind-up-and-go machine.

I run 3 mailservers (1 primary, 2 backup).  They all talk a single Greylisting Daemon, set to allow mail through after 1 minute.  Should the greylisting daemon not be available, the servers are set to accept the mail.

Before greylisting takes place however, the mail gets a bunch of checks.  First of all, High Quality DNS Whitelists are checked, if a server is listed in here it can be Trusted to not be sending Spam.  Then Blacklists are checked.  Then remaining whitelists are checked, if a server is listed it is allowed to bypass Greylisting. NOTE: Don’t use SORBS! Their data is out of date and crap. Way too many false positives. Avoid at all costs. I made this mistake once.

Here’s the full logic that all my mail servers use.  You have to ensure you share the greylisting database correctly, otherwise you’ll end up delaying mail much longer than necessary.

  1. REJECT anyone who doesn’t say HELO
  2. REJECT invalid Hostnames in HELO
  3. REJECT senders not using <user@domain.domain> correctly as per RFC821.
  4. REJECT Unknown Recipients
  5. ALLOW from a list of Known IPs (Backup MX hosts, other trusted devices)
  6. ALLOW from Authenticated Senders (To send mail from anywhere, using username/password)
  7. ALLOW from a set of DNS Whitelists that state an entry in their list can be considered “Non-Spam”
  8. REJECT from a list of DNS Blacklists
  9. ALLOW from a second set of DNS Whitelists that are verified to be SMTP servers (skips the need to greylist)
  10. Send to Greylisting Daemon to ACCEPT/DELAY
  11. ACCEPT

Step 7 could be amalgamated with step 9, but I prefer to “trust” the lists of known, trusted  email senders before checking blacklists, as sometimes blacklists can be a bit “over zealous” in their flagging a server a spam, i.e. one that sends newsletters etc.  This way I get check of this logic:

  1. Verified quality sender – ACCEPT.
  2. Check for blacklists – DENY.
  3. Verified RFC compliant SMTP server, skip greylisting (because we know it’ll just retry anyway, no point delaying) – ACCEPT.
  4. Send to Greylisting for DELAY/ACCEPT decision.

With these rules in place, I get almost zero spam making it through, probably 2-3 spams per week.  However the amount of mail that is rejected via the Blacklists and the Greylisting is amazing, in the thousands per day.

Once I’ve finally accepted a mail, I send it to Spamassassin for checking, just to be sure.

The other thing that’s important that I’ve done fairly recently (in the last couple of years) is to ensure that Postfix is setup correctly to send and receive mail using encryption. SSLv2 and SSLv3 are disabled, weak ciphers are disabled, Perfect Forward Secrecy is enabled.

Here’s my main.cf for Postfix.

smtpd_banner = $myhostname ESMTP - SMTP BANNER GREETING
biff = no

# appending .domain is the MUA's job.
append_dot_mydomain = no

# Send a warning if mail is delayed after 1 hour
delay_warning_time = 1h
# If mail can't be delivered after 7 days, we give up
maximal_queue_lifetime = 7d

readme_directory = no
inet_protocols = ipv4

# Incoming Mail
smtpd_tls_cert_file=/etc/letsencrypt/live/<hostname>/fullchain.pem
smtpd_tls_key_file=/etc/letsencrypt/live/<hostname>/privkey.pem
smtpd_tls_CAfile = /etc/ssl/certs/ca-certificates.crt
smtpd_tls_dh1024_param_file = ${config_directory}/dh2048.pem
smtpd_tls_dh512_param_file = ${config_directory}/dh512.pem
smtpd_use_tls=yes
smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache
smtpd_tls_session_cache_timeout = 604800
smtpd_tls_eecdh_grade = strong
smtpd_tls_security_level = may
smtpd_tls_ciphers = high
smtpd_tls_protocols = !SSLv2, !SSLv3
smtpd_tls_exclude_ciphers = aNULL, eNULL, RC4
#Don't offer Auth until STARTTLS has setup
smtpd_tls_auth_only = yes

#Ask for a Client Cert
smtpd_tls_ask_ccert = yes

# Outgoing Mail
smtp_tls_cert_file=/etc/letsencrypt/live/<hostname>/fullchain.pem
smtp_tls_key_file=/etc/letsencrypt/live/<hostname>/privkey.pem
smtp_tls_CAfile = /etc/ssl/certs/ca-certificates.crt
smtp_use_tls=yes
smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache
smtp_tls_session_cache_timeout = 604800
smtp_tls_security_level = may
smtp_tls_ciphers = high
smtp_tls_protocols = !SSLv2, !SSLv3
smtp_tls_exclude_ciphers = aNULL, eNULL, RC4

#TLS Params
tls_preempt_cipherlist = yes

myhostname = <my hostname>
alias_maps = hash:/etc/aliases
alias_database = hash:/etc/aliases
myorigin = /etc/mailname
mydestination = <hostnames I accept mail for>
virtual_alias_domains = <other domains I host>
virtual_alias_maps = hash:/etc/postfix/virtual
relayhost =
mynetworks = 127.0.0.0/8 [::ffff:127.0.0.0]/104 [::1]/128 <backup MX1> <backup MX2>
mailbox_command = procmail -a "$EXTENSION"
mailbox_size_limit = 0
recipient_delimiter = +
inet_interfaces = all
html_directory = no

# Procmail to deliver
mailbox_command = /usr/bin/procmail

# sasl! You want to eat it!
smtpd_sasl_auth_enable = yes
smtpd_sasl_security_options = noanonymous
smtpd_sasl_local_domain = $myhostname
smtpd_sasl_authenticated_header = yes

# Mailing Signing with OpenDKIM
milter_protocol = 2
milter_default_action = accept
smtpd_milters = inet:localhost:12301 # Don't copy unless you have setup DKIM
non_smtpd_milters = inet:localhost:12301 # Don't copy unless you have setup DKIM

# Proper Mail Protocol Please
strict_rfc821_envelopes = yes

# Verify? No thanks!
disable_vrfy_command = yes

# Demand a polite conversation!
smtpd_helo_required = yes

# Delay before reject
smtpd_delay_reject = yes

smtpd_helo_restrictions = permit_mynetworks,
 reject_non_fqdn_hostname,
 reject_invalid_hostname,
 permit

smtpd_recipient_restrictions =
 reject_invalid_hostname,
 reject_unknown_recipient_domain,
 reject_unauth_pipelining,
 permit_mynetworks,
 permit_sasl_authenticated,
 reject_unauth_destination,
 check_client_access cidr:/etc/postfix/rbl_override,
 permit_dnswl_client iadb.isipp.com=127.0.1.255,
 permit_dnswl_client sa-trusted.bondedsender.org,
 permit_dnswl_client sa-accredit.habeas.com,
 permit_dnswl_client list.dnswl.org=127.0.[0..255].[2..3],
 permit_dnswl_client hostkarma.junkemailfilter.com=127.0.0.[1;5],
 reject_rhsbl_reverse_client dbl.spamhaus.org,
 reject_rhsbl_sender dbl.spamhaus.org,
 reject_rhsbl_client dbl.spamhaus.org,
 reject_rbl_client zen.spamhaus.org,
 reject_rbl_client dnsbl-1.uceprotect.net,
 reject_rbl_client psbl.surriel.com,
 reject_rbl_client hostkarma.junkemailfilter.com=127.0.0.2,
 reject_rbl_client bl.mailspike.net,
 reject_rbl_client b.barracudacentral.org,
 reject_rbl_client truncate.gbudb.net,
 permit_dnswl_client iadb.isipp.com=127.0.2.[1;2],
 permit_dnswl_client iadb.isipp.com=127.3.100.[5..100],
 permit_dnswl_client wl.mailspike.net,
 permit_dnswl_client list.dnswl.org,
 permit_dnswl_client hostkarma.junkemailfilter.com=127.0.0.3,
 check_policy_service inet:127.0.0.1:10060,

message_size_limit = 81920000

Once configured like that, it’s set and forget pretty much.  I occasionally check the logs to ensure that nothing is being greylisted due to the dumb policy some senders have of retrying each time from a DIFFERENT IP Address.  When I do see such stupidity I usually just add the sending /24 network to the Greylist Whitelist.

The final thing to note is that you should run your own caching DNS server.  If you’re using your ISPs, or a big public provider like Google etc, then the black/whitelists often won’t work as they implement rate-limiting against abuse, and the big public name-servers are almost always blocked.  Running your own small caching DNS server is easy and will give you a working RBL setup.

 

Update: 11/4/2017 – Turns out Protected Sky are just a bunch of rip-off merchants. Removed them from my list of checked RBLs.