Email Delivery – What Pen Testers Should KnowOctober 3, 2013
I get a lot of questions about spear phishing. There’s a common myth that it’s easy to phish. Start a local mail server and have your hacking tool relay through it. No thinking required.
Not quite. Email is not as open as it was ten years ago. Several standards exist to improve the security of email delivery and deter message spoofing. Fortunately–these standards are a band-aid at best. They’re not evenly implemented across all networks and with a little knowledge of how the system works–you can avoid triggering these protections.
SMTP is the Simple Mail Transfer Protocol. It’s one of the oldest internet protocols still in use. This is the protocol mail servers use to relay email to each other. SMTP runs on port 25.
Each domain that receives email has a mail server designated to receive these messages. A domain owner designates this mail server through an MX or mail exchanger record in its DNS zone file.
Anyone may query a domain’s MX record to find the server that receives email. Here’s how to do it with dig:
# dig +short MX gmail.com 5 gmail-smtp-in.l.google.com. 10 alt1.gmail-smtp-in.l.google.com. 40 alt4.gmail-smtp-in.l.google.com. 30 alt3.gmail-smtp-in.l.google.com. 20 alt2.gmail-smtp-in.l.google.com.
From the query above, we can see that gmail.com accepts mail through these five servers. Anyone in the world may connect to one of these servers on port 25 and attempt to relay a message to a gmail.com user.
The SMTP protocol is easy to work with (S stands for Simple, right?) Here’s what an SMTP exchange looks like:
# telnet 192.168.95.187 25 Trying 192.168.95.187... Connected to 192.168.95.187. Escape character is '^]'. 220 mint ESMTP Sendmail 8.14.3/8.14.3/Debian-9.1ubuntu1; Thu, 3 Oct 2013 15:37:30 -0400 ; (No UCE/UBE) logging access from: [192.168.95.210](FAIL)-[192.168.95.210] HELO strategiccyber.com 250 mint Hello [192.168.95.210], pleased to meet you MAIL FROM: <[email protected]> 250 2.1.0 <[email protected]>... Sender ok RCPT TO: <[email protected]> 250 2.1.5 <[email protected]>... Recipient ok DATA 354 Enter mail, end with "." on a line by itself From: "The Dude" <[email protected]> To: "Lou User" <[email protected]> Subject: Haaaaaay! This is message content. . 250 2.0.0 r93JbUN2002491 Message accepted for delivery QUIT 221 2.0.0 mint closing connection Connection closed by foreign host.
The HELO and EHLO command starts a conversation with the mail server. The HELO message does nothing. The EHLO message asks the mail server to list its abilities. This information tells the SMTP client whether or not a feature (such as STARTTLS) is supported.
The MAIL FROM command tells the mail server who sends this message. This is akin to a return address on an envelope. If a mail server encounters an error it will send a non-delivery notice to the sender. This value is not part of the message the user sees.
The RCPT TO command tells the mail server who to deliver the message to. This information does not need to match the email headers themselves. This value is not part of the message the user sees.
DATA tells the mail server that we’re ready to send the message. The mail server will assume that anything after DATA is message content. This message content will contain the headers and encoded content that the user receives. An SMTP client sends a single period to end this part of the conversation.
If all goes right, the mail server will return a message id and state that the message is in the queue. When the user receives our message, here’s what they see:
This blog post focuses on SMTP and I consider a full discussion of email messages and their format out of scope for this post. In short though, a message consists of content and headers.
Headers tell the mail reader who the message is from, who it is to, its subject, and other information. Here are a few typical headers:
From: "Raphael Mudge" <[email protected]> To: "Scumbag Sales People" <[email protected]> Subject: Do you do reseller discounts?
SMTP is a plaintext protocol. All email is sent as ASCII text. There are ways to encode binary attachments and rich content messages. For our purposes, message content follows the headers. We can skip the message encoding and specify a message as-is:
Dear Sales Team, I have a client that wants to buy your software. I will issue a purchase order no matter what your reply is. Do you offer reseller discounts? Thanks Purchasing Person
Now that you know what a message looks like, I suggest that you open a terminal and try to send yourself a message by hand. Look up your email domain’s SMTP server with the dig command. Use telnet or nc to connect to port 25 of the mail server. Go through the HELO, MAIL FROM, RCPT TO, and DATA steps. Paste in a message. Type period. Press enter twice. Wait one minute. Then go check your email.
If your message ends up in your spam folder–read the rest of this post for reasons why.
Who connects to SMTP servers?
Mail servers cater to two types of users.
Mail servers receive connections from systems that want to relay a message to a user in the mail server’s domain. If I run a mail server for foobar.com, I must accept that anyone, anywhere on the internet, may connect to me to relay a message to a foobar.com user.
This last statement is important–Any system on the internet may connect to a mail server to relay a message to one of its users. This system does not have to be a mail server.
The RCPT TO command indicates who the message is for. If the mail server is an open relay it will accept a message for anyone and relay it to their server. Open relays are rare now because spammers abuse(d) them so much. Most likely the mail server is not an open relay. You will need to specify a user in the mail server’s domain when you use RCPT TO.
Mail servers must also cater to authorized users who want to send messages. An authorized user may provide any address for RCPT TO and the mail server will queue it for delivery.
How does one become an authorized user? It depends on the server. Some servers will assume you’re authorized based on the address you connect from. Others will require you to authenticate before they will relay email for you.
With all of that background out of the way–let’s talk about reasons why a mail server may reject your message. There are quite a few.
The MAIL FROM message indicates who the message is from. If I connect to a mail server and I claim to have a message from one of its users–the mail server will likely reject it. If I am relaying a message to a user on the mail server’s domain, I must claim the message is from a user on another domain.
Some mail servers will reject messages from a system with an internet address that does not resolve to a fully qualified domain name.
If your IP address is associated with an internet blacklist–expect mail servers to reject messages from you. For example, when I try to send a message through a tethered internet connection:
# telnet mta6.am0.yahoodns.net 25 Trying 188.8.131.52... Connected to mta6.am0.yahoodns.net. Escape character is '^]'. 553 5.7.1 [BL21] Connections will not be accepted from 184.108.40.206, because the ip is in Spamhaus's list; see http://postmaster.yahoo.com/550-bl23.html Connection closed by foreign host.
Sender Policy Framework
When I connect to a mail server and send the MAIL FROM command–I am claiming the message is from the address I provide. By default, SMTP does not have a way to verify this statement. It takes what I say at face value.
Sender Policy Framework (or SPF) is a standard to verify this statement. To take advantage of SPF, the owner of a domain creates a DNS TXT record that states which hosts may send email for their domain.
When I connect to a mail server and try to relay a message–the mail server has the opportunity to check the SPF record of the domain I claim the message is from in the MAIL FROM command. If an SPF record exists and my IP address is not in the record–the mail server may reject my message. SPF does not verify the message’s From header.
It takes two for SPF to work. The mail server that receives a message must verify the SPF record. The domain owner must create an SPF record as well. Without both of these elements in place, there is no protection.
To lookup the SPF record for a domain, use:
# dig +short TXT wordpress.com "v=spf1 ip4:220.127.116.11/20 ip4:18.104.22.168/32 ip4:22.214.171.124/32 ip4:126.96.36.199/26 ip4:188.8.131.52/32 a mx ?all"
SPF does not verify message content. DKIM is the standard to verify message content. DKIM is DomainKeys Identified Mail. This is a mechanism for a mail server to sign a message and its contents to confirm that it originated from that server. The signature is added to a message as a DKIM-Signature header.
The DKIM-Signature header is added to a message by a mail server. The DKIM header includes the domain the message is signed for. Another mail server may query the domain’s public key (via DNS) and verify that the message originated from that domain.
By itself, DKIM has no teeth. The lack of a DKIM header does not mean a message is valid or invalid. Large webmail providers, like Google, have made deals with owners of highly phished domains to check for a DKIM signature and spam a message if it’s not present or verifiable. This protection requires tight cooperation between a domain owner and a mail provider.
Tight cooperation between all email receivers and senders is not a tractable solution to stop email spoofing. Domain-based Message Authentication, Reporting and Conformance (or DMARC) is a standard that allows a domain owner to signal that they use DKIM and SPF. DMARC also allows a domain owner to advise other mail servers about what they should do when a message fails a check.
To check if a domain uses DMARC, use dig to lookup a TXT record for _dmarc.domain.com:
$ dig +short TXT _dmarc.gmail.com "v=DMARC1\; p=none\; rua=mailto:[email protected]"
Check if the domain you will send a message from uses DMARC before you phish. Remember, DMARC only works if the mail server that receives the message checks for the record and acts on it.
Much like SPF, DMARC requires a domain owner to opt-in to the protection. If they don’t, there is no protection against spoofing. Likewise, if a mail server does not check for DMARC, SPF, or DKIM there is no protection for the users on that domain either.
Without DMARC, SPF, and DKIM it’s difficult to discard a message as a spoof. There’s one exception to this. Your client should have a good handle on which domains they own. They should also have protections in place to prevent an outsider (you) from emailing their users with a message that spoofs their domain.
One mechanism to stop outsider’s spoofing a local user is the Accepted Domains feature in Microsoft Exchange. If you can spoof your customer’s domain as an external actor through their mail server–I would consider this a finding.
Let’s say your message gets through the initial checks. It’s still at risk of finding its way to the spam folder. Different mail servers and tools check a lot of factors to decide if a message is spam or dangerous. Here are a few to think about:
- How old is the domain you’re phishing from? If you send a phish from a domain registered last week–it’s possible a mail server may flag it as spam. Older domains are more trustworthy.
- Does your message contain a link to an IP address? Sometimes a link to an IP address looks suspicious.
- Does your message link to a URL with a different URL? For example–does your message contain a link that looks like this:
This is suspicious.
- Pay attention to your attachment. Most mail servers block known executable files (e.g., .exe, .pif, .scr, etc.) out of the box. Suspicious attachments won’t help your spam score.
- Make sure your message content is not broken. Missing HTML close tags, missing headers, and other errors are potential signs of spam. I prefer to repurpose an existing email message for my phishes. An email client does a better job generating valid messages than a hacking tool ever will.
- Check that your MAIL FROM address matches the email in the From header in your message. Some webmail providers will flag your message as spam if these values do not match. You may not have the same problem with corporate email infrastructure.
So far, in this post, I’ve raised your awareness of message delivery, how it works, and what stops it. If you’re planning to spoof a message from another domain:
- Check if the domain has an SPF, DMARC, or DKIM record. The mail server that receives your phish has to verify these records–but if they don’t exist, there’s nothing for it to verify
- Try to send your message to an inbox you control through email infrastructure that is similar to your clients. For example, many corporations use Outlook and Exchange. Microsoft Outlook has its own junk filter. Email yourself at your corporate address to see how Microsoft’s junk filter processes your message content.
- Reconnaissance is your friend. Send a message to a non-existent user at the domain you’re trying to send a phish to. Make sure MAIL FROM is an address that you control. If you’re lucky, you will get a non-delivery notice. Inspect the headers from the non-delivery notice to see your spam score, SPF score, and other indicators about your message. If you get a non-delivery notice–it’s likely that your message passed other pre-delivery checks (a local junk filter may still send your message to the spam folder though).
If all else fails–go legitimate. There’s no hard requirement that you must phish from a spoofed domain. Try to register a phishing domain that relates to a generic pretext. Create the proper SPF, DKIM, and DMARC records. Use this domain when you need something that looks legitimate. There’s nothing wrong with this approach–so long as your message makes it to the target user and it gets clicks.
Finally, don’t get discouraged when you can’t get a spoofed message to your Gmail account. Large webmail providers are early adopters and consumers of standards such as DKIM, SPF, and DMARC. It’s possible that your corporate pen testing client hasn’t heard of this stuff. Once you complete a successful phishing engagement–you can suggest these things in your report.