Email Authentication Crash Course: SPF

In my experience: SPF is the most misunderstood part of all email authentication.

To put it simply: if you tell customers to include your own SPF record in theirs, you’ve immediately signaled you (probably) don’t understand SPF.

The “too long; didn’t read” summary:

Man I sure did write a lot more than I intended to!

So if you want to send emails using a client’s domain, here’s what to do:

• Do not touch the client’s existing SPF record.
• Do not force the message’s “from” address to match the envelope address.
• Do create a subdomain for bounce-tracking.
• Do sign messages with DKIM.

Read on for a better understanding of why.

Email has two “from” addresses, one of them is virtually invisible, and the invisible one is the only one SPF cares about.

The SMTP protocol has a MAIL FROM command, used when trying to send email to an SMTP server. This is referred to as the envelope address, I’ve also seen it referred to as the “return path.”

In most mail clients, you have to view the message headers or message source, and look for the “Return-Path” header to see this address. Most people will never, ever do this. So the address is basically invisible.

SPF only authenticates the envelope address.

When a receiving system gets the MAIL FROM command from the sending system, it will query the SPF record using the domain found in the envelope address. SPF will never, ever authenticate the more important, user-visible “From” address found inside the message.¹

A message’s envelope address does not have to match the user-facing “From” address seen in the message headers. It is perfectly valid for these to be two completely different addresses at different domains.

How do companies get SPF wrong?

All too often, I see platforms produce email authentication instructions like this:

Add this “include” statement to your existing SPF record: include:_spf.buffering.rocks

This is fundamentally flawed.

You’re not tracking bounces!

Bounces can be delayed for days. Intermediary servers can keep trying to send a message when they receive a temporary error. By the time this happens, your outgoing TCP connection has been long-closed, so now the recipient system needs to generate and send a deferred bounce message.

The deferred bounce will always go to the envelope address. If you’re asking the client to include your SPF record in theirs, then presumably, you’re using their domain for the envelope address, and not a domain that you own.²

When that intermediary server gives up and generates the bounce, it will look at the message’s envelope address. Since it’s using your client’s domain, the server will look up your client’s MX records and send the bounce there.

Hopefully, hopefully your client used an actual address and not some ignored, no-reply address.

Ideally you should be tracking bounce rates. If your client is getting bounced a lot, guess what? They’re probably a spammer and you should drop them to maintain a good reputation. Or maybe they’ve messed up, and you can go the extra mile and help them out.

Either way: since you’re using their domain as the envelope domain, and having bounces go to their SMTP servers, not yours - you’ll never know.

You’re adding more DNS lookups to your client’s SPF record!

SPF has a hard limit of 10 DNS lookups beyond the initial lookup. Once a receiving server has to do an 11th DNS query, that’s an error and the email automatically fails SPF.

Your client probably already has an include for their existing email provider. If they’re using Google Workspace, they’re already at four out of ten lookups. You are, at best, only adding 1, but if your record utilizes any additional includes, or macros, etc, your single include could easily blow the record over 10 lookups.

Sure, you could provide a list of IP addresses instead but that opens another can of worms, like requiring clients to update their SPF record when you change server IPs. You can also start dealing with maximum DNS TXT record sizes. Plus the whole “you’re not tracking bounces” thing is kind of a big deal.

Trying to “verify” your SPF is included probably won’t work.

So as I stated, you really shouldn’t be having clients include your SPF record into theirs, unless you’re sure they’re creating a dedicated domain just for your platform (detailed below).

If for whatever reason, you have to use their domain, don’t try to “verify” you’ve been included by scanning the text contents of their record for your include string.

SPF has some amazing features like macros, that let you replace portions of your SPF record on-the-fly with pieces of data, like a system’s IP address or reverse IP address.

One way to get around that 10 DNS limit is to utilize macros. If I create an SPF record like:

v=spf1 exists:%{ir}.%{v}._spf.buffering.rocks -all

I can instruct the receiving mail system to take the incoming IP address and reverse it, then do a DNS query with it, and if it gets a result - that’s a pass.

This requires me to store my SPF info elsewhere, outside of the record - it’s essentially stored in DNS now. Your stupid “include” statement won’t show up, and your even stupider “scan the SPF record for the include statement” won’t pass.

How can companies get SPF right?

The most important thing is to not require the “From” address in the message header’s to match the envelope address. I see this as a requirement everywhere and it has got to go. If you allow the envelope address and “From” address to be different, that opens up a world of possibilities.

In both of these recommendations I’m going to mention DKIM. and DMARC. The important part you need to know right now is:

• DKIM allows you to cryptographically sign messages that can be verified using keys published in DNS.
• While SPF on its own doesn’t care about the “From” address, DMARC can pass if you have a passing SPF result that’s in alignment with the “From” address.
• You can be in relaxed alignment by using a subdomain as the envelope domain.
• DMARC can also pass if the message has a valid DKIM signature on the message’s “From” domain.
• DMARC does not require both SPF and DKIM to pass, just one.

Be lazy, don’t mess with the client’s domain at all, and just use DKIM.

This is honestly a great way to handle SPF, is to just not do it. Use your own envelope addresses on your own domain with your own SPF record. Handle bounces and so on.

You won’t ever be in SPF alignment with the client’s domain, but great news: DMARC only requires you to pass 1 authentication check, so you can just use DKIM. Generate some DKIM keys for the client and have them publish the public keys, sign the messages, bam.

So long as you’re DKIM signing the messages with a domain that’s in alignment the “From” address of the email, the envelope address can be literally anything.

Another option: create an SPF subdomain; still use DKIM.

Another good option is to establish a subdomain and rely on relaxed SPF alignment to pass DMARC.

Say you’re going with sales.example.com as your new, SPF subdomain. Have the client publish an SPF record for sales.example.com with one (and only one) include mechanism: yours.

Additionally they’ll need to publish MX records for sales.example.com with your own SMTP servers, so you can handle those deferred bounces I talked about earlier.

If a message passes SPF using sales.example.com, it’s in relaxed alignment for example.com, so you can use example.com in the user-visible “From” address and still pass DMARC.

If you want to be really slick - just have the client make sales.example.com a CNAME entry to your own, say, bounces.buffering.rocks records. Make sure bounces.buffering.rocks has your needed SPF record and MX record, now your client has to publish less records.³

As in the previous option - you should still sign messages using a key published on the client’s domain. Even if you’re passing DMARC via relaxed SPF alignment, SPF really breaks down with automated forwarding. You should always perform DKIM signing.

Thanks for reading!

I sincerely hope this was helpful. Here’s links to other parts of the series.

• Part 0: Intro.
• Part 1: SPF (this post).
• Part 2: DKIM.
• Part 3: DMARC.
• Part 4: Wrap-Up.