Kimler Sidebar Menu

Kimler Adventure Pages: Journal Entries

random top 40

Writing an SPF Record

Filed in:The Web
Site News

Writing an SPF Record

February 7th, 2007  · stk

Spammers are forging sender email addresses to make it look like their SPAM comes from our domain! ACK!! SPF to the rescue! Learn what it is, how it works & how to write your own SPF record.

spf logo

A Case for SPF Records

Nearly a month ago, we reported that our Randsco domain had been hijacked by spammers. They were sending their SPAM email, around the world, using bogus sender addresses from randsco.com. To anyone receiving the SPAM, it would look like it was coming "From: randsco.com"! 88|

The cure for this was to add a Sender Policy Framework (SPF) record to our DNS. For mail servers that check, the SPF record tells them if the email is really from randsco.com or not. Spammers will quickly learn that their "From: Whoever<at>randsco.com" emails won't get through and quit trying to forge the randsco.com domain.

Every domain owner should publish an SPF record.

If you own a domain, you should publish an SPF record. Even if you never send email from that domain, spammers can hijack it, which may result in your site being blacklisted and it also erodes people's confidence in the email medium.

Publishing an SPF record is easy. Knowing what the SPF record should contain can be confusing, depending on your email situation. Here is what I learned in publishing ours. Hopefully, it will be of value to you.

To learn about how SPF works and how to publish your own, read on ...

How Sender Policy Framework Works

First, you publish an SPF record, for YourDomain.com, in your DNS. Usually, the policy is written as a specially formatted TXT record [1] on the domain. The line provides information about which mail servers (IP Addresses) are allowed to send email for the domain and how to handle the email if it's not from one of those mail servers.

spf workflow

When an email server receives an email claiming to be "From: YourDomain.com", if it's been configured to check for an SPF policy, it will look for the one you published. The receiving email server then interprets your SPF record to ascertain if the incoming email complies with your stated policy (does the IP address of the sending mail server match one of those allowed by your SPF policy?). If there is an IP address match, the email is delivered. If not, it's a forgery. The email isn't delivered (or flagged and delivered) as per your SPF instructions.

To work, everyone must play nicely (i.e., receiving servers must check for SPF records and in turn, SPF records must be published and valid.). SPF is a relatively new concept and as such, is gaining groundswell, as more and more mail servers utilize it.

You can help stop sender mail forgery by publishing SPF records and requesting (or verifying) that your ISP and/or Host Provider enable SPF checking for all incoming email.

 

Publishing Your SPF Record

How you publish your SPF record depends largely on your access to your DNS records. Because our domain is served from a hosting company, that hosting company controls access to the host's name-servers. I cannot (as it is presently configured) directly modify or update the text records in our DNS.

To me, how one modifies a DNS records is an unknown, since I don't have direct access. However, my understanding is that it's possible to have a host company (like we do) and pay for DNS hosting separately, where you DO have direct control over the text records on the name-servers. Scott Kitterman, at OpenSPF.org, maintains a list of "Domain Registrars and DNS Providers that Support TXT Records" (if you're interested in going that route).

For us, publishing an SPF record is as simple as: (1) Telling our host that we wanted an SPF record and letting them do it for us; or (2) Determining what we wanted for our SPF record and then telling them to add it.

My problems began by relying on our hosts to write our SPF record. Apparently, they didn't know what they were doing and set me up with an invalid record.

 

Diving into the Deep Blue SPF Sea

I merely wanted to stop spammers from sending SPAM in our name. I did not intend to learn nitty-gritty details about SPF syntax. Unfortunately, I didn't have much choice. I (erroneous) assumed that my host provider knew what they were doing and I was happy to see the following SPF record show up, after a few hours of requesting one, when I used the tools at DNSreport.

v=spf1 ip4:207.218.208.0/24 a mx a:randsco.com a:siteground126.com a:serv01.siteground126.com mx:randsco.com mx:siteground126.com mx:serv01.siteground126.com include:siteground126.com include:serv01.siteground126.com ~all

After all, it was all ÅëëÜäá (Greek) to me! And it looked sufficiently complicated that I didn't really WANT to get into it.

I first realized there was a problem, when I ran that SPF record through Scott Kitterman's online SPF record testing tools. (I believe I ran the first one, but didn't log the actual results. I can't redo them up again, because our SPF record has been changed (i.e., corrected). However, I believe that the "evaluation" return was something like "Permanent failure: no SPF record found for siteground126.com, no SPF record found for serv01.siteground126.com").

I didn't know much about SPF records, but it was pretty obvious that an evaluation resulting in "Permanent Failure" wasn't a good thing. ;)

 

Righting My SPF Record

Starting with incorrect syntax is never a good start down the path of enlightenment. I sought help in the "Deploying SPF" section of the openSPF.org website, specifically: The SPF Wizard. (A bad move, it turns out, as this 'old' wizard doesn't really spit out the best SPF record). This didn't really do much to further my understanding of SPF record syntax, so (because there wasn't much else to turn to in the "Deploying" section - I headed straight to the SPF Syntax document.

Reading the bit there about 'includes', quickly told me that my hosts weren't using them correctly. I tried to get my mind around this new syntax and write my own record, but I had to admit - I was lost. :(

When you're lost with something new ... you seek help in the forum, which is where I went. (The SPF Help Forum is really a listServ, so I had to sign up and post my questions there). Eventually, I became enlightened, but having started from a bad place ... it took a while.

What I DID learn, however, was that my host provider's SPF record was incorrect (on many levels). I now have an (much shorter) SPF record that's tailored to my particular mail setup (see below). I also learned the proper way of going about setting up an SPF record!

Code:

randsco.com. 14400 IN TXT "v=spf1 ip4:67.15.255.71 -all" 

 

Writing YOUR SPF Record

Sadly, the best instructions for writing an SPF record, on the openSPF.org website, are buried in their FAQ section and called "Common mistakes when creating an SPF Record". (I've recommended that they re-title this document "How to write an SPF Record" and move it to the "Deploying" section, on the front page. Hopefully, they'll listen! ;) )

The goal of the SPF record is to publicize ALL the IP addresses from which, your domain email may be sent. (While it's not necessary to have IP addresses in your SPF record - you can use domain names - using an IP address is more efficient. Putting a domain name means a DNS look-up to obtain the IP address of the domain, so you're speeding the process, by using IP addresses.)

The SPF record does three things:

  1. Identifies itself as an SPF record & identifies which version of SPF it's using (v=spf1)
  2. Publicizes all the IP addresses (perhaps via domain name) from which mail is sent, for any email address on that domain. (variety of SPF mechanisms: a, mx, ptr, IP4, IP6, exists, include. Ours was simple: "ip4:207.218.208.15").
  3. Tell the receiving (SPF-enabled only) mail-servers, what to do with mail, when the incoming mail IP address doesn't match any in the SPF Record ("-all" in our case, which means "reject" all incoming mail that's not from the one mail server).

The complexity of your SPF record depends on the different outgoing mail servers you might use. Our case was very simple (we only really send mail to and from our randsco.com addresses through our host's mail server, which doesn't change).

Do you use any local mail clients (like Outlook, Eudora, Outlook Express, etc.) to send mail? If you do, you would need to include your ISP outgoing mail server IP address (or range of addresses). You could also include it by name, using "include:yourISP.com" ... but ONLY if "yourISP.com" has an SPF Record. (You can check to see if this is so, by using the first tool on this page).

Do you forward mail to another online mail service, like Yahoo!Mail, gMail or Hotmail? Again, if you do (and respond to those mails from those services) you'll need to include the IP address(es) of their outgoing mail servers (or "include" them, by name, if they have published SPF Records).

Key to all of this, is knowing the IP addresses or names of the outgoing mail servers from which, you send yourDomain.com mail. Because I'm not an email guru, nor was our set-up complicated, I'm not 100% sure of the best way to find these IP addresses or names. My recommendation would be to (1) Check the mail headers, which will list "received from" IP addresses; or (2) contact your host/isp/service and ASK. (If anyone knows of a better, faster or easier way ... please comment).

NOTE: The MX lines in DNS records only indicates INCOMING email servers. They may (or may not) be used for outgoing email. (In our case, they do, but they might be - and often are - two completely different servers, with different IP addresses).

Even if your domain doesn't send ANY email, you should STILL HAVE an SPF record. A record of "v=spf1 -all" says "This domain doesn't send mail, reject anything from this domain." (For those receiving mail servers that are configured to LOOK FOR SPF Records).

You must also tell receiving servers how to handle sender email forgeries. There are 4 choices:

  • -all (reject or fail them - don't deliver the email)
  • ~all (soft-fail them - accept them, but mark it as 'suspicious')
  • +all (pass regardless of match - accept anything from the domain)
  • ?all (neutral - accept it, nothing can be said about the validity if there isn't an IP match)

This is hardly a thorough discussion on writing SPF records, but I hope that it cuts through the chatter and gets you off on the right foot (and saves you from the 'cart before the horse' experience I had).

 

Bottom Line

SPF records work bloody well!

Before I published a (valid) SPF record, I was receiving anywhere between 20 and 30 bounced email messages (delivery failure - no such address, mostly). It took a couple of weeks for the SPF Record to do it's job, but after that, we have not had ANY blow-back from SPAM messages (the only indication I had that our domain name was being forged by spammers).

protected by sender policy framework

When you get done writing up your SPF record, you can slap this little graphic onto your email signature, to tell others "You'll never get SPAM from this domain! We're doing our part to promote Sender Policy Framework and fight SPAM." :D

I hope this helps.

 

Additional Resources

  1. SPF Project Pages - Home Page (Your link to everything "SPF")
    • Writing an SPF Record - A Tutorial (of sorts) (The article is about "Common Mistakes", but it's the best description of "How to Write an SPF Record" I found.)
    • SPF Record Testing - Online Tools (Check your SPF record, syntax & performance).
    • SPF Record Syntax - Mechanisms & Modifiers (All the detail).
  2. Scott Kitterman - List of Registrars & DNS Providers that Support SPF TXT Records
  3. Mice & Men - DIG Online (See your DNS records, including your SPF record, if you have one.) [1]
  4. DNSstuff.com - Online DNS Tester (DNS Report or Mail Server Test) Plug in your domain & get get DNS info (SPF record shows up in the "Mail" section - if you have one).
  5. Zytrax.com - Another SPF How-to. Article provides a brief SPF introduction, SPF syntax rules and five example records, accompanied by comments and notes.
(Permalink)
Views: 123558 views
21 Comments · GuestBook
default pin-it button
Updated: 3-Jun-2008
Web View Count: 123558 viewsLast Web Update: 3-Jun-2008

Your Two Sense:

XHTML tags allowed. URLs & such will be converted to links.


Subscribe to Comments

Auto convert line breaks to <br />

02/25/07
It's good to read another excellent story about SPF at work. Thanks for describing your experiences.

I've had similar problems in the past and set up SPF for my new domain straight away. I described how to set up SPF records with 123-reg, though the procedure is pretty much the same with most domain registrars that allow TXT DNS records.
2.flag stk Comment
02/25/07
Dave,

Thanks for the link. Your succinct description helps to fill in my gap of understanding how easy it can be when you have access to a control panel for setting up TXT DNS records.

I think my mate in Blackpool uses 123-reg too, for his domains.

-stk
3.flag Eric Comment
10/16/07
Hi just resolved my SPF problem !
In fact this was working but microsoft didn't see my spf.(every other site did)

The problem came from additionnal TXT DNS records i had (with the address of the company).

I removed it and now it runs !
4.flag stk Comment
10/16/07
Eric - I didn't realize you had a problem. ;) Glad you got it sorted and thanks for posting the problem/solution.
5.flag Ruud Comment
12/12/07
How about the quotes? It seems you need to enclose the SPF records in quotes.
  Here I still have trouble with Hotmail; I don't even get bounces back but the mails sure also don't arrive. :(
6.flag stk Comment
12/12/07
Ruud - Yes the quotes are needed (a bit confusing, because of my blockquote styling, sorry).

Our current SPF record, as queried against our nameserver, at Dig Online is:

randsco.com. 14400 IN TXT "v=spf1 ip4:67.15.255.71 -all"

RE: Hotmail ... (I don't use hotmail, but) in an effort to see if Hotmail has SPF records, I found this FAQ question: "My message didn't arrive at Hotmail, not even in the SPAM foler. Why not?"

Hope it helps (and let me know about the hotmail issue). Cheers.
7.flag Sokobanja Comment
06/03/08
What about the quotes? It seems you need to enclose the SPF records in quotes.
8.flag stk Comment
06/03/08
Sokobanja - Hmmm ... I thought I cleared that up with this comment. :|

OKAY ... I can take a hint. ;)

I changed the blockquote code in the post to be an actual code block instead. :D

(Yeah, you need the quotes). :p
9.flag Aaron Dwyer Comment
07/20/08
This was an excellent article that really helped me to understand SPF and get it working after many frustrating hours.

Much appreciated.

Aaron
10.flag stk Comment
07/20/08
Aaron - I also, had many frustrating hours, during my initial foray into SPF and thought I could write a better tutorial. Looks like it did help (you anyway) and for that, I'm glad. :D Thanks for taking the time to comment.
11.flag Dayre Comment
08/05/08
One of your issues was finding the IP's for all the valid sources of e-mail for users of your domain. One method is to scan your existing mail logs (IMAP/POP) for the source ip address then do domain lookup. This one liner scans the mail log files, parses out the source ip, does a lookup, keeps the domain portion and sorts by frequency. The idea here is that the source ip address for those CHECKING mail will also be the network they will be sending mail FROM... so would need to be added to your SPF rules.

Note: we use dovecot, so it contains a rip=xxx.xxx.xxx.xxx line in the log... your log format may be different so this may need some tweaking.

cat /var/log/maillo* | grep rip | grep Login | awk '{print $10}' | sed -r 's/rip=(.*),/1/g' | awk '{system("host " $1)}' | grep pointer | awk '{print $5}' | sed -r 's/[^.]*(.*)/1/g' | sort | uniq -c | sort -nr | more
12.flag stk Comment
08/05/08
David - That's quite a mouthful! :p

I'll have to give it a try (after looking at the format of the log file).

Thanks for posting this. It looks like it may be very useful. :D
13.flag Nick Comment
08/09/08
Nice post, hope you're keeping well :)
14.flag stk Comment
08/09/08
Nick - LOL ... It's a small Internet, eh? (Took you ages to find it) :|

Glad you like. Just about to head off on an adventure vacation - our only one for the year :(

Good news is ... it'll be grand! :D

HI to "the stooges"!

15.flag Mahvin Comment
09/23/08
stk - wow, thanks to your site and information, and Google cached websites (openSPF.org is gone or down) I figured out what I was doing wrong.

First point is "Stay away from the SPF wizards", read the SPF Syntax page first. That advice alone was the best.

That's what I did and it help me resolve failing authentication.

Thanks again for your valuable website!
16.flag Dan Comment
09/23/08
Hey there,

Thanks for this post, this is great, especially because www.openspf.org is down and this is the best resource about spf I could find. Now, I have one more question in case you know. My case is simple, the spf I use is:

datecs-web.ro. 3600 IN TXT "v=spf1 ip4:89.37.127.22 -all"

The problem is that my emails get to gMail, to Hotmail and to a couple of other domains but I’ve never managed to send an email to Yahoo. The email doesn’t even get to the spam folder, nothing.

Do you have any idea if this could be spf related or if there might be another problem?
I’ve run all the tests from www.kitterman.com/spf/validate.html and everything seems alright.
Thanks.
17.flag stk Comment
09/23/08
Mahvin & Dan - Glad you liked the article and found it helpful.

Dan - I have no idea why your mail isn't showing up at Yahoo, but doubt it has anything to do with your SPF record.

Yahoo!Mail doesn't use SPF records for authentication, favoring their own "DomainKeys", which they developed and put out there for others to use.
18.flag Jay Comment
10/15/08
Thanks a lot! this was well-written
19.flag stk Comment
10/15/08
Jay - We're glad you found the article useful. However, you've been "de-linked". Your message is (a) poorly punctuated; (b) a non-descript version of "your site is great" and (c) your link to bniconnection.com isn't your personal site. So all we heard you say was: "I'm a spammer". ;)
20.flag Shmanny Comment
11/21/08
Very nice indeed!

Here's a really simple way to get IP addresses for your smtp server(s): http://www.xav.com/mx_lookup.pl
21.flag stk Comment
11/21/08
Shmanny - This tool is nice for looking up MX records, but ... and this is important ... MX records point to the server that RECEIVES email (for a given domain).

Often it WILL be the same as the sending server, but not always. IF the SENDING server(s) are different, then SPF records based on the MX record will yield a "FAIL" result.

Also, it's common to send from:whatever@domain.com email from places other than domain.com. (i.e., SPF records need to take into account ALL the places that someone might send mail FROM and may or may not be limited to & identical to where it's received). ;)