Recently I've been on a quest to simplify the way I deliver my websites to my customers. Not that my customers know: the primary changes are relating to server monitoring, being proactive about a few things, and getting rid of elements that I don't understand.
One of the things I really didn't understand that well and wanted to offload to somebody else was my email delivery. I already use a company called Tuffmail to handle my mailboxes (that is, for my actual email boxes - not email sent from within my Django applications) and I'm extremely happy with their service. I use and trust Tuffmail because I know I can't keep on top of everything I need to know to run SMTP and IMAP services properly and securely.
The next step in my outsourcing process was to find a company to deliver email sent from within my Django applications. Until now I've been running a local Postfix server used for delivering mail but not receiving it, and for most emails that works fine. As websites grow, though, it becomes harder to manage what happens with bounced email and also to guarantee delivery - many email providers seem more than happy to mark emails form virtual servers as spam!
Getting Somebody Else to Deliver Your Mail
My shortlist for outsourced SMTP came down to two companies: Sendgrid and Postmark. Both of these services charge a small fee to become your SMTP server for outgoing email with a number of great features.
Sendgrid, in particular, has some fantastic features that you usually only see from mailing list providers such as the ability to track e-mail opens and links within emails. Both providers deal well with unsubscribe requests and let you easily categorise emails so you can get statistics based on deliverability of signup emails compared to purchase receipts.
My final choice ended up being Postmark, primarily for two reasons:
- Postmark allow you to control your SPF & DKIM settings per-sender, so email looks as if it comes from who you say it comes from. Sendgrid only allow this on more expensive plans, which are overkill for my needs - the end result is that some e-mail clients display the sender as "Sendgrid on behalf of XYZ".
- Postmark allow you to configure multiple 'servers', each with different API credentials, to separate email delivery for each of your apps or websites. I want a feature like this to be able to see different statistics for WhisperGifts as opposed to DjangoSites.
Configuring Postmark
Getting started with Postmark is easy. Just sign up and follow the instructions to create a server, a sender signature and then configure your Django installation to use them as an SMTP server (which needs to be enabled per-server within the Postmark system). If you're on Django 1.2 there is an EmailBackend by Paul Martorana which you can use and gain even more features than using plain ol' SMTP.
One of the real benefits, as fast as I'm concerned, is managing your e-mail bounces. I've recently implemented this code over at DjangoSites to help deal with users who mistype their e-mail address. I currently get about 4-5 of these a week, and as the site gets more popular the volume of bounced email is growing.
Configuring Django to Handle Bounces via Postmark
I have created a Python file called postmarkBounces.py in my DjangoSites project folder. It's contents are as follows:
from django.core.mail import send_mail
from django.http import HttpResponse
from django.contrib.auth.models import User
from django.conf import settings
try:
import json
except ImportError:
try:
import simplejson
except:
from django.utils import simplejson
def postHandler(request):
"""
Gets POST'd data from Postmark's BOUNCE handler. We put a message against
the user letting them know their email bounced.
Requires HTTP Basic authentication with a username set in settings.POSTMARK_BOUNCE_PASSWORD.
Configure Postmark to use 'http://mypassword:blank@www.mysite.com/handle_post/'
Sample of the received JSON data in the HTTP POST body is:
{
"BouncedAt": "2010-06-03T21:00:19.0155096-04:00",
"CanActivate": true,
"Description": "Test bounce description",
"Details": "Test bounce details",
"DumpAvailable": true,
"Email": "john@example.com",
"ID": 42,
"Inactive": true,
"Name": "Hard bounce",
"Tag": "Test",
"Type": "HardBounce",
"TypeCode": 1
}
"""
authorised = False
if request.META.has_key('HTTP_AUTHORIZATION'):
(authmeth, auth) = request.META['HTTP_AUTHORIZATION'].split(' ',1)
auth = auth.strip().decode('base64')
username, password = auth.split(':',1)
if username == settings.POSTMARK_BOUNCE_PASSWORD:
authorised = True
if not authorised:
response = HttpResponse('Authorization Required', mimetype="text/plain")
response['WWW-Authenticate'] = 'Basic realm="Bounce Handler"'
response.status_code = 401
return response
if request.method != 'POST':
response = HttpResponse('Data must be provided via HTTP POST', mimetype="text/plain")
response.status_code = 405 # Bad Method
return response
# The data we're interested in comes in via the body of the HTTP POST,
# rather than as post parameters. As such we need to read in the
# raw_post_data of the POST rather than using request.POST.
raw_json = request.raw_post_data
data = simplejson.loads(raw_json)
email_address = data.get('Email', None)
bounce_type = data.get('Type', None)
bounce_desc = data.get('Description', None)
# We only write a message if the bounce is a 'Hard Bounce', this means
# we aren't bothering the user for transient errors such as DNS lookup
# failures, offline mail servers, or full mailboxes. Of course, if
# these errors persist eventually we'll get a hard bounce and let the
# user know.
if email_address and bounce_type == 'HardBounce':
try:
user = User.objects.get(email=email_address)
# Using messaging system from Django pre-1.2.
user.message_set.create(message="An email to %s seems to have bounced back to us - the response we got said '%s'. Can you please check it by clicking 'Account Settings'?" % (user.email, bounce_desc))
except:
# Not the best way to do this - however we don't want to
# raise an error if the user doesn't exist (eg we are getting
# a bounce for an email not directly related to a user in
# our system)
pass
# All Postmark needs to get is a response, any response. No templates or
# fancy content needed here!
return HttpResponse("OK")
You will note that this refers to a particular settings called POSTMARK_BOUNCE_PASSWORD - configure a random password in your settings.py file:
POSTMARK_BOUNCE_PASSWORD='MyS3kr1tPassw0rd'
Due to the way HTTP Basic authentication works (which we'll be using later) you should avoid punctuation in this password.
Lastly, set up urls.py to point a URL to our new bounce handler:
(r'postmark_bounce/', 'postmarkBounces.postHandler'),
Reset your Django application (if required - depending on your deployment method) and you're good to go.
Telling Postmark To Tell Us About Bounces
Over at Postmark, log in to your control panel and go to the 'Settings & API Credentials' page for your server. Note that each Postmark 'server' has it's own settings screen, so bounces are handled differently per-application.
In the 'Bounce Hook' field, tell Postmark the full HTTP URL to your bounce handler, including the password you entered earlier:
http://MyS3kr1tPassw0rd:none@www.mysite.com/postmark_bounce/
It should look something like the below screenshot:
Click the 'Check' button and you should see a message letting you know that the response code was 200 - all is OK! If you get an authorisation error (HTTP error 401) then you have entered an incorrect password. If you see a HTTP error 500, check out the e-mail Django should have sent to your server administrator for more details.
Show The User
The last piece of the puzzle is to alert the user when the email has bounced. Over at DjangoSites, I do this by using Django's messaging framework (it's a Django 1.1 installation - the syntax for this has changed in 1.2) as seen in the above code snippet. Note that I don't force users to confirm their email address before they log in - if your users have to do that, you'll need to be more creative with your notification mechanisms!
Add some snazzy CSS and you're good to go. This is what a DjangoSites user will see while browsing the site if we realise that emails to them have bounced:
This is pretty unobtrusive, and it is only shown to the user once. If you want to be stricter about enforcing a valid e-mail address, just modify the way you store the 'failed email' flag and report it to the end-user.
In Closing...
I moved DjangoSites to Postmark about a fortnight ago, and configured the e-mail bouncing in less than an hour last week. That's not a long time to a solid comparison, but I can already report that I am happy with the service I've received from Postmark. My other Django sites including Jutda and WhisperGifts are already sending e-mail via the service, and the per-website reporting is wonderful.
If you want to do something more complex with your bounces, then I suggest you take a look at the Postmark Bounce Documentation which explains a full API to review bounces as needed.
Seeing fewer delivery errors come into my personal inbox as a result of the Postmark service is a welcome change. The fact that the process of alerting the user and fixing their account is now automated means I save a bit of time and as my websites grow the time saved will continue to grow. As a small business owner who continues to work a full-time "day job", I can only see that as a good thing.