Driving Behavioral Change for Information Management through Data-Driven Gree...
Email architecture using SES
1. Email
Architecture
using
Amazon SES
1
Deliverability and Scalability
2. Introduction
• Email is a method of exchanging
digital messages from an author to
one or more recipients
• Email is one of the most critical
component in an application for
communication between the
application and its users
• Sending effective emails help
business to bring in more people to
use their sites and to share
information that helps the end
users 2
3. How Email is sent?
• Internet was built on
trust and email also
works in the same
fashion
• No authentication
exists between ISP to
send emails
• SPAM abuse is high as
this trust is misused
3
4. What ISP’s are doing?
• ISP’s nowadays actively
block IP ranges that
they believe generate
SPAM
• ISP’s employ a range of
software to categorize
email as JUNK, SPAM,
Phishing, etc…
• Countries are actively
legislating to punish
offenders 4
5. What companies are worried about?
• Email deliverability
• Ensure email reaches
the Inbox of the
recipients
• Handle ISP reports
• Complaints
• Bounces
• Statistics
• Keep delivery costs low
5
6. Deliverability
• Refers to the likelihood of the email reaching the intended
destination
• Majority of global email traffic is either unsolicited or undesired
bulk email
• 88 to 92% of email traffic in year 2010 were spam
• Due to the security measures of ISP, even genuine emails are
marked as SPAM causing the company’s reputation to be
downgraded from a users perspective
• Users are nowadays extremely cautious about clicking any links in
email that are marked as SPAM 6
7. ISP reports
• ISP’s respond to origin with reports on emails that were
processed by the systems
• Bounces – Messages sent to non existent email address
• Rejections – Size limitations and attachment types
• Complaints – Complaints about email content, etc…
• Email systems need to handle these responses and need to
proactively handle the issue
• Ignoring reports can force an ISP to block the IP address of origin 7
server causing email outage
8. Statistics
• Email Statistics are an essential component of a business
application to understand the reach of emails
• Successful deliveries
• Click tracking
• Failure reports
• The above statistics are vital to understand the success of email
marketing and campaigns
• This data adds business benefit to ensure that the customer can
reach your system in case of emergency
• Unable to deliver newly generated passwords since the users email 8
box is closed
10. Amazon Simple Email Service
Features Benefits
• Simple Setup • Simple
• Zero Infrastructure • Inexpensive
• Bulk & transactional email-
sending service • Reliable
• Standard SMTP service • Scalable
• DKIM support • No upfront costs
• API availability • Workflow Integration
10
11. About SES
What is it What its not
• Its an email delivery service • Not a replacement for your
for bulk and transactional exchange servers. It doesn’t
system receive emails
• Not a marketing tool for
• Supports both SMTP and API sending bulk messages and
based access track status. You’ve to build
the system on top of SES
• Offers support for setting up
DKIM and signing messages • Not a guaranteed email
service that handles
everything for you. It helps
• Offers endpoint to receive you to build a guaranteed
information on email delivery delivery system.
11
12. How SES helps business?
• Offers easy interface to help with complex setups like DKIM, etc..
• Offers a scalable, highly available service for sending emails
• Offers feedback loops to handle bounces and complaints
• Offers a no CAPEX model for your email delivery setup
• Automatically filters your content to prevent SPAM messages
from being sent out if your infrastructure is compromised
• Requires no changes to existing applications since SES is 12
compliant with SMTP protocol
13. DKIM Signing
• DomainKeys Identified Mail (DKIM) is a standard that allows
senders to sign their email messages and ISPs to use those
signatures to verify that those messages are legitimate and have
not been modified by a third party in transit.
• DKIM signing helps in deliveryability of email messages as ISP’s
can verify the sender’s domain for authenticity.
• DKIM Setup in SES can be done in 2 ways
• Easy DKIM using the SES console
• Manual DKIM signing by using the SES REST API
13
14. Easy DKIM Signing
1
• Verify a new Domain
• Generate DKIM Settings
• Add the DNS records with
your registrar
• 1 TXT record 2
• 3 CNAME records
3
14
15. Error Handling
• SES offers endpoints to handle the following errors:-
• Bounces
• Complaints
• SES can be configured to handle errors in the following
ways:-
• SNS topics (HTTP endpoints)
• Email Forwarding
15
16. Send Rates
• Depending on the needs of the application, SES throttles the
maximum messages that you can send in a second
• Adherence to this rate is essential to ensure SES service
availability
• You can request the SES team to increase the rates if your
application needs are more
• Send rates improve based on email quality and decreases based
on JUNK/SPAM/Bounces
16
19. Delivery Workflow
• Application logs a queue message for sending email
• Email service polls the queue to get the message
• Locks the queue message as being processed
• Parses the message to identify the message and recipient
• Calls SES to send the email message to the recipient
• Deletes the message from queue on success
19
• Marks the message as unread if failure in SMTP connectivity
20. Email Service
• Independent Tier to handle Email delivery
• Can be multi threaded / multi instance for scale
• Distributed architecture for handling failures
• Has logic to handle the following
• Log bounces to prevent resending to an non existent address
• Control send rates based on configuration
• Failover in case SES is down to switch to a different SMTP service
• Can be independently tested
20
• Can offload logic to generate email messages based on a email
event – template pattern
21. Bounce Handling
• SES receives the bounce message from ISP
• SES sends the message to the SNS topic configured for bounces
• SNS sends the message to the subscriptions
• Can be an email address to manually handle the bounces
• Can be an HTTP endpoint for an application to process the message
immediately
• Can be an SQS queue for distributed processing (Current Architecture)
• Email Service polls the SQS bounce queue to receive the bounces
• Takes appropriate actions based on the setup
• Mark the email address as non functional in database
• Maintain its own store of failed email address (Validates against this
every time when an email needs to be sent out)
• Sends an alert to the user’s alternate email address about the 21
applications inability to reach the user
22. Load Testing your service
• The email service can be independently tested as it is not directly
linked to the main application
• Email service capabilities like bounce handling can be tested by
using the new SES mailbox simulator
• You can read more about the mailbox simulator here
• http://aws.amazon.com/about-aws/whats-
new/2012/10/03/amazon-simple-email-service-announces-
mailbox-simulator/
22
• Using the mailbox simulator will not affect your SES reputation
23. SES safety measures
• SES is a cost-effective email delivery service
• Being cost-effective makes it attractive for spammers
• Bounces are unavoidable in an internet accessible application that
allows registrations.
• SES evaluates your reputation on the following basis:-
• Bounce Percentage
• Complaints received
• Quality of emails
• SES takes these parameters seriously and can block production access
when SES service is abused
23
• As long as the bounce percentage and the quality of emails are good,
SES is a cost-effective service for your emailing needs