The document describes how email works using SMTP and POP3 protocols. It provides a detailed example of an email being sent from one client to a mail server using SMTP commands like EHLO, MAIL FROM, RCPT TO, DATA, and QUIT. It then shows how the recipient can retrieve the email from their mail server using POP3 commands like USER, PASS, LIST, RETR, and DELE.

1. Sonoma State’s
Email
SSU CS Colloquium, October 8, 2009
Eric Eisenhart
<eric.eisenhart@sonoma.edu>
http://eric.eisenhart.name/presentations/

2. Disclaimers

3. Disclaimers
•
My opinions

4. Disclaimers
•My opinions
•No Credit

5. Disclaimers
•My opinions
•No Credit
•Network == Air

6. Disclaimers
•My opinions
•No Credit
•Network == Air
•Life

7. Too Simple

8. Too Simple
A
Mail Server
#1

9. Too Simple
A B
Mail Server Mail Server
#1 #2

10. Big Picture
Alice Bob
Load Balancer #1
ns authns mail ldaps webmail
ns cluster mail cluster authns cluster
moltar zorak blinky snorky
ns-a ns-b ns-c chalmers clancy cletus krusty milhouse authns-a authns-b authns-c
NetApp Cluster
mojo
ns3.csu.net flanders
heckle jeckle
heckle-
jeckle-
shelf-1
shelf-1
heckle- jeckle-
shelf-2 shelf-2

11. Big Picture Chuck
Red Condor #1 Red Condor #2
Alice Bob Red Condor Offsite
Load Balancer #1
ns authns mail ldaps webmail
ns cluster mail cluster authns cluster
moltar zorak blinky snorky
ns-a ns-b ns-c chalmers clancy cletus krusty milhouse authns-a authns-b authns-c
NetApp Cluster
mojo
ns3.csu.net flanders
heckle jeckle
heckle-
jeckle-
shelf-1
shelf-1
heckle- jeckle-
shelf-2 shelf-2

12. Simpler Picture?

13. How Email
Works

14. An Email
Return-path: <root@eisenhae>
Envelope-to: eisenhae@sonoma.edu
Delivery-date: Wed, 30 Sep 2009 09:57:52 -0700
Received: from exim by nelson with spam-scanned (Exim 4.63)
(envelope-from <root@eisenhae>)
id 1Mt2Ut-0007cJ-M2
for eisenhae@sonoma.edu; Wed, 30 Sep 2009 09:57:52 -0700
X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on nelson
X-Spam-Level: *
X-Spam-Status: No, score=1.6 required=5.0
tests=MISSING_DATE,MISSING_HEADERS,
MISSING_MID autolearn=no version=3.2.5
Received: from eisenhae ([1.2.3.4] helo=eisenhae)
by nelson with esmtp (Exim 4.63)
(envelope-from <root@eisenhae>)
id 1Mt2Uf-0007cC-FN
for eric.eisenhart@sonoma.edu; Wed, 30 Sep 2009 09:57:47 -0700
Subject: test message
Message-Id: <E1Mt2Ut-0007cJ-M2@nelson>
From: root@eisenhae
Date: Wed, 30 Sep 2009 09:57:52 -0700
just a test.

15. SMTP + MSP
(Simple Mail Transport Protocol +
Mail Submission Protocol)

16. SMTP + MSP
(Simple Mail Transport Protocol +
Mail Submission Protocol)
220 nelson ESMTP Exim Wed, 30 Sep 2009 09:55:40 -0700

17. SMTP + MSP
(Simple Mail Transport Protocol +
Mail Submission Protocol)
220 nelson ESMTP Exim Wed, 30 Sep 2009 09:55:40 -0700
EHLO eisenhae

18. SMTP + MSP
(Simple Mail Transport Protocol +
Mail Submission Protocol)
220 nelson ESMTP Exim Wed, 30 Sep 2009 09:55:40 -0700
EHLO eisenhae
250-nelson Hello eisenhae [1.2.3.4]
250-SIZE 20971520
250-PIPELINING
250-STARTTLS
250 HELP

19. SMTP + MSP
(Simple Mail Transport Protocol +
Mail Submission Protocol)
220 nelson ESMTP Exim Wed, 30 Sep 2009 09:55:40 -0700
EHLO eisenhae
250-nelson Hello eisenhae [1.2.3.4]
250-SIZE 20971520
250-PIPELINING
250-STARTTLS
250 HELP
MAIL FROM:root@eisenhae.sonoma.edu

20. SMTP + MSP
(Simple Mail Transport Protocol +
Mail Submission Protocol)
220 nelson ESMTP Exim Wed, 30 Sep 2009 09:55:40 -0700
EHLO eisenhae
250-nelson Hello eisenhae [1.2.3.4]
250-SIZE 20971520
250-PIPELINING
250-STARTTLS
250 HELP
MAIL FROM:root@eisenhae.sonoma.edu
250 OK

21. SMTP + MSP
(Simple Mail Transport Protocol +
Mail Submission Protocol)
220 nelson ESMTP Exim Wed, 30 Sep 2009 09:55:40 -0700
EHLO eisenhae
250-nelson Hello eisenhae [1.2.3.4]
250-SIZE 20971520
250-PIPELINING
250-STARTTLS
250 HELP
MAIL FROM:root@eisenhae.sonoma.edu
250 OK
RCPT TO:eric.eisenhart@sonoma.edu

22. SMTP + MSP
(Simple Mail Transport Protocol +
Mail Submission Protocol)
220 nelson ESMTP Exim Wed, 30 Sep 2009 09:55:40 -0700
EHLO eisenhae
250-nelson Hello eisenhae [1.2.3.4]
250-SIZE 20971520
250-PIPELINING
250-STARTTLS
250 HELP
MAIL FROM:root@eisenhae.sonoma.edu
250 OK
RCPT TO:eric.eisenhart@sonoma.edu
250 Accepted

23. SMTP + MSP
(Simple Mail Transport Protocol +
Mail Submission Protocol)
220 nelson ESMTP Exim Wed, 30 Sep 2009 09:55:40 -0700
EHLO eisenhae
250-nelson Hello eisenhae [1.2.3.4]
250-SIZE 20971520
250-PIPELINING
250-STARTTLS
250 HELP
MAIL FROM:root@eisenhae.sonoma.edu
250 OK
RCPT TO:eric.eisenhart@sonoma.edu
250 Accepted
DATA

24. SMTP + MSP
(Simple Mail Transport Protocol +
Mail Submission Protocol)
220 nelson ESMTP Exim Wed, 30 Sep 2009 09:55:40 -0700
EHLO eisenhae
250-nelson Hello eisenhae [1.2.3.4]
250-SIZE 20971520
250-PIPELINING
250-STARTTLS
250 HELP
MAIL FROM:root@eisenhae.sonoma.edu
250 OK
RCPT TO:eric.eisenhart@sonoma.edu
250 Accepted
DATA
354 Enter message, ending with "." on a line by itself

25. SMTP + MSP
(Simple Mail Transport Protocol +
Mail Submission Protocol)
220 nelson ESMTP Exim Wed, 30 Sep 2009 09:55:40 -0700
EHLO eisenhae
250-nelson Hello eisenhae [1.2.3.4]
250-SIZE 20971520
250-PIPELINING
250-STARTTLS
250 HELP
MAIL FROM:root@eisenhae.sonoma.edu
250 OK
RCPT TO:eric.eisenhart@sonoma.edu
250 Accepted
DATA
354 Enter message, ending with "." on a line by itself
Subject: test message
just a test.
.

26. SMTP + MSP
(Simple Mail Transport Protocol +
Mail Submission Protocol)
220 nelson ESMTP Exim Wed, 30 Sep 2009 09:55:40 -0700
EHLO eisenhae
250-nelson Hello eisenhae [1.2.3.4]
250-SIZE 20971520
250-PIPELINING
250-STARTTLS
250 HELP
MAIL FROM:root@eisenhae.sonoma.edu
250 OK
RCPT TO:eric.eisenhart@sonoma.edu
250 Accepted
DATA
354 Enter message, ending with "." on a line by itself
Subject: test message
just a test.
.
250 OK id=1Mt2Uf-0007cC-FN

27. SMTP + MSP
(Simple Mail Transport Protocol +
Mail Submission Protocol)
220 nelson ESMTP Exim Wed, 30 Sep 2009 09:55:40 -0700
EHLO eisenhae
250-nelson Hello eisenhae [1.2.3.4]
250-SIZE 20971520
250-PIPELINING
250-STARTTLS
250 HELP
MAIL FROM:root@eisenhae.sonoma.edu
250 OK
RCPT TO:eric.eisenhart@sonoma.edu
250 Accepted
DATA
354 Enter message, ending with "." on a line by itself
Subject: test message
just a test.
.
250 OK id=1Mt2Uf-0007cC-FN
QUIT

28. SMTP + MSP
(Simple Mail Transport Protocol +
Mail Submission Protocol)
220 nelson ESMTP Exim Wed, 30 Sep 2009 09:55:40 -0700
EHLO eisenhae
250-nelson Hello eisenhae [1.2.3.4]
250-SIZE 20971520
250-PIPELINING
250-STARTTLS
250 HELP
MAIL FROM:root@eisenhae.sonoma.edu
250 OK
RCPT TO:eric.eisenhart@sonoma.edu
250 Accepted
DATA
354 Enter message, ending with "." on a line by itself
Subject: test message
just a test.
.
250 OK id=1Mt2Uf-0007cC-FN
QUIT
221 nelson.sonoma.edu closing connection

29. POP3
(Post Office Protocol)

30. POP3
(Post Office Protocol)
+OK POP3 server ready <1896.697170952@dbc.mtview.ca.us>

31. POP3
(Post Office Protocol)
+OK POP3 server ready <1896.697170952@dbc.mtview.ca.us>
USER mrose

32. POP3
(Post Office Protocol)
+OK POP3 server ready <1896.697170952@dbc.mtview.ca.us>
USER mrose
+OK User accepted

33. POP3
(Post Office Protocol)
+OK POP3 server ready <1896.697170952@dbc.mtview.ca.us>
USER mrose
+OK User accepted
PASS mrosepass

34. POP3
(Post Office Protocol)
+OK POP3 server ready <1896.697170952@dbc.mtview.ca.us>
USER mrose
+OK User accepted
PASS mrosepass
+OK Pass accepted

35. POP3
(Post Office Protocol)
+OK POP3 server ready <1896.697170952@dbc.mtview.ca.us>
USER mrose
+OK User accepted
PASS mrosepass
+OK Pass accepted
STAT

36. POP3
(Post Office Protocol)
+OK POP3 server ready <1896.697170952@dbc.mtview.ca.us>
USER mrose
+OK User accepted
PASS mrosepass
+OK Pass accepted
STAT
+OK 2 320

37. POP3
(Post Office Protocol)
+OK POP3 server ready <1896.697170952@dbc.mtview.ca.us>
USER mrose
+OK User accepted
PASS mrosepass
+OK Pass accepted
STAT
+OK 2 320
LIST

38. POP3
(Post Office Protocol)
+OK POP3 server ready <1896.697170952@dbc.mtview.ca.us>
USER mrose
+OK User accepted
PASS mrosepass
+OK Pass accepted
STAT
+OK 2 320
LIST
+OK 2 messages (320 octets)

39. POP3
(Post Office Protocol)
+OK POP3 server ready <1896.697170952@dbc.mtview.ca.us>
USER mrose
+OK User accepted
PASS mrosepass
+OK Pass accepted
STAT
+OK 2 320
LIST
+OK 2 messages (320 octets)
1 120

40. POP3
(Post Office Protocol)
+OK POP3 server ready <1896.697170952@dbc.mtview.ca.us>
USER mrose
+OK User accepted
PASS mrosepass
+OK Pass accepted
STAT
+OK 2 320
LIST
+OK 2 messages (320 octets)
1 120
2 200

41. POP3
(Post Office Protocol)
+OK POP3 server ready <1896.697170952@dbc.mtview.ca.us>
USER mrose
+OK User accepted
PASS mrosepass
+OK Pass accepted
STAT
+OK 2 320
LIST
+OK 2 messages (320 octets)
1 120
2 200
.

42. POP3
(Post Office Protocol)
+OK POP3 server ready <1896.697170952@dbc.mtview.ca.us>
USER mrose
+OK User accepted
PASS mrosepass
+OK Pass accepted
STAT
+OK 2 320
LIST
+OK 2 messages (320 octets)
1 120
2 200
.
RETR 1

43. POP3
(Post Office Protocol)
+OK POP3 server ready <1896.697170952@dbc.mtview.ca.us>
USER mrose
+OK User accepted
PASS mrosepass
+OK Pass accepted
STAT
+OK 2 320
LIST
+OK 2 messages (320 octets)
1 120
2 200
.
RETR 1
+OK 120 octets

44. POP3
(Post Office Protocol)
+OK POP3 server ready <1896.697170952@dbc.mtview.ca.us>
USER mrose
+OK User accepted
PASS mrosepass
+OK Pass accepted
STAT
+OK 2 320
LIST
+OK 2 messages (320 octets)
1 120
2 200
.
RETR 1
+OK 120 octets
<the POP3 server sends message 1>

45. POP3
(Post Office Protocol)
+OK POP3 server ready <1896.697170952@dbc.mtview.ca.us>
USER mrose
+OK User accepted
PASS mrosepass
+OK Pass accepted
STAT
+OK 2 320
LIST
+OK 2 messages (320 octets)
1 120
2 200
.
RETR 1
+OK 120 octets
<the POP3 server sends message 1>
.

46. POP3
(Post Office Protocol)
+OK POP3 server ready <1896.697170952@dbc.mtview.ca.us>
USER mrose
+OK User accepted
PASS mrosepass
+OK Pass accepted
STAT
+OK 2 320
LIST
+OK 2 messages (320 octets)
1 120
2 200
.
RETR 1
+OK 120 octets
<the POP3 server sends message 1>
.
DELE 1

47. POP3
(Post Office Protocol)
+OK POP3 server ready <1896.697170952@dbc.mtview.ca.us>
USER mrose
+OK User accepted
PASS mrosepass
+OK Pass accepted
STAT
+OK 2 320
LIST
+OK 2 messages (320 octets)
1 120
2 200
.
RETR 1
+OK 120 octets
<the POP3 server sends message 1>
.
DELE 1
+OK message 1 deleted

48. POP3
(Post Office Protocol)
+OK POP3 server ready <1896.697170952@dbc.mtview.ca.us>
USER mrose
+OK User accepted
PASS mrosepass
+OK Pass accepted
STAT
+OK 2 320
LIST
+OK 2 messages (320 octets)
1 120
2 200
.
RETR 1
+OK 120 octets
<the POP3 server sends message 1>
.
DELE 1
+OK message 1 deleted
QUIT

49. POP3
(Post Office Protocol)
+OK POP3 server ready <1896.697170952@dbc.mtview.ca.us>
USER mrose
+OK User accepted
PASS mrosepass
+OK Pass accepted
STAT
+OK 2 320
LIST
+OK 2 messages (320 octets)
1 120
2 200
.
RETR 1
+OK 120 octets
<the POP3 server sends message 1>
.
DELE 1
+OK message 1 deleted
QUIT
+OK dewey POP3 server signing off (maildrop empty)

50. IMAP
(Internet Message Access Protocol)

51. IMAP
(Internet Message Access Protocol)
* OK [CAPABILITY IMAP4rev1 UIDPLUS CHILDREN NAMESPACE THREAD=ORDEREDSUBJECT
THREAD=REFERENCES SORT QUOTA IDLE AUTH=PLAIN] Courier-IMAP ready. Copyright
1998-2008 Double Precision, Inc. See COPYING for distribution information.

52. IMAP
(Internet Message Access Protocol)
* OK [CAPABILITY IMAP4rev1 UIDPLUS CHILDREN NAMESPACE THREAD=ORDEREDSUBJECT
THREAD=REFERENCES SORT QUOTA IDLE AUTH=PLAIN] Courier-IMAP ready. Copyright
1998-2008 Double Precision, Inc. See COPYING for distribution information.
1 AUTHENTICATE PLAIN abcdefghiBASE64jklmnop=

53. IMAP
(Internet Message Access Protocol)
* OK [CAPABILITY IMAP4rev1 UIDPLUS CHILDREN NAMESPACE THREAD=ORDEREDSUBJECT
THREAD=REFERENCES SORT QUOTA IDLE AUTH=PLAIN] Courier-IMAP ready. Copyright
1998-2008 Double Precision, Inc. See COPYING for distribution information.
1 AUTHENTICATE PLAIN abcdefghiBASE64jklmnop=
1 OK LOGIN Ok.

54. IMAP
(Internet Message Access Protocol)
* OK [CAPABILITY IMAP4rev1 UIDPLUS CHILDREN NAMESPACE THREAD=ORDEREDSUBJECT
THREAD=REFERENCES SORT QUOTA IDLE AUTH=PLAIN] Courier-IMAP ready. Copyright
1998-2008 Double Precision, Inc. See COPYING for distribution information.
1 AUTHENTICATE PLAIN abcdefghiBASE64jklmnop=
1 OK LOGIN Ok.
2 SELECT INBOX

55. IMAP
(Internet Message Access Protocol)
* OK [CAPABILITY IMAP4rev1 UIDPLUS CHILDREN NAMESPACE THREAD=ORDEREDSUBJECT
THREAD=REFERENCES SORT QUOTA IDLE AUTH=PLAIN] Courier-IMAP ready. Copyright
1998-2008 Double Precision, Inc. See COPYING for distribution information.
1 AUTHENTICATE PLAIN abcdefghiBASE64jklmnop=
1 OK LOGIN Ok.
2 SELECT INBOX
* FLAGS (Draft Answered Flagged Deleted Seen Recent)
* OK [PERMANENTFLAGS (Draft Answered Flagged Deleted Seen)] Limited
* 751 EXISTS
* 1 RECENT
* OK [UIDVALIDITY 1201564895] Ok
* OK [MYRIGHTS "acdilrsw"] ACL
2 OK [READ-WRITE] Ok

56. IMAP
(Internet Message Access Protocol)
* OK [CAPABILITY IMAP4rev1 UIDPLUS CHILDREN NAMESPACE THREAD=ORDEREDSUBJECT
THREAD=REFERENCES SORT QUOTA IDLE AUTH=PLAIN] Courier-IMAP ready. Copyright
1998-2008 Double Precision, Inc. See COPYING for distribution information.
1 AUTHENTICATE PLAIN abcdefghiBASE64jklmnop=
1 OK LOGIN Ok.
2 SELECT INBOX
* FLAGS (Draft Answered Flagged Deleted Seen Recent)
* OK [PERMANENTFLAGS (Draft Answered Flagged Deleted Seen)] Limited
* 751 EXISTS
* 1 RECENT
* OK [UIDVALIDITY 1201564895] Ok
* OK [MYRIGHTS "acdilrsw"] ACL
2 OK [READ-WRITE] Ok
3 FETCH 1 BODY[]

57. IMAP
(Internet Message Access Protocol)
* OK [CAPABILITY IMAP4rev1 UIDPLUS CHILDREN NAMESPACE THREAD=ORDEREDSUBJECT
THREAD=REFERENCES SORT QUOTA IDLE AUTH=PLAIN] Courier-IMAP ready. Copyright
1998-2008 Double Precision, Inc. See COPYING for distribution information.
1 AUTHENTICATE PLAIN abcdefghiBASE64jklmnop=
1 OK LOGIN Ok.
2 SELECT INBOX
* FLAGS (Draft Answered Flagged Deleted Seen Recent)
* OK [PERMANENTFLAGS (Draft Answered Flagged Deleted Seen)] Limited
* 751 EXISTS
* 1 RECENT
* OK [UIDVALIDITY 1201564895] Ok
* OK [MYRIGHTS "acdilrsw"] ACL
2 OK [READ-WRITE] Ok
3 FETCH 1 BODY[]
* 1 FETCH (BODY[] {2390}
<server sends full message with headers>
3 OK FETCH completed.

58. IMAP
(Internet Message Access Protocol)
* OK [CAPABILITY IMAP4rev1 UIDPLUS CHILDREN NAMESPACE THREAD=ORDEREDSUBJECT
THREAD=REFERENCES SORT QUOTA IDLE AUTH=PLAIN] Courier-IMAP ready. Copyright
1998-2008 Double Precision, Inc. See COPYING for distribution information.
1 AUTHENTICATE PLAIN abcdefghiBASE64jklmnop=
1 OK LOGIN Ok.
2 SELECT INBOX
* FLAGS (Draft Answered Flagged Deleted Seen Recent)
* OK [PERMANENTFLAGS (Draft Answered Flagged Deleted Seen)] Limited
* 751 EXISTS
* 1 RECENT
* OK [UIDVALIDITY 1201564895] Ok
* OK [MYRIGHTS "acdilrsw"] ACL
2 OK [READ-WRITE] Ok
3 FETCH 1 BODY[]
* 1 FETCH (BODY[] {2390}
<server sends full message with headers>
3 OK FETCH completed.
4 LOGOUT

59. IMAP
(Internet Message Access Protocol)
* OK [CAPABILITY IMAP4rev1 UIDPLUS CHILDREN NAMESPACE THREAD=ORDEREDSUBJECT
THREAD=REFERENCES SORT QUOTA IDLE AUTH=PLAIN] Courier-IMAP ready. Copyright
1998-2008 Double Precision, Inc. See COPYING for distribution information.
1 AUTHENTICATE PLAIN abcdefghiBASE64jklmnop=
1 OK LOGIN Ok.
2 SELECT INBOX
* FLAGS (Draft Answered Flagged Deleted Seen Recent)
* OK [PERMANENTFLAGS (Draft Answered Flagged Deleted Seen)] Limited
* 751 EXISTS
* 1 RECENT
* OK [UIDVALIDITY 1201564895] Ok
* OK [MYRIGHTS "acdilrsw"] ACL
2 OK [READ-WRITE] Ok
3 FETCH 1 BODY[]
* 1 FETCH (BODY[] {2390}
<server sends full message with headers>
3 OK FETCH completed.
4 LOGOUT
* BYE Courier-IMAP server shutting down
4 OK LOGOUT completed

60. LDAP
(Lightweight Directory Access Protocol)
dn: uid=eisenhae,ou=people,o=sonoma,o=edu
uid: eisenhae
ssuMailHomeDirectory: /nfs/jeckle/maildirs/92/eisenhae
ssuMailLocalAddress: eisenhae@sonoma.edu
ssuMailLocalAddress: eric.eisenhart@sonoma.edu
ssuMailRoutingAddress: eisenhae@smtp-vip.sonoma.edu
ssuMailSpamAssassinPrefs: required_hits 5
ssuMailSpamAssassinPrefs: whitelist_from *@example.com
ssuMailDiskQuota: 1024M

61. Maildir (++)
/nfs/jeckle/maildirs/92/eisenhae/
courierimapsubscribed
courierimapuiddb
tmp/
new/
new/1254971281.H995082P22240.cletus.sonoma.edu,S=3593
cur/
cur/1251296264.H25153P29370.krusty.sonoma.edu,S=3045:2,RS
.GrayMai/cur/
.GrayMail/new/
.GrayMail/tmp/
.Lists/
.Lists.puppet/

62. The
aka: requirements

63. Email ==
“Sonoma State University
will employ these email
accounts for official
University
communications to
students effective
immediately.”
http://www.sonoma.edu/it/get_connected/
provost_letter.shtml

64. Reliance

65. Reliance
Assumed

66. Reliance
Assumed
Addicted

67. Growth

68. Population

69. 1500
1125
GB
750
375
0
2004 2005 2006 2007 2008 2009
~300G 700G 1.35T
Storage Usage

70. SPA

71. SPA

72. Traffic
2006-08 to now-

73. Peak Traffic
2006-08 to now-

74. Moore’s Law:
Friend or Foe?

79. The
aka: what we did

80. Limits

81. Limits
•
Money

82. Limits
•Money
•Time / Staff

83. Goals

84. Goals
•
Reliable

85. Goals
•Reliable
•Minimal Update Costs

86. Goals
•Reliable
•Minimal Update Costs
•Concurrency

87. Goals
•Reliable
•Minimal Update Costs
•Concurrency
•Asynchronous

88. What We Run

89. What we Run
Hardware

90. What we Run
Hardware
• MAIL: 5 + test. 8GB, 2 dual-core 2.4GHz AMD

91. What we Run
Hardware
• MAIL: 5 + test. 8GB, 2 dual-core 2.4GHz AMD
• LDAP: 2 + master + standby. 2GB, 2 Intel
3GHz

92. What we Run
Hardware
• MAIL: 5 + test. 8GB, 2 dual-core 2.4GHz AMD
• LDAP: 2 + master + standby. 2GB, 2 Intel
3GHz
• webmail: 2 VMs. 2GB

93. What we Run
Hardware
• MAIL: 5 + test. 8GB, 2 dual-core 2.4GHz AMD
• LDAP: 2 + master + standby. 2GB, 2 Intel
3GHz
• webmail: 2 VMs. 2GB
• lists: 1 VM. 1GB

94. What we Run
Hardware
• MAIL: 5 + test. 8GB, 2 dual-core 2.4GHz AMD
• LDAP: 2 + master + standby. 2GB, 2 Intel
3GHz
• webmail: 2 VMs. 2GB
• lists: 1 VM. 1GB
• MySQL: 1 VM. 1GB.

95. What we Run
Hardware
• MAIL: 5 + test. 8GB, 2 dual-core 2.4GHz AMD
• LDAP: 2 + master + standby. 2GB, 2 Intel
3GHz
• webmail: 2 VMs. 2GB
• lists: 1 VM. 1GB
• MySQL: 1 VM. 1GB.
• storage: NetApp FAS 3070c cluster; 30*136GB
15k

96. Virtual Machines?

97. What we Run
“MAIL”

98. What we Run
“MAIL”
•exim

99. What we Run
“MAIL”
•exim
•clamav

100. What we Run
“MAIL”
•exim
•clamav
•SpamAssassin

101. What we Run
“MAIL”
•exim
•clamav
•SpamAssassin
•courier-auth

102. What we Run
“MAIL”
•exim
•clamav
•SpamAssassin
•courier-auth
•courier-imap

103. What we Run
Storage

104. What we Run
Storage
•2 * NetApp FAS 3070s in
active-active cluster

105. What we Run
Storage
•2 * NetApp FAS 3070s in
active-active cluster
•10 shelves of 15 disks

106. What we Run
Storage
•2 * NetApp FAS 3070s in
active-active cluster
•10 shelves of 15 disks
•EMC NAS (going, going, gone)

107. What we Run
“webmail”

108. What we Run
“webmail”
•apache httpd

109. What we Run
“webmail”
•apache httpd
•php

110. What we Run
“webmail”
•apache httpd
•php
•squirrelmail + plugins

111. What we Run
“webmail”
•apache httpd
•php
•squirrelmail + plugins
•up-imapproxy

112. What we Run
“ldaps”

113. What we Run
“ldaps”
•apache httpd

114. What we Run
“ldaps”
•apache httpd
•OpenLDAP (slapd)

115. What we Run
“ldaps”
•apache httpd
•OpenLDAP (slapd)
•Lots of custom code

116. What we Run
“flanders”

117. What we Run
“flanders”
•mysqld

118. What we Run
“lists”

119. What we Run
“lists”
•apache httpd

120. What we Run
“lists”
•apache httpd
•mailman

121. What we Run
Secret Sauces

122. What we Run
Secret Sauces
•Tsunami

123. What we Run
Secret Sauces
•Tsunami
•puppet

124. What we Run
Secret Sauces
•Tsunami
•puppet
•Registry

125. SPAM

126. SPAM
•
SpamAssassin

127. SPAM
•SpamAssassin
•Red Condor

128. Changes (so far)
• storage backend (wiggum; otto;
bruno; heckle+jeckle)
• Directory layout (f/foo -> ac/foo)
• Red Condor
• IMAP + SMTP + SPAM + clamav
(same servers now)
• SPAM prefs from MySQL to LDAP
• Virtualization

129. The Past

130. The Past
• My First: one box

131. The Past
• My First: one box
• VAX; cluster

132. The Past
• My First: one box
• VAX; cluster
• 2003: VAX + student server

133. The Past
• My First: one box
• VAX; cluster
• 2003: VAX + student server
• Later 2003: wiggum -> otto

134. The Past
• My First: one box
• VAX; cluster
• 2003: VAX + student server
• Later 2003: wiggum -> otto
• Summer 2005: performance?

135. Failures
aka: what went wrong

136. Spring 2006

137. Fall 2007-Spring
2008

138. Problems

139. Problems
•
Cache

140. Problems
•Cache
•SPAM accuracy

141. Problems
•Cache
•SPAM accuracy
•Local SPAM

142. Problems
•Cache
•SPAM accuracy
•Local SPAM
•Quotas

143. Alternatives /
Future
• Smarter Load Balancers
• IMAP server with local caching
• Shared block storage (mini
clusters?)
• Exchange
• Outsourced (Google)

144. Building your own?
• Plan for Growth
• Understand how to scale everything
O(1) > O(log(n)) > O(n)

145. Questions!