What good Web Developers should know about sending E-mail(diaryofaninja.com)
diaryofaninja.com
What good Web Developers should know about sending E-mail
http://www.diaryofaninja.com/blog/2011/09/27/what-all-good-web-developers-should-know-about-sending-email#.ToFdSHQ-fN0.hackernews
38 comments
I managed my own mail server for scribophile.com for a year or two. Figuring out how to reliably send email is a complete nightmare and as much black magic as it is engineering. When I finally had to move to new hosting, the thought of having to migrate my email server settings and basically starting over scared the crap out of me. That's when I discovered Sendgrid, and I've been using them since then. Whatever their monthly cost is, it's worth it for not having to deal with the email configuration nightmare.
Hear, hear! At $work we started using Sendgrid last month, and it's worth 10x the money we pay every month (Silver level is $80/mo for 300k emails). Our usage will be ramping up over the coming months, and I feel so much more secure knowing that the chances of getting on a blacklist are greatly reduced because of Sendgrid.
Related, anyone know of anyone providing hosted mailing lists, e.g. ezmlm or mailman? I've been looking for someone providing this and not finding much of anything.
Have you looked at Mailchimp? It's not as much for transactional emails as it is for scheduled, repeated broadcast emails, so I suspect it's probably more along the lines of what you're looking for.
I know they also provide the web interface (and APIs) for allowing users to subscribe, unsubscribe and all that jazz.
I know they also provide the web interface (and APIs) for allowing users to subscribe, unsubscribe and all that jazz.
I am using mailchimp for multiple clients, and AWS simple email sending for some transactional clients; they are all quite helpful. Anyone recommending sendgrid, why is it a better choice?
When faced with sending email from my app, I chose to outsource everything, in order to not learn this stuff..
I set up an account at PostMark (postmarkapp.com), which made it very simple to start sending emails. I did have to change a few DNS settings, but they explained exactly how, and after that everything was a breeze.
I really recommend them to anyone who doesn't want to learn this stuff. It's pretty cheap too (I'm on the free tier so far, but it's a few bucks for a 1000 emails).
I set up an account at PostMark (postmarkapp.com), which made it very simple to start sending emails. I did have to change a few DNS settings, but they explained exactly how, and after that everything was a breeze.
I really recommend them to anyone who doesn't want to learn this stuff. It's pretty cheap too (I'm on the free tier so far, but it's a few bucks for a 1000 emails).
I looked into PostMark, but sendgrid is less than 1/10 of the price, so I am not quite sure why anybody would stay with postmark.
Sendgrid would not reliably send our emails fast enough. We were getting queued for any gmail address we were sending too and sendgrid told us that there was nothing they could do about it. Since we have a requirement to get into peoples inbox ASAP we bailed on them as they couldn't help.
Then we went to critsend, which was great until we ran into a problem and couldn't get in touch with customer service for a week (and they never got back to us).
Then we tried Dyn since they called us up after reading a press hit and offered us a free trial period. Things were good there until we found out we couldn't deliver to comcast.com email addresses. Again, they were no help.
So we switched to Postmark and things have been awesome. Yes, you pay more than some other places, but your email gets to you recipients and it does so fast.
And yes, we had our DNS configured configured correctly for each of those services.
Then we went to critsend, which was great until we ran into a problem and couldn't get in touch with customer service for a week (and they never got back to us).
Then we tried Dyn since they called us up after reading a press hit and offered us a free trial period. Things were good there until we found out we couldn't deliver to comcast.com email addresses. Again, they were no help.
So we switched to Postmark and things have been awesome. Yes, you pay more than some other places, but your email gets to you recipients and it does so fast.
And yes, we had our DNS configured configured correctly for each of those services.
Good point. I looked into SendGrid, but went with PostMark afterwards. It was easier to set up.
The reason, if I recall, is that SendGrid made emails come out with a "sent via SendGrid" header, which I didn't like. To remove it, I'd have had to sign up for a paid deal with SendGrid. PostMark let me get things working first, then decide whether to pay or not.
The reason, if I recall, is that SendGrid made emails come out with a "sent via SendGrid" header, which I didn't like. To remove it, I'd have had to sign up for a paid deal with SendGrid. PostMark let me get things working first, then decide whether to pay or not.
When implementing Sendgrid, I think I spent about 30 min reading over the DNS setup docs and 30 min actually creating the records. I'm no sysadmin, but I thought it was quite easy.
Also, they have a free account which allows you to send 200 emails per month. That would allow you to try it out and then decide that Yes Virginia, it is worth the money. :)
Also, they have a free account which allows you to send 200 emails per month. That would allow you to try it out and then decide that Yes Virginia, it is worth the money. :)
Is Amazon Simple Email Services an option?
It sure is.
We are currently using Postmark to send notifications to our users. Postmark does not allow bulk emails like newsletters, and our user count is way beyond the ceiling of Newsberry (Postmark's bulk sending service). Yes, we could negotiate a deal with them, but it's just not as transparent as we want it to be. Frankly, having to worry about the number of subscribers is kinda painful for a fast-growing startup---we cannot predict how many more users we will get next month!
Not to mention the price of Postmark is 15x that of Amazon SES. It's not a big deal for a small amount of emails, but it accumulates pretty quickly when we need to send frequent updates, weekly newsletters, and occasionally campaign messages. We have our own analytic systems and template designers, which make Postmark's value added services useless for us (keep in mind that's not to say it's useless to you).
What we need is a reliable and affordable mailing service with a flat, per use fee. Amazon SES fits the bill nicely. We rolled out a new delivery system using SES and just finished sending a testing batch of newsletters. Simple put, SES is amazing: bounce rate is well below 0.2%, almost no complaints, and zero rejects for a 100k batch! We will test a few more times later, and if everything goes well like this, we will completely switch over to SES.
We are currently using Postmark to send notifications to our users. Postmark does not allow bulk emails like newsletters, and our user count is way beyond the ceiling of Newsberry (Postmark's bulk sending service). Yes, we could negotiate a deal with them, but it's just not as transparent as we want it to be. Frankly, having to worry about the number of subscribers is kinda painful for a fast-growing startup---we cannot predict how many more users we will get next month!
Not to mention the price of Postmark is 15x that of Amazon SES. It's not a big deal for a small amount of emails, but it accumulates pretty quickly when we need to send frequent updates, weekly newsletters, and occasionally campaign messages. We have our own analytic systems and template designers, which make Postmark's value added services useless for us (keep in mind that's not to say it's useless to you).
What we need is a reliable and affordable mailing service with a flat, per use fee. Amazon SES fits the bill nicely. We rolled out a new delivery system using SES and just finished sending a testing batch of newsletters. Simple put, SES is amazing: bounce rate is well below 0.2%, almost no complaints, and zero rejects for a 100k batch! We will test a few more times later, and if everything goes well like this, we will completely switch over to SES.
It is; we use it and it is very reasonably priced. However their APIs are very painful to work with and there is no web interface.
Is setting up a (free!) google apps account on your domain, and then using google (gmail) a viable option? (as long as you are within gmails limits on sending emails)
Not a very good option in my experience. Google has some pretty complicated abuse detection algorithms in place. For example, if an account sends email through SMTP, but never receives any email, logs in via POP, or through the web interface, the account may get shut down. Also, Google's tolerance for "Mark as Spam" on the email you send is very, very low. It boils down to this: Google Apps isn't the same as SendGrid, Postmark, etc. Google Apps is intended for use by people, not web apps.
Having said all that, these are my experiences. I've tried it on three separate occasions in the past, and every time, I ended up fighting deliverability problems until I finally moved to a different solution. The most common is that the Google Apps user through which you send email gets disabled for "abuse", and you have to log in through the web interface in order to re-enable it.
Having said all that, these are my experiences. I've tried it on three separate occasions in the past, and every time, I ended up fighting deliverability problems until I finally moved to a different solution. The most common is that the Google Apps user through which you send email gets disabled for "abuse", and you have to log in through the web interface in order to re-enable it.
I do this for a variety of reasons (including that not all my dev machines have working email), and it works well for us. Not a massive amount of email being sent, but seems to work well. Any horror stories?
I used Google AppEngine for a mobile game that sent email to notify players. While there was a link to turn off emails in every email and a setting in the app, enough people clicked "mark as spam" that we got shut off.
It happened a couple of times and each time Google turned us back on. My general experience is that Google is more heavy handed than a true ESP like Sendgrid or Postmark. ESPs take spam seriously, but they also know spam reports are a fact of life.
It happened a couple of times and each time Google turned us back on. My general experience is that Google is more heavy handed than a true ESP like Sendgrid or Postmark. ESPs take spam seriously, but they also know spam reports are a fact of life.
I'm the Product Manager for PostageApp (www.postageapp.com) and we built our email delivery engine from the ground up (rather than using existing solutions like Port25) and we learned a huge deal about getting emails into inboxes.
This article is a fantastic start and will definitely help you out with delivering email, but there are some minor nuances among different SMTP servers and email providers that are just so quirky that you really can't account for 100% of everything.
Organic growth of your email volume to users who genuinely want your emails is a great way to make sure you stay in their good books, as well as observing some SPAMAssassin rules and auditing your subject line and content to not seem spammy.
It's a huge and complicated matter (getting emails into inboxes) and it's not documented, thankfully we are getting a better and better idea of how to do it.
This article is a fantastic start and will definitely help you out with delivering email, but there are some minor nuances among different SMTP servers and email providers that are just so quirky that you really can't account for 100% of everything.
Organic growth of your email volume to users who genuinely want your emails is a great way to make sure you stay in their good books, as well as observing some SPAMAssassin rules and auditing your subject line and content to not seem spammy.
It's a huge and complicated matter (getting emails into inboxes) and it's not documented, thankfully we are getting a better and better idea of how to do it.
What good web developers should know about sending e-mail:
fuck it, use postmark.
fuck it, use postmark.
Lol - agreed, although send grid is more my favourite especially for small sites where the free account that allows for 200 emails usually covers you
That site hurts my eyes.
lol - thats what happens when you get a developer to design a site.
its on the never ending list of todo's thats for sure
its on the never ending list of todo's thats for sure
Other things that may help:
- Promptly remove addresses that bounce back (mail to dead addresses is a red flag).
- Set up feedback loops with the major ISPs so you can remove people who report you as spam.
- Track image opens and clickthroughs and unsubscribe people who haven't read your emails in a year.
- Make your unsubscribe link prominent.
- Promptly remove addresses that bounce back (mail to dead addresses is a red flag).
- Set up feedback loops with the major ISPs so you can remove people who report you as spam.
- Track image opens and clickthroughs and unsubscribe people who haven't read your emails in a year.
- Make your unsubscribe link prominent.
This is why SendGrid is so valuable. I learned all those tricks for getting emails into inboxes, but it took me a while to learn and it was a constant battle keeping up with everything.
Good web developers should have their logo / sitename linked to the home page - usability 101. :p
Email, still not dead.
My work has gradually moved from Web -> Email. I learned about email while working as a sys admin for a website sending 100k mailouts and now I've become a bit of a specialist writing email apps. Postfix is my weapon of choice, you can extend it externally via its unix pipe & socket routing.
Currently I'm building an outgoing SMTP service for people network hopping - Wifi, 3g etc.
It has been interesting to see the rise in inter-SMTP server communications going SSL. Google & Hotmail, for instance, both use TLS to send their email, even when there's no requirement to do so, TLS is chosen first.
So people often trumptet BBM as end-to-end secure but if you use Google Mail with your mobile email and set it to TLS only and only send to other Google & Hotmail users then the same is true.
I've also had a report of a German ISP that deep packets SMTP traffic and changes STARTTLS to XXX "for security reasons". I suspect that they have a tap in front of their closed source MTA to log the traffic to satisfy the European Data Retention Directive.
My work has gradually moved from Web -> Email. I learned about email while working as a sys admin for a website sending 100k mailouts and now I've become a bit of a specialist writing email apps. Postfix is my weapon of choice, you can extend it externally via its unix pipe & socket routing.
Currently I'm building an outgoing SMTP service for people network hopping - Wifi, 3g etc.
It has been interesting to see the rise in inter-SMTP server communications going SSL. Google & Hotmail, for instance, both use TLS to send their email, even when there's no requirement to do so, TLS is chosen first.
So people often trumptet BBM as end-to-end secure but if you use Google Mail with your mobile email and set it to TLS only and only send to other Google & Hotmail users then the same is true.
I've also had a report of a German ISP that deep packets SMTP traffic and changes STARTTLS to XXX "for security reasons". I suspect that they have a tap in front of their closed source MTA to log the traffic to satisfy the European Data Retention Directive.
>So people often trumptet BBM as end-to-end secure but if you use Google Mail with your mobile email and set it to TLS only and only send to other Google & Hotmail users then the same is true.
I actually thought they had end-to-end encryption with different keys for every device (PGP-like). That would make them much more secure than that, since the servers wouldn't be able to access the contents.
But I've been reading about it before replying and apparently they use a single key per server, not to mention that if you're not on a private BES, you're using a global key (they call it 'scrambled', not encrypted). What a joke.
I actually thought they had end-to-end encryption with different keys for every device (PGP-like). That would make them much more secure than that, since the servers wouldn't be able to access the contents.
But I've been reading about it before replying and apparently they use a single key per server, not to mention that if you're not on a private BES, you're using a global key (they call it 'scrambled', not encrypted). What a joke.
interesting you've found that about BES - link?
EDIT: It's mostly about BIS; BES servers can actually implement end-to-end, if they're IT department enables the S/MIME module and create, distribute and teach users how to use PKI certificates.
But if you're not doing that, you're not really protected.
This[1] document from the Communications Security Establishment of Canada explains it well. Citing:
This[1] document from the Communications Security Establishment of Canada explains it well. Citing:
PIN-to-PIN transmission security: PIN-to-PIN is not suitable for exchanging
sensitive messages. Although PIN-to-PIN messages are encrypted using
Triple-DES, the key used is a global cryptographic “key” that is common to
every BlackBerry device all over the world. This means any BlackBerry device
can potentially decrypt all PIN-to-PIN messages sent by any other BlackBerry
device, if the messages can be intercepted and the destination PIN spoofed.
Further, unfriendly third parties who know the key could potentially use it to
decrypt messages captured over the air. Note that the “BlackBerry Solution
Security Technical Overview” document published by RIM specifically
advises users to “consider PIN messages as scrambled, not encrypted”.
I'd love to hear more about the outgoing SMTP service for people network hopping - I'm not sure I completely understand it and my curiosity is just getting the best of me. :)
Tell us more!
Tell us more!
Buy a Data sim, what to do as outgoing SMTP ?
Vodafone for instance : http://www.google.co.uk/search?q=vodafone+smtp
Our service, one of a few, offers SMTP Auth'd or just relying on MAIL FROM: envelope header for outgoing email, utilising TLS, especially useful when on free-wifi.
Set your SPF settings to allow our sending IP in order to improve one's SPAM score.
The TLS route means that if one is sending to a GMAIL account then it is the recipient who can break the chain of trust.
If you have an SMTP receiver you can send from an open wi-fi access point in Karachi and know that only in server was the plain text processed. I guess that's not saying much but you can easily set it up for yourself.
Vodafone for instance : http://www.google.co.uk/search?q=vodafone+smtp
Our service, one of a few, offers SMTP Auth'd or just relying on MAIL FROM: envelope header for outgoing email, utilising TLS, especially useful when on free-wifi.
Set your SPF settings to allow our sending IP in order to improve one's SPAM score.
The TLS route means that if one is sending to a GMAIL account then it is the recipient who can break the chain of trust.
If you have an SMTP receiver you can send from an open wi-fi access point in Karachi and know that only in server was the plain text processed. I guess that's not saying much but you can easily set it up for yourself.
Hopefully I'm understanding this correctly: you're creating a service that lets you send emails securely using your hosted SMTP, from anywhere in the world?
Useful for traveling on business, I would assume!
Useful for traveling on business, I would assume!
Yes, that's the one. There are more than one available. I'm not going to spam the name of mine.
It's a service I wanted myself as an early adopter of handheld computers, outgoing SMTP is one of those things where the ISP does everything it can to make you use [email protected] and even when it is plain, remembering the settings if your phone gets wiped, is it mail.example.isp.com or smtp.example.isp.com. Hopefully it is a bit easier to remember the smtp service one uses.
What am I saying: end to end encryption isn't there, just in-flight encryption. Encrypting the payload before it leaves your network is the only way to do that, not relying on intermediaries.
It's a shame we can't sign it for you as it leaves our network, but that would defeat the point.
It's a shame we can't sign it for you as it leaves our network, but that would defeat the point.