Re: “Hey:” – An Analysis of the Obama/Romney Emails(klaviyo.com)
klaviyo.com
Re: “Hey:” – An Analysis of the Obama/Romney Emails
http://www.klaviyo.com/blog/2012/11/06/re-hey-an-analysis-obamaromney-emails/
8 comments
Along the same topic, Mother Jones had an article last month about the inside of the Obama campaign's tech operation (or at least as much as they felt comfortable revealing) that was pretty intriguing: http://www.motherjones.com/politics/2012/10/harper-reed-obam...
(I just noticed OP is the author, so I'll put my comment here) [edit : correction, OP is the author's brother, sorry :]
I'm absolutely no mailing specialist, but here is my take :
Your study is quite interesting, but has it any value without you saying which mails your read when ?
Tracking if an email has been read or not is quite simple, providing the email is sent as HTML and the one who reads it is no privacy-nerd (noscript, ghostery, other ways to block external ressources, highly secure mailbox softwares, etc). What if you regularly "read" Obama's mails and not Romney's ones ?
If I were to manage this kind of mailing campain, I'd probably put on hold for a while email addresses who haven't read the last X mails. On the contrary, I would go on sending mails to people reading them. Etc.
Please tell us what you did "read" and when, and what you did not.
I'm absolutely no mailing specialist, but here is my take :
Your study is quite interesting, but has it any value without you saying which mails your read when ?
Tracking if an email has been read or not is quite simple, providing the email is sent as HTML and the one who reads it is no privacy-nerd (noscript, ghostery, other ways to block external ressources, highly secure mailbox softwares, etc). What if you regularly "read" Obama's mails and not Romney's ones ?
If I were to manage this kind of mailing campain, I'd probably put on hold for a while email addresses who haven't read the last X mails. On the contrary, I would go on sending mails to people reading them. Etc.
Please tell us what you did "read" and when, and what you did not.
Don't most modern email clients block image (and other resource) downloads until the user specifically requests them or whitelists the sender?
The best guidelines I have found for crafting HTML emails supported by all major clients:
Campaign Monitor Email Design Guidelines (scroll down for their discussion on images)
http://www.campaignmonitor.com/resources/will-it-work/guidel...
Image Blocking:
http://www.campaignmonitor.com/resources/will-it-work/image-...
CSS Support:
http://www.campaignmonitor.com/css/
Campaign Monitor Email Design Guidelines (scroll down for their discussion on images)
http://www.campaignmonitor.com/resources/will-it-work/guidel...
Image Blocking:
http://www.campaignmonitor.com/resources/will-it-work/image-...
CSS Support:
http://www.campaignmonitor.com/css/
The one I use does, yes, and probably most of them do. And I really like this feature. But I'm certain that if I wanted to support a candidate and subscribed to his newsletter, I would allow all images from sender because I would want to see the pictures, or just because the newsletter layout looks better with images.
And people managing these kind of newsletters probably know that. They design nice layouts for their newsletters, with images and fancy stuff to be "visually friendly". And they know people will want to let pictures in, and they know they will then have the possibility to track views. They even include the infamous if this email isn't properly displayed, see it here [http: //link/mail/online/?here_is_who_clicked_this_link=your_name] (usually more subtle than that.)
And people managing these kind of newsletters probably know that. They design nice layouts for their newsletters, with images and fancy stuff to be "visually friendly". And they know people will want to let pictures in, and they know they will then have the possibility to track views. They even include the infamous if this email isn't properly displayed, see it here [http: //link/mail/online/?here_is_who_clicked_this_link=your_name] (usually more subtle than that.)
I was testing out mailchimp the other day, and I could not for the life of me get the thing to recognize that I had read my test e-mails to my accounts, even though I had.
IIRC (been over a year since I used Mailchimp) they don't do analytics on test emails.
It's a great call - I'll post a follow-up with that detail and try to find a couple other examples from people in other geographies and demographics. My take (from this exercise and looking at ProPublica's data) is that behavior varies significantly based on both WHO campaigns think you are and WHAT you've done lately (what you've read, which sites you've visited, have you donated or volunteered, etc).
This complexity is going to make analysis nearly impossible in the future as political (and marketing) messaging becomes incredibly personalized.
(I'm actually author and OP here is my also-HN reading brother who beat me to the punch)
This complexity is going to make analysis nearly impossible in the future as political (and marketing) messaging becomes incredibly personalized.
(I'm actually author and OP here is my also-HN reading brother who beat me to the punch)
Can you please fix the first graph in the post? It's very hard to interpret 'gaps' in emails sent when you have Jul-12 and Sep-12 repeated.
Does anyone know what firms the campaigns are using for the campaign emails, or is that not public knowledge?
You can check by inspecting the mailing headers.
The Obama campaign uses Blue State Digital to send emails. The Romney campaign uses ExactTarget.
The Obama campaign uses Blue State Digital to send emails. The Romney campaign uses ExactTarget.
That's pretty interesting to learn. Blue State Digital's allegiance is obvious. ExactTarget is based in Indiana, a red state.
(Though I'm sure their greatest allegiance is to green.)
(Though I'm sure their greatest allegiance is to green.)
At least two years ago, most campaigns were using systems built by their respective parties, which were tightly integrated with their own voter databases. Things may have changed, but having a sibling working a big senate campaign this election cycle, I can tell you there's still a lot of room for improvement.
Page is down. Google Cache:
http://webcache.googleusercontent.com/search?pws=0&q=cac...
http://webcache.googleusercontent.com/search?pws=0&q=cac...
I'm working to get it back up, but it looks like our host will keep us down for 30 minutes or an hour. Good sign that we need to invest in better hosting.
As the OP mentioned, ProPublica has also done an analysis of emails, which they asked readers to send to them:
http://projects.propublica.org/emails/
It's hard to know how sophisticated the actual targeted emails are. The PP analysis likely didn't capture the whole universe of emails received (because it was mostly from their readership)...and the skeptical part of me wonders: how sophisticated are campaigns at this, really?
For one thing, a limiting factor is how much they are willing to spend to tailor a large variance of emails. I guess from the limited knowledge I have of campaign internals and campaign writers, knowing how to work a binary decision tree would be a rare skill. Of course they might have a program that abstracts this process, but you still have the core issue of writing an intelligible email that remains intelligible (and won't be an inadvertent embarrassment) across all variances.
Also, I wonder if open-rates have a stronger effect on the email format than any other kind of personal information, other than gender, age, geography, and donation history.
--
I have no doubt that campaigns think they are very sophisticated, especially compared to what they had in the last cycle four years ago. I'm talking about sophistication as it compares to what Facebook and Google have. That's an unfair bar here but c'mon, that's what we should be comparing against in the big picture.
It's hard to know how sophisticated the actual targeted emails are. The PP analysis likely didn't capture the whole universe of emails received (because it was mostly from their readership)...and the skeptical part of me wonders: how sophisticated are campaigns at this, really?
For one thing, a limiting factor is how much they are willing to spend to tailor a large variance of emails. I guess from the limited knowledge I have of campaign internals and campaign writers, knowing how to work a binary decision tree would be a rare skill. Of course they might have a program that abstracts this process, but you still have the core issue of writing an intelligible email that remains intelligible (and won't be an inadvertent embarrassment) across all variances.
Also, I wonder if open-rates have a stronger effect on the email format than any other kind of personal information, other than gender, age, geography, and donation history.
--
I have no doubt that campaigns think they are very sophisticated, especially compared to what they had in the last cycle four years ago. I'm talking about sophistication as it compares to what Facebook and Google have. That's an unfair bar here but c'mon, that's what we should be comparing against in the big picture.
This point about Google and Facebook is interesting, not least because Google has certainly had numerous people join the campaigns. My guess would actually be that the level of testing by the campaigns (just based on the small subset of data that PP has) is incredible and much, much better than either of the tech giants - regarding email.
As a benchmark, the most analytically savvy paper mailers / marketers of the last 20 years could arguably be Capital One. They did elaborate testing that involved not running TV ads in markets for years at a time, tons of different message styles, frequencies, etc - all because optimizing these results was worth so much to them.
In short, if getting email right means millions of dollars, then there's a premium on getting it right - just as there is for Google for nailing search testing.
As a benchmark, the most analytically savvy paper mailers / marketers of the last 20 years could arguably be Capital One. They did elaborate testing that involved not running TV ads in markets for years at a time, tons of different message styles, frequencies, etc - all because optimizing these results was worth so much to them.
In short, if getting email right means millions of dollars, then there's a premium on getting it right - just as there is for Google for nailing search testing.
I think this is a good point about how the campaigns do have virtually of the resources to put together a sophisticated emailing system...but it may not be a question of pure resources, but of logistics and ingenuity.
Here are two things that I think we can assume:
1) The campaigns have enough data to draw a good guess of who/what you are like, even without you giving anything more than your location, gender and age. This was true at least a decade ago when campaigns (and other third-parties) had access to databases such as subscriber data.
2) The campaigns have the resources to tie this data to individual identities, even if you haven't explicitly done it yourself when signing up for a newsletter. The matching won't be 100% accurate, but the campaigns can be reasonably sure that you are this particular Jane Doe at this address who is 28-32 years old, who subscribes to Rolling Stone and Harper's, and who drives a 2-door sedan.
Then what?
There has to be a middleman who can use this granular data to write a coherent message that leverages the insights from the data...using a model that makes many permutations across the most common combinations of characteristics.
This is similar to how Google can tailor search results for "what's a good movie" for a near infinite combination of user characteristics...but producing discrete databits (search results) is a different problem than producing a coherent fundraising letter.
And while money/resources/desire may not be an obstacle, the question is logistics and other practical concerns that divide the data from the content producers. There are many industries (such as the medical community) in which money and brains are no factor and yet have unsophisticated ways of dealing with information.
Here are two things that I think we can assume:
1) The campaigns have enough data to draw a good guess of who/what you are like, even without you giving anything more than your location, gender and age. This was true at least a decade ago when campaigns (and other third-parties) had access to databases such as subscriber data.
2) The campaigns have the resources to tie this data to individual identities, even if you haven't explicitly done it yourself when signing up for a newsletter. The matching won't be 100% accurate, but the campaigns can be reasonably sure that you are this particular Jane Doe at this address who is 28-32 years old, who subscribes to Rolling Stone and Harper's, and who drives a 2-door sedan.
Then what?
There has to be a middleman who can use this granular data to write a coherent message that leverages the insights from the data...using a model that makes many permutations across the most common combinations of characteristics.
This is similar to how Google can tailor search results for "what's a good movie" for a near infinite combination of user characteristics...but producing discrete databits (search results) is a different problem than producing a coherent fundraising letter.
And while money/resources/desire may not be an obstacle, the question is logistics and other practical concerns that divide the data from the content producers. There are many industries (such as the medical community) in which money and brains are no factor and yet have unsophisticated ways of dealing with information.
My guess is that over a billion dollars can buy you some pretty damn sophisticated marketing. What email campaigns have more on the line than these?
Well, multi-billion dollar corporations like Google, for one.
[deleted]
I get Bluehost's "The website you were trying to reach is temporarily unavailable."
[deleted]