I co-founded a company that found itself with a similar set of criteria for what we wanted. For us, VC worked out well. We weren't the typical company they saw, but going in with a very clear set of things we wanted from a partner was very helpful. Ultimately, they were high value add when relevant but otherwise let us focus on growing the business.
My guess is it comes down to the right VC/Partner and finding mutually agreeable terms.
I think it probably is, it just takes more planning. Take email A/B testing - if email is a main driver of your usage activity, you could split your users or list into two groups and then send them different emails over a period of months. It'd be slightly more manual (you'd have to segment the list) and then leave one group off for a few months, but it would let you easily gather very significant data.
These are great points, and speak to why measuring the impact of email is really, really important. Until you're holding your emails accountable to performance (whether it's getting someone to come back or to buy), you're just throwing darts in the air and potentially annoying someone.
At Klaviyo (I'm one of the co-founders, so note bias), we've helped many web apps and ecommerce sites setup full email strategies - and we definitely find that some emails don't work. You've got to test different emails and strategies, see what works - and then ideally optimize and personalize what (or even whether) you are sending based on individuals' actions.
Having now spent a lot of time sending and analyzing email campaigns/triggered emails (I'm one of the co-founders of Klaviyo), we keep seeing that email is one of the best ways to turn analysis into action - not least because it gives you a complete feedback loop.
For example, say an Ecommerce store realizes most people only make 1 purchase and don't come back. With email, you can target exactly those people, but you also can quickly see if it worked (because you know who you sent it to, you can see if they actually made a purchase).
Retention is definitely really important, but I think what's at root here is that email is an ideal way to interact with users in a more targeted and personal way outside of when they proactively visit your website. The same feedback loop idea should (and will) apply to push notifications, texts, in-app messaging, etc as time goes on.
On the note of smart views, any idea how representative my data is of the typical email user?
Separately, I'm curious to know how far smart views go with aggregation - i.e. if I get 10 facebook friend request emails, are they combined? Or is this an irrelevant question in the smart view paradigm?
This is a really interesting point - I think the question will really come down to how well campaigns / businesses can measure the impact of their marketing. If your messages/posts aren't driving donations or purchases (or hurt the long term usefulness of the person you're marketing to), then you won't run them.
The problem is that today most marketing is blind - completely unlinkedin from the impact it causes. I'm counting on technology / big data / analytics to change that.
This point about Google and Facebook is interesting, not least because Google has certainly had numerous people join the campaigns. My guess would actually be that the level of testing by the campaigns (just based on the small subset of data that PP has) is incredible and much, much better than either of the tech giants - regarding email.
As a benchmark, the most analytically savvy paper mailers / marketers of the last 20 years could arguably be Capital One. They did elaborate testing that involved not running TV ads in markets for years at a time, tons of different message styles, frequencies, etc - all because optimizing these results was worth so much to them.
In short, if getting email right means millions of dollars, then there's a premium on getting it right - just as there is for Google for nailing search testing.
I'm working to get it back up, but it looks like our host will keep us down for 30 minutes or an hour. Good sign that we need to invest in better hosting.
It's a great call - I'll post a follow-up with that detail and try to find a couple other examples from people in other geographies and demographics. My take (from this exercise and looking at ProPublica's data) is that behavior varies significantly based on both WHO campaigns think you are and WHAT you've done lately (what you've read, which sites you've visited, have you donated or volunteered, etc).
This complexity is going to make analysis nearly impossible in the future as political (and marketing) messaging becomes incredibly personalized.
(I'm actually author and OP here is my also-HN reading brother who beat me to the punch)
It's good to see Mixpanel building out its people analytics offering, but more broadly good to see momentum in the trend towards measuring the impact of email and other communications on customer behavior. To this point in time, to many companies send emails in a spray and pray fashion - not knowing if they really impacted customer behavior on feature usage, purchases, etc.
We're working on the some similar problems at Klaviyo (customer lifecycle management / targeting customers and measuring the impact). One of the most interesting related things we've seen is that measuring the impact of emails goes along way towards eliminating debate about email frequency or whether to send campaigns - because you can always just send the campaign to a subset and actually know if it works.
From Patrick's course to articles like these, it's great to see the growth in interest in this. The challenge with LTV (as with all analysis) is making sure that you can translate it into action. For many companies, the barrier is just getting started.
On a broader note, it's great to see how many companies are tackling customer lifecycle management (though with very different approaches and different end markets). It's time people stopped building one-offs for this.
A few companies I know of (the first of which I'm a co-founder of):
I think you make a great point here about the confusion that often gets put out there between data and analysis - a confusion which I'd say is implicit in the term big data as well (and hence I ran with - caveat, I'm the author).
As far as the problem with the term "intelligent data" - I think what you say is exactly true if you do data analysis one time; however, the issue is that for those of us running startups, we find ourselves doing analysis over and over - so intelligently selecting data (in a way that takes us less time together and leads to the same decisions) is a huge win. Read intelligent data as being data + intelligence - not a new type of data.
Likewise, the problem with asserting that more data is always better ignores how most companies are making decisions. At the end of the day, our analysis is completely meaningless without a new action. So a better analysis that doesn't get implemented is worth far less (nothing) than an analysis that gets implemented successfully and drives results.
Helpful feedback. For most of our current clients, we find that the amount of value we generate / cost of what we replace (lots of engineering work and excel usage) is a lot higher than this.
That said, I think our sense is that for startups / newer firms it would be great to come up with some sort of plan that makes this more feasible. We'd love to hear thoughts and we're certainly glad to figure out something that works based on specific needs of people. Email me at [email protected] and glad to chat.
Never thought of the Klout point, but it's a good way for us to think about it.
Good question. Our current focus has been on high-margin businesses with relatively few (hundreds to tens of thousands, not millions) customers, primarily because they seem to really feel this problem.
Integrations is both through 3rd parties (for things like Mailchimp, Zendesk, etc) where it just takes a couple of clicks and no development and through our API - we have both a javascript and HTTP API. Our javascript snippet gives us a logins per user, but you can really pass us anything you want (has this user finished setup, have they used feature x, etc)
This stuff is hard to get right. I think the answer for most web apps is to shift more towards trigger based emails, and then to measure how particular rules change customer behavior over the next 24 hours, week, 30 days, etc. Rather than an "A/B" test per se, you really need to split customers into 2 groups and intentionally not send an additional email to a group.
Good look at the history of the marketing term big data. I think this fits squarely in the realm of big data hype, or at least a review of why it is big data hype (something I've blogged about here http://www.klaviyo.com/blog/2012/07/16/the-curse-analytics-b...). The SAS discussion makes it especially clear that it's viewed as a trend.
But - posts like these frequently jump to the top of hacker news, and there's clearly a feeling that access to new analyses and more data is going to change the world. I think this is probably right. I'd love to hear more discussion of how big data has already started this process.
This article gets a few things right, but I find it largely to be another example of playing up the hype without talking about the real meat of big data's potential.
Some of the more useful points:
1.) Big data's promise has led to massive advances in technology, and companies are spending billions on this new technology
2.) The limiting factor with Big Data is analytical in nature, not necessarily in storage, processor speed, etc.
What this article fails to address is why big data is so valuable. While it gave one example of a real use case (identifying influencers in social networks), if anything this served to highlight the problem with Big Data. Are companies completely shifting their marketing budgets to target influencers? Is this driving major financial impact for Facebook?
I have no doubt that companies are actively identifying incredibly powerful new uses of big data, but for big data to be truly revolutionary, we need to develop the analytic methodologies and decision-making processes to benefit. Reading the McKinsey report cited in the article is illuminating. There's a ton of value cited, but it isn't particularly clear how companies changed their decisions based on big data in a way that drove the value.
This is a great addition to the unfortunately too limited category of articles about how big data / analysis changed actual decision-making.
That said, the definition of big data used by this article doesn't strike me as what I would think of as big data. Based on what it says, the dairy industry was changed by analysis and data collection but nothing that couldn't be stored on your typical phone. The article hints at big data (via greater genetic analysis) transforming the dairy industry in the future, but the massive changes in cow DNA so far are seemingly due to "small" data.
This confusion of big data with just solid analysis and decision-making happens a lot, but does a good job of highlighting how much progress there is to be made in using data to drive decisions (independent of how much data we use).
Great post - if anything, I think this problem goes way beyond just real-time dashboards to extend to the vast majority of analysis done today (both by web and more traditional companies).
The statement that particularly resonated for me was [figure out] "if you’re looking at stats now because you’re curious and impatient, or because those stats will actually drive business decisions", but I'd take it one step further.
Analysis and stats are incredibly valuable when:
1) they are applied to a real business decision that is tied to actual value
2) you are willing to change what you are doing based on the result
3) you don't already know the answer
4) you have sufficient confidence in the result to act on it.
We need to get more used to stopping and thinking before we start analysis by laying out the decision we need help making, the different paths we're willing to take, and the amount of confidence we'll need to change our decisions. I could definitely be a lot more disciplined about it.
(I actually wrote a blog post last week on this same topic that lays out the above criteria in more detail and might be useful - though that was a reaction to the profusion of ads out there calling Big Data and Analytics "hot", that ignore how they actually drive real value. Blog post is here: http://www.klaviyo.com/blog/2012/07/16/the-curse-analytics-b...)
I think this distinction between personal and impersonal emails is a good one. The frustration (for me at least) comes in when email isn't relevant, when it's redundant, or when it's unnecessary. My guess is that the problem isn't actually email itself - but it's instead the systems that keep it impersonal.
- There's no reason a company should email me about a feature I already use or a product I already buy. They know this - they just haven't bothered to integrate their systems.
- There's no reason I should have to send 10 emails to schedule a meeting where/when everyone can make it. Our calendars should be able to make this easy and painless.
- There's no reason I should get 4 separate emails from friends containing the same article. Can't I get one message saying four people sent you this?
As Seth Godin said in his Ted talk, something's broken. All of these examples I cite can be solved - and there's a ton more just like them.
My guess is it comes down to the right VC/Partner and finding mutually agreeable terms.