Ask HN: How to learn Data Analysis?
2 comments
You're fluent with regressions and stats. Sounds like you're competent with the mechanics. Where you might have gaps, there are some great tools that can do your heavy lifting. But, that's only half of the battle.
When you're trying to do meaningful data analysis, you really have to understand your dataset. Fancy math can't substitute for domain expertise. Think long and hard about what's in the set, what the causal connections might be, and how an "expert" in the field might approach the problem.
The guys over at OKCupid are awesome at this. Check out this post to see what I'm talking about.
http://blog.okcupid.com/index.php/dont-be-ugly-by-accident/
Their advice on taking a good picture is just about exactly what my professional photographer mother recommends. Good stuff. But, the way they analyzed and presented the data shows (a) exactly how powerful putting numbers on something subjective can be and (b) that they know their domain.
If you read through the other posts (do that), you'll see that they have a solid understanding of their dataset. They know what to look for, namely photo attractiveness. They know how to get good data on that (the dependent variable) and they know which independent variables probably matter the most.
Throwing math at a complex dataset can be useful (e.g. bayes spam classifiers), but if you really want to do something that will work well or "speak" to a client, invest a bit of time in understanding the field.
When you're trying to do meaningful data analysis, you really have to understand your dataset. Fancy math can't substitute for domain expertise. Think long and hard about what's in the set, what the causal connections might be, and how an "expert" in the field might approach the problem.
The guys over at OKCupid are awesome at this. Check out this post to see what I'm talking about.
http://blog.okcupid.com/index.php/dont-be-ugly-by-accident/
Their advice on taking a good picture is just about exactly what my professional photographer mother recommends. Good stuff. But, the way they analyzed and presented the data shows (a) exactly how powerful putting numbers on something subjective can be and (b) that they know their domain.
If you read through the other posts (do that), you'll see that they have a solid understanding of their dataset. They know what to look for, namely photo attractiveness. They know how to get good data on that (the dependent variable) and they know which independent variables probably matter the most.
Throwing math at a complex dataset can be useful (e.g. bayes spam classifiers), but if you really want to do something that will work well or "speak" to a client, invest a bit of time in understanding the field.
thank you for your valuable suggestion, I will definitely spend time in understanding the data first!
http://blog.stephenwolfram.com/2012/03/the-personal-analytic...
This is on the front page now, but holy crap is it beautiful. This is everything data analysis should be.
This is on the front page now, but holy crap is it beautiful. This is everything data analysis should be.
+1 to this. This book rocks.
There's also a UC Berkeley data science class - http://datascienc.es/ - that's helpful.
There's also a UC Berkeley data science class - http://datascienc.es/ - that's helpful.
This class looks good, also Jeff is a well known person in this field, I am going to attend it offline for sure.
Thank you !
Thank you !
[deleted]
this book just shipped at my home, excited! also started learning R from 'R in Action'
Thank you for your feedback!
Thank you for your feedback!
this book is on sale for 50% off at oreilly.com until march 2nd, along with other Data related ebooks and video.
I started trying one of the problems in Kaggle
When I fit a logistic Regression, the algorithm gave me poor results.
I saw that people do lot of data analysis before applying any machine learning algorithm
I completed a book in Statistics and want to learn about Data Analysis
Please help me identifying resources/courses/tools that I can take to learn Data Analysis
Thank you