Bridging Economics and Data Science(medium.com)
medium.com
Bridging Economics and Data Science
https://medium.com/about-data/9351f95863e0
23 comments
I have an econ/finance degree and also worked for several years in finance. Then I quit my job and learned how to code. Now I have a web app with a paying and growing customer base. Knowing how to program will give you special powers when combined with your econ background. Congrats!
I have an econs background too. I learned to code in my final year of uni and now do some freelancing. You are right that knowing how to program will give you special powers!
Care to share your web app? I have a similar background, working in finance and learned coding, but I am still missing the step to building a web up and leaving work...
He nails the point, and then misses it himself:
the key to success is in cleverly selecting, finding, or creating a data source that answers a particular question
It's about asking the right question and then finding or generating the right data to answer that question. That's what makes it science.
the key to success is in cleverly selecting, finding, or creating a data source that answers a particular question
It's about asking the right question and then finding or generating the right data to answer that question. That's what makes it science.
When you need to study 30 plus income statements from 30 plus entities around the globe, you wonder less about the tools and more about getting it done. Then you become a "spreadsheet monkey", but that is only because nobody has been able to create a better tool since 1995.
I understand the pain of seeing people using old tech and taking hours to do tasks that should take seconds, but do not underestimate the importance of specific field knowledge and the fact that many people do not have time to learn coding. If you think about it, it is an opportunity for you to build a bridge between those 2 worlds. For example, nobody has produced yet a decent tool to consolidate financials...
I understand the pain of seeing people using old tech and taking hours to do tasks that should take seconds, but do not underestimate the importance of specific field knowledge and the fact that many people do not have time to learn coding. If you think about it, it is an opportunity for you to build a bridge between those 2 worlds. For example, nobody has produced yet a decent tool to consolidate financials...
This is the same in the social sciences. I'm currently a PhD student in one of the top Communication departments in the US, and it's painful to see how far behind in technical skills and tools the curricula are (eg., Excel and SPSS). I've been self-teaching Python, R, and SQL, and extending my knowledge base from simple regression-based stats to data mining and machine learning, to make up for it. Not only does that allow me to work on massive datasets (and push the field forward across methodologies), but it allows me to improve more 'traditional' approaches by sharing data and models (eg., with .R scripts).
I'm in a well regarded sociology department and there's a huge gap between the sociologists who take technical things seriously (programming, stats) and those who don't. People have actually rolled their eyes in classes when we read papers about simulations. I was lucky: we have a core group of students who program and gather data using modern tools, but I gather this is rare both within my discipline and without.
That's a real shame, data science will hopefully cut through the theoretical BS that are the results of so many social science thesis'.
Quite interesting; I've heard similar things. A good friend of mine, not as math-y as me, was a psychology PhD student in John Bargh's lab in Yale. She ended up getting her doctorate and leaving the field because she didn't want to be the statistics police.
A financial economist professor once mused that "current phd students are limited in the problems they can solve because they can't program." Of course the programming language he preferred was Fortran. :-)
Excel is the best tool for 80% of what bankers and consultants do. It can middle through the next 10%. The problem is it has just no way to do the last 10%. Either it's too slow or just can't handle the size or computations required.
Excel is the best tool for 80% of what bankers and consultants do. It can middle through the next 10%. The problem is it has just no way to do the last 10%. Either it's too slow or just can't handle the size or computations required.
Like the author, I have an Economics background but have gotten into programming as I graduated from working on Excel. Economics has suffered because of a lack of good data - this is why so many explanations by economists begin with assumptions. I'm hopeful that the data sets now available will improve economic models and that people working in the public sector will put them to good use.
I also come from an Economics background, and am now a software engineer/budding data scientist. As I've delved more into machine learning topics, I'm amazed (though not surprised!) at how both academic and industry economists are still mostly focused on running OLS/logit/probit regressions, and not other classification techniques. My undergraduate thesis did use some computational models that sought convergence for dynamic & stochastic conditions, but that was definitely not the norm.
Macroeconomics and empirical industrial organization are leading the forefront in terms of theoretical and applied technical advances. You ought to look at discrete choice analysis sometime--great stuff.
I can't speak for industry economists, but the reason we academics tend to spend so much time with OLS/Logit/Probit is their flexibility and scalability.
I can't speak for industry economists, but the reason we academics tend to spend so much time with OLS/Logit/Probit is their flexibility and scalability.
Macro was my favorite subject! I was lucky enough to take the first year PhD sequence during my last year, which was my first taste of coding =D
I think in industry (anti-trust at least), they stick with the older models because their value has legal precedent, and using new methods would require some more legal hand waving by the attorneys.
I think in industry (anti-trust at least), they stick with the older models because their value has legal precedent, and using new methods would require some more legal hand waving by the attorneys.
Economics has suffered because of a lack of good models and ignoring a great deal of available data.
Source: economics degree, research of the neoclassical / neoclassical synthesis model, its origins, and various heterodoxies. I'm partial to thermoeconomics / biophysical economics.
Source: economics degree, research of the neoclassical / neoclassical synthesis model, its origins, and various heterodoxies. I'm partial to thermoeconomics / biophysical economics.
Analysts in finance/consulting certainly haven't had to upgrade their skills, but analysts in media/marketing definitely have had to.
Certainly there are people in this space who can't do much beyond spreadsheets, but there are many analyst now who use python/pandas or R to do work.
Certainly there are people in this space who can't do much beyond spreadsheets, but there are many analyst now who use python/pandas or R to do work.
The biggest issue I see from the ivory tower I'm surrounded by is that economists typically doubt the results of data mining. Neural networks, machine learning, etc. are all well known toolkits in computational economics (one of my specialties) but the results from their application are rarely believed.
Jared Bernstein answers Barry Ritholz on related themes today:
http://jaredbernsteinblog.com/economics-as-market-failure/
It's really hard to tell, especially outside the field, whether someone's computation has found signal and not noise in their data series, or even whether that data series has any significance for different times and different places ...
(You can "Monte Carlo" the past as much as you want, it won't become the future.)
Edit: I probably should have just referenced Sliver's Signal and Noise and left it at that.
http://jaredbernsteinblog.com/economics-as-market-failure/
It's really hard to tell, especially outside the field, whether someone's computation has found signal and not noise in their data series, or even whether that data series has any significance for different times and different places ...
(You can "Monte Carlo" the past as much as you want, it won't become the future.)
Edit: I probably should have just referenced Sliver's Signal and Noise and left it at that.
Structural models are more 'insightful' than generative models in machine learning. Econ guys are more interested in Pr(y|x) than reproducing the data.
That being said, I really hope computational work gains more traction... this might be a marketing issue.
That being said, I really hope computational work gains more traction... this might be a marketing issue.
Ditto. I am following a similar path. I am curious if there are any groups dedicated to people like us (econ/consulting/finance people learning data science) to help facilitate the learning process?
It's much easier (relatively speaking) for an economist to pick up some programming than it is for a programmer to pick up some economics. Economists are already familiar with the types of questions that are important to economists, and more importantly, how to frame them. The trouble with economics is that you can't just pick it up overnight as it is a way of thinking more than it is a tool set. Programming on the other hand is something that you can "get working" overnight (economic programmers don't need to be algorithmic theorists -- they just need to be really good at getting/scraping data and organizing it so they can run analyses on it).
Over a year ago, I dropped out of my PhD program in economics because I was not at a school that was going to allow me to do the econ/cs type work I was working on. Leaving my PhD program was one the best things I ever did because it has allowed me to pursue whatever I want to do with the skills I've acquired.
The problem with academic economics is that the data most economists use is so bad and outdated -- such as data from FRED, BLS, and other publicly available sources where everyone and their uncle can download the same CSV dataset that was aggregated by some government employee. The race then is to see who can put together the most elegant econometric model to handle all the issues with the data. The rules of the game change when you create your own dataset and thus have control over while variables to include, the aggregation, the frequency, etc.
Long story short, if you are an economist wanting to do programming, learn to adapt those skills in academia (best way is to find a great advisor -- if there isn't one in your economics department, check the business school as bschool professors are often much more open to highly empirical analyses and care [marginally] less about getting the theory perfect). Or, if you want an easier lifestyle that is much more rewarding, ditch academia for the private sector. You'll find the economists in the private sector to be much more knowledgeable about cutting edge technologies and willing to listen and learn from what you have to say.