Computing the Perfect Model: Why Do Economists Shun Simulation? [pdf](www2.econ.iastate.edu)
www2.econ.iastate.edu
Computing the Perfect Model: Why Do Economists Shun Simulation? [pdf]
http://www2.econ.iastate.edu/tesfatsi/WhyEconomistsShunSimulation.Lehtinen2008.pdf
9 comments
The Santa Fe Institute is awesome. I'm especially a fan of Mark Buchanan (http://physicsoffinance.blogspot.com/). I also found that simulations weren't exciting to the economists in my program. My advisor suggested I move to Paris. He followed his own advice shortly after.
I've kept running into people doing work at SFI (Geoffrey West recently). Though I know little about the place itself. From what comes out of it, though, I'm awfully impressed.
Having been indoctrinated in economics myself (it got better), your experience doesn't much surprise me. The discipline strikes me as largely bankrupt.
Having been indoctrinated in economics myself (it got better), your experience doesn't much surprise me. The discipline strikes me as largely bankrupt.
Haven't had time to read and digest the entire paper yet, but I would like to point out that not all economists shun simulation. There has been quite a bit of work around Sugarscape[1], although I'm not sure what the most recent such work is. But Sugarscape was mentioned quite prominently in Eric Beinhocker's The Origin of Wealth, which paints a picture where economists who associate themselves with "Complexity Economics"[2] are more amenable to simulation.
And, indeed, the description of Complexity Economics from Wikipedia reads:
Complexity economics is the application of complexity science to the problems of economics. It studies computer simulations to gain insight into economic dynamics, and avoids the assumption that the economy is a system in equilibrium.
[1]: http://en.wikipedia.org/wiki/Sugarscape
[2]: http://en.wikipedia.org/wiki/Complexity_economics
And, indeed, the description of Complexity Economics from Wikipedia reads:
Complexity economics is the application of complexity science to the problems of economics. It studies computer simulations to gain insight into economic dynamics, and avoids the assumption that the economy is a system in equilibrium.
[1]: http://en.wikipedia.org/wiki/Sugarscape
[2]: http://en.wikipedia.org/wiki/Complexity_economics
I think 'complexity economics' is well out of the mainstream. I sincerely doubt this type of modeling is taught in economics departments (outside Iowa State). That being said, there are departments that teach "experimental economics" - which sounds like the same thing. You are now just doing "computational" experiments.
I think this kind of stuff has tremendous space to add value, but we should always keep in mind that experimentation by simulation gives us insight into the extrapolation of theories, and only into reality so far as those theories are correct. Of course, done well it can help to validate or disprove theories when we have sufficient ability to measure starting points...
It's interesting you would say that... one of the big arguments of the "complexity economics" folks, as I understand it, is that since economies are not actually equilibrium systems, all of the math imported into economics from physics (and elsewhere) that assumes equilibrium, fails to represent reality.
TBH, I'm not an economist (just a computer science guy who's really interested in economics) but from what I have read, Beinhocker makes a pretty strong case that mainstream economics started going off the rails way back when Walras "borrowed" equilibrium based math from physics and brought it into economics in an attempt to establish economics as a science with the same grounding as the natural sciences.
TBH, I'm not an economist (just a computer science guy who's really interested in economics) but from what I have read, Beinhocker makes a pretty strong case that mainstream economics started going off the rails way back when Walras "borrowed" equilibrium based math from physics and brought it into economics in an attempt to establish economics as a science with the same grounding as the natural sciences.
"Analytical solutions are considered necessary for a model to be accepted as a genuine theoretical contribution."
That's hardly true, the usual macro model is solved numerically using approximate dynamic programming.
That's hardly true, the usual macro model is solved numerically using approximate dynamic programming.
Here we go, they contradict themselves later: "Computational general equilibrium (CGE) models provide an example of accepted computerized problem solving in economics."
I don't see this as a contradiction. The authors probably would include simulation and computerized solvers as "computerized problem solving". Such applied techniques are certainly welcomed in the field of economics. That said, they are not usually considered a theoretical approach. So, using them in a paper won't get you published if a journal is looking for a theoretical advance.
You are talking about the aggregate demand–aggregate supply model, right? https://en.wikipedia.org/wiki/AD-AS_model
Here's my take: the paper would say that solving the AD-AD model with a computational technique with specific choices for parameters would not be considered a theoretical contribution.
I think that is an accurate characterization of economics, based on my experience of what 'theoretical' means. Why? When you use a numerical technique (i.e. not a symbolic manipulation) with particular values for the parameters, few would call that theoretical work. They'll call it applied economics.
(Side note: I can't speak from experience if the AD-AS model is usually solved with dynamic programming.)
Here's my take: the paper would say that solving the AD-AD model with a computational technique with specific choices for parameters would not be considered a theoretical contribution.
I think that is an accurate characterization of economics, based on my experience of what 'theoretical' means. Why? When you use a numerical technique (i.e. not a symbolic manipulation) with particular values for the parameters, few would call that theoretical work. They'll call it applied economics.
(Side note: I can't speak from experience if the AD-AS model is usually solved with dynamic programming.)
There's a distinction between numerical approximation and simulation. The former is a technique to solve an analytical model of the world. The latter would have an analytical model of the simulation outcomes rather than of the world directly.
A problem with simulation in economics is that there is always a feeling that that simulation results are inferior to what could have been done if the writer was better at math. Let's say you derive some insight on heterogeneous agents transacting on a network. This is likely quite hard to tackle without simulation, but most of your peers reading the paper will hope there is some cleaner way to get the insight of the model. The result is that it's unlikely that economists will consider your result a classic, which is ultimately what is required to have a big influence in the discipline.
Since I know a couple of mathematicians (and like to pretend I understand what they work on) I am always a little skeptical of economists' claims to be mathlords. But perhaps my skepticism is misplaced?
How rigorous/sophisticated is the math in research-level econ, really? The econ undergrads I knew went up to basic engineering math (calc 3, linear algebra, diff eq's) at most, and those who did were considered to be really hardcore. Do research economists use higher-level math than that?
How rigorous/sophisticated is the math in research-level econ, really? The econ undergrads I knew went up to basic engineering math (calc 3, linear algebra, diff eq's) at most, and those who did were considered to be really hardcore. Do research economists use higher-level math than that?
It's not necessarily about pushing the boundaries of math. The simpler you can make the math the better for a given level of insight. But I don't think it's "easy" to derive clean mathematical results for many economic models. It's a combination of math, insight, and style.
Imagine this: prove to me some conditions under which equilibrium exist for prices in a market of consumers and producers with heterogenous preferences and skills, that can only communicate on a network with a cost per interaction. Now introduce time and savings and prove the equilibrium interest rate over time. Now prove the optimal tax rate on capital and labor over the agents lifecycle. Now prove that when skill and preferences are not observable to the government. Now introduce default and unobservable savings. Are there conditions under which the markets unravel when faced with exogenous shocks to productivity or beliefs. What is the optimal interest rate policy?
These are classic questions in economics in a more realistic market that captures heterogeneity, limited information, and locality or search costs.
Imagine this: prove to me some conditions under which equilibrium exist for prices in a market of consumers and producers with heterogenous preferences and skills, that can only communicate on a network with a cost per interaction. Now introduce time and savings and prove the equilibrium interest rate over time. Now prove the optimal tax rate on capital and labor over the agents lifecycle. Now prove that when skill and preferences are not observable to the government. Now introduce default and unobservable savings. Are there conditions under which the markets unravel when faced with exogenous shocks to productivity or beliefs. What is the optimal interest rate policy?
These are classic questions in economics in a more realistic market that captures heterogeneity, limited information, and locality or search costs.
A lot of economists who have a phd in in economics, were Engineering, Math and Physics majors as undergrads. To get into a good phd program in Econ, you have to have taken and aced a lot of math classes such as Real Analysis and some very advanced Probability courses. Taking some grad level math classes would probably increase an applicants chance. So yes a lot of economists are really really good at math. And yes a lot of it is used in research.
I know economists are good at math, I'm more curious what kinds of math they tend to use.
It very much depends on the application. Econometrics uses lots of statistics. Finance uses stochastic calculus. Microeconomics uses linear algebra and calculus. To see how incomprehensible things can get, here's a paper by one of my old econometrics professors: http://econweb.ucsd.edu/~bbeare/pdfs/vinecopula.pdf
Financial economics literature is quite heavy on stochastic processes. This requires a sophisticated treatment of probability, which in turn necessitates familiarity with measure theory. Throw some stochastic differential equations into the mix and you're probably also going to need some background in functional analysis. Not so familiar with the rest of the econ literature, but even basic econometrics requires familiarity with point set topology.
Game theory
It is sad that this has the potential to stop progress in the science (not to mention the potential to make it more reputable to people outside the discipline).
Because there are four kinds of lies: lies, damned lies, statistics and computer models.
As a matter of context, this paper was published in Philosophy of Science. This tells you quite a bit about its goals and audience.
The central claim is "Our claim is that economists are willing to accommodate mere computation more readily than simulation mainly because the epistemic status of computational models is considered acceptable while that of simulation models is considered suspect." and "We argue that a major reason why simulation is not granted independent epistemic status is that it is not compatible with the prevailing image of understanding among economists." (pages 306-307)
Somewhat surprisingly, they also write "The claim that economists shun simulation for epistemic and understanding-related reasons is a factual one. Our aim is to explain and evaluate these reasons by considering the philosophical presuppositions of economists." (page 306)
These claims make sense from the tradition of philosophy, where the study of meaning and knowledge is central. But, from the perspective of a practicing economist, you have to question how many are conscious of (or aware of) the biases and preferences of economics when it comes to what counts as valid knowledge, experimentation, and reasoning.
In addition to the explanations inside the paper, it is implicitly or indirectly promoting the importance of the field of philosophy of science. Put this way, if economists were more aware and open to their biases, they would be more likely to "break away from ... methodological constraints" (page 326) and escape the constraints of traditional economic orthodoxies.
In particular, I'd be interested to see a survey of economists to see how many have been exposed to the key concepts from the philosophy of science.
The central claim is "Our claim is that economists are willing to accommodate mere computation more readily than simulation mainly because the epistemic status of computational models is considered acceptable while that of simulation models is considered suspect." and "We argue that a major reason why simulation is not granted independent epistemic status is that it is not compatible with the prevailing image of understanding among economists." (pages 306-307)
Somewhat surprisingly, they also write "The claim that economists shun simulation for epistemic and understanding-related reasons is a factual one. Our aim is to explain and evaluate these reasons by considering the philosophical presuppositions of economists." (page 306)
These claims make sense from the tradition of philosophy, where the study of meaning and knowledge is central. But, from the perspective of a practicing economist, you have to question how many are conscious of (or aware of) the biases and preferences of economics when it comes to what counts as valid knowledge, experimentation, and reasoning.
In addition to the explanations inside the paper, it is implicitly or indirectly promoting the importance of the field of philosophy of science. Put this way, if economists were more aware and open to their biases, they would be more likely to "break away from ... methodological constraints" (page 326) and escape the constraints of traditional economic orthodoxies.
In particular, I'd be interested to see a survey of economists to see how many have been exposed to the key concepts from the philosophy of science.
I had Dr. Tesfatsion (from whose page this paper is hosted) for a Masters-level intro to macroeconomics course. This is a course that all first-semester graduate students in economics take at Iowa State, and Dr. Tesfatsion begins the course with a small examination of the epistemology of economics, and a discussion of where it fits between the "soft" and "hard" sciences.
I was taking this course in 2009, and we spent much of the course discussing the assumptions that economists make in building their models in light of the economic meltdown of the previous years. The course was heavy on traditional Keynsian-derived macro theory and the many historical assumptions that have eventually been shown to be problematic, but it also included an analysis of economics through diverse methods that included behavioral psychology and game theory, many of which emphasised testable small-scale interactions and the way in which they which may lead to larger-scale emergent patterns. (Tesfatsion is very much into agent-based simulation of markets)
I was coming from a religious liberal arts background and an engineering program which was fairly heavy on the philosophy of science, and I appreciated a well-rounded and diverse approach to the analysis of economic systems. I'm not sure, however, what my peers thought of all of this.
I was taking this course in 2009, and we spent much of the course discussing the assumptions that economists make in building their models in light of the economic meltdown of the previous years. The course was heavy on traditional Keynsian-derived macro theory and the many historical assumptions that have eventually been shown to be problematic, but it also included an analysis of economics through diverse methods that included behavioral psychology and game theory, many of which emphasised testable small-scale interactions and the way in which they which may lead to larger-scale emergent patterns. (Tesfatsion is very much into agent-based simulation of markets)
I was coming from a religious liberal arts background and an engineering program which was fairly heavy on the philosophy of science, and I appreciated a well-rounded and diverse approach to the analysis of economic systems. I'm not sure, however, what my peers thought of all of this.
Thanks for sharing your experience!
Philosophy professors talking about something they don't fully understand. Analytic solutions are almost always prefferable when they are available because they are much easier to analyze. Naturally economists will prefer models with analytic solutions. However when simulation is required to understand something economists embrace it as is common in finance, pricing theory and even macroeconomics.
I guess the point is to bash economists. And why is this even on HN? From 2007?
I guess the point is to bash economists. And why is this even on HN? From 2007?
It's on HN because someone had referenced it in a different discussion, it looked interesting and I wanted to see what people had to say about it, so I submitted it. It's on the front page because people voted it up more than they voted other things up.
spikels, The authors most likely see your point (and hear it all the time), but I'm not sure if you see their arguments and perspective.
Yes, philosophy can be less than useful at times. But sometimes it simply doesn't get credit because other disciplines borrow its ideas, simplify them, and use them on a day-to-day basis. In doing so, other fields often create orthodoxies (e.g. practices where certain beliefs are so baked in that you don't think about them). That's fine for getting stuff done, but it can be dangerous when the ideas really aren't resting on bedrock and need reexamination from time to time. The key point of the paper, as I see it, is to talk about why the field of economics is missing the boat on simulation.
Yes, philosophy can be less than useful at times. But sometimes it simply doesn't get credit because other disciplines borrow its ideas, simplify them, and use them on a day-to-day basis. In doing so, other fields often create orthodoxies (e.g. practices where certain beliefs are so baked in that you don't think about them). That's fine for getting stuff done, but it can be dangerous when the ideas really aren't resting on bedrock and need reexamination from time to time. The key point of the paper, as I see it, is to talk about why the field of economics is missing the boat on simulation.
I'm not an Economist, my background is in Physics and CS, so my experience is anecdotal. But I was really surprised by this story, so let me share my thoughts.
Once, I was programming a game and wanted to implement a simple economical simulation in it (how goods are produced and consumed by the population, maybe some price adjustments?.. I did not want anything complex or realistic, very simple linear approximation could be more than enough, I did not want to model agents, I wanted some sort of realistically-looking Production/Consumption/Supply/Demand behavior).
I asked a PhD student to give me an advice. Well, it took quite a lot of time, maybe half an hour, simply to explain the task I wanted to accomplish. Really, the whole idea of simulating things seemed to be foreign to her, and she could not really help me a lot. The economical models seemed to be almost orthogonal to the simulationist perspective. At least, this is how it looked.
It seems that Economics is very much like the laws of conservation in Mechanics. It describes the system from the high-level perspective, but does not really help too much, if you want to simulate individual particles (where you need Newton's laws, for example).
I have the impression that there is, probably, some intellectual predisposition among Economists. Maybe because of the way they are taught.
Once, I was programming a game and wanted to implement a simple economical simulation in it (how goods are produced and consumed by the population, maybe some price adjustments?.. I did not want anything complex or realistic, very simple linear approximation could be more than enough, I did not want to model agents, I wanted some sort of realistically-looking Production/Consumption/Supply/Demand behavior).
I asked a PhD student to give me an advice. Well, it took quite a lot of time, maybe half an hour, simply to explain the task I wanted to accomplish. Really, the whole idea of simulating things seemed to be foreign to her, and she could not really help me a lot. The economical models seemed to be almost orthogonal to the simulationist perspective. At least, this is how it looked.
It seems that Economics is very much like the laws of conservation in Mechanics. It describes the system from the high-level perspective, but does not really help too much, if you want to simulate individual particles (where you need Newton's laws, for example).
I have the impression that there is, probably, some intellectual predisposition among Economists. Maybe because of the way they are taught.
The web link is off Dr. Tesfatsion's website. She has long advocated that the agents in an economy are Turing complete, thus you run into the halting problem in the general case.
At which point it was subtly suggested to me by senior faculty members that -- if I wanted to graduate and become a "real" economist -- I should really focus on proving things, not on simulating things.
(Shortly thereafter I left the program, although not particularly because of this episode.)