Real money, invested with algorithms(blog.quantopian.com)
blog.quantopian.com
Real money, invested with algorithms
http://blog.quantopian.com/real-money/
140 comments
I come from a similar background (automated trading) but have the opposite opinion. More people ought to understand how the markets work. One good way to learn is to trade a small account. Yes you will probably lose a little money (or fail to beat the indexes) but you will also come away with a much better grasp of markets and trading (and hopefully also risk management).
If you look at a typical discussion of trading here on HN you will see lots of totally uninformed hyperbolic assertions about what goes on in the market. How everything is rigged and how corrupt the players are. I think trying to scare people out of the markets only promotes that kind of thinking.
If you look at a typical discussion of trading here on HN you will see lots of totally uninformed hyperbolic assertions about what goes on in the market. How everything is rigged and how corrupt the players are. I think trying to scare people out of the markets only promotes that kind of thinking.
Thank you for saying this!
IMO there's way too much fearmongering (largely from people who have no experience trading, or from traders who don't want to see other people do better than them) about this. I've traded options on a very volatile commodity ETF before (SLV) and came out quiet well on top by playing with patterns and timing. This was in 2010, but there was the same "THE COMPUTERS CONTROL THE MARKETS AND YOU'LL GET CRUSHED" fearmongering hyperbole as there is today. It'll be there four years from now, too.
There's smart money and super fast algorithms and all that for sure, but there's plenty of dumb money moving around too. There are also plenty of people (I will note: often nontechnical individuals) who make a good living day trading. You basically just have to find a pattern ANYWHERE, IN ANYTHING and trade it over and over until it stops working (...and catch on quick). You might lose money (especially starting out), but it's not at all the experience of "you will step in and BE CRUSHED BY EXPERT TRADING ALGORITHMS" that people portray it as.
Start a brokerage account, do some reading, put in a sum of money that's consequential but that won't break you if you 100% lose it, and play around a little bit. Try focusing on one specific (preferably fairly liquid) security. You'll be amazed at how sometimes you make the right call and magically you have more money than you started with, and how you don't immediately die and get shot and lose all of your money to some corrupt evil high-speed algorithm while being eaten by wolves because, you shameful Icarus fool, you dared to trade on the public markets by yourself.
Risk-averse people (especially programmers) who have no experience trading will think you're completely insane and lambaste you, especially if you make money.
IMO there's way too much fearmongering (largely from people who have no experience trading, or from traders who don't want to see other people do better than them) about this. I've traded options on a very volatile commodity ETF before (SLV) and came out quiet well on top by playing with patterns and timing. This was in 2010, but there was the same "THE COMPUTERS CONTROL THE MARKETS AND YOU'LL GET CRUSHED" fearmongering hyperbole as there is today. It'll be there four years from now, too.
There's smart money and super fast algorithms and all that for sure, but there's plenty of dumb money moving around too. There are also plenty of people (I will note: often nontechnical individuals) who make a good living day trading. You basically just have to find a pattern ANYWHERE, IN ANYTHING and trade it over and over until it stops working (...and catch on quick). You might lose money (especially starting out), but it's not at all the experience of "you will step in and BE CRUSHED BY EXPERT TRADING ALGORITHMS" that people portray it as.
Start a brokerage account, do some reading, put in a sum of money that's consequential but that won't break you if you 100% lose it, and play around a little bit. Try focusing on one specific (preferably fairly liquid) security. You'll be amazed at how sometimes you make the right call and magically you have more money than you started with, and how you don't immediately die and get shot and lose all of your money to some corrupt evil high-speed algorithm while being eaten by wolves because, you shameful Icarus fool, you dared to trade on the public markets by yourself.
Risk-averse people (especially programmers) who have no experience trading will think you're completely insane and lambaste you, especially if you make money.
No offense but it sounds like you just got lucky and quit while you were ahead
In fairness, quitting while one is ahead is one of the rare yet essential skills. Some people just have it. Alas, I'm not one of them...
> If you look at a typical discussion of trading here on HN you will see lots of totally uninformed hyperbolic assertions about what goes on in the market. How everything is rigged and how corrupt the players are.
Yes, one does see such remarks, but an excellent counterargument is almost never heard-- and that is that (a) business leaders aren't stupid and (b) if the equities market was really significantly rigged or inefficient, these people would refuse to raise capital using equities.
The fact that some of the most skeptical, cautious people in existence, people with plenty of practical experience, are willing to fund their businesses with equity capital, means that equity trading is substantially fair and that the efficient market hypothesis is at least approximately true.
Yes, one does see such remarks, but an excellent counterargument is almost never heard-- and that is that (a) business leaders aren't stupid and (b) if the equities market was really significantly rigged or inefficient, these people would refuse to raise capital using equities.
The fact that some of the most skeptical, cautious people in existence, people with plenty of practical experience, are willing to fund their businesses with equity capital, means that equity trading is substantially fair and that the efficient market hypothesis is at least approximately true.
> The fact that some of the most skeptical, cautious people in existence, people with plenty of practical experience, are willing to fund their businesses with equity capital, means that equity trading is substantially fair and that the efficient market hypothesis is at least approximately true.
That's a pretty bold claim. Given the financial crash of 08, from what I can tell the EMH has been thoroughly debunked. Markets are not efficient and all data is not priced in. Information is not absorbed quickly into the market so inefficiencies crop up everywhere all the time, those are the profit opportunities that traders make a living off of. If EMH were correct, the systematic risk that crashed the market could not have occurred. Remember, it's a hypothesis, not a theory.
That's a pretty bold claim. Given the financial crash of 08, from what I can tell the EMH has been thoroughly debunked. Markets are not efficient and all data is not priced in. Information is not absorbed quickly into the market so inefficiencies crop up everywhere all the time, those are the profit opportunities that traders make a living off of. If EMH were correct, the systematic risk that crashed the market could not have occurred. Remember, it's a hypothesis, not a theory.
> That's a pretty bold claim. Given the financial crash of 08, from what I can tell the EMH has been thoroughly debunked.
Not at all. The EMH isn't falsified by people engaging in widespread cheating, and it isn't falsified by big market reversals driven by public psychology. It could only be falsified by the market's inability to accurately set a price on average, across all equities, perpetually.
Does an airline disaster contradict the claim that air travel is safe? No, that can only be contradicted by average flight outcomes. It's the same with the equities market.
> Markets are not efficient and all data is not priced in.
This claim is obviously contradicted by the fact that people are willing to use equities to raise business capital. If the market wasn't efficient, they would think of another way to raise capital -- something more efficient. That's hardly controversial.
> If EMH were correct, the systematic risk that crashed the market could not have occurred.
The EMH isn't falsified by cheating. Does the fact that insider trading takes place contradict a hypothesis that a market without cheating is fair and efficient?
> Remember, it's a hypothesis, not a theory.
Yes, and it can never be a theory in the scientific sense -- there's no way to gather objective data in a controlled way. It will probably remain a hypothesis in a pseudoscientific twilight zone forever. But given all the alternatives, the fact that people invest in the equities market argues for the truth of the EMH -- in an unscientific and dubious way.
Not at all. The EMH isn't falsified by people engaging in widespread cheating, and it isn't falsified by big market reversals driven by public psychology. It could only be falsified by the market's inability to accurately set a price on average, across all equities, perpetually.
Does an airline disaster contradict the claim that air travel is safe? No, that can only be contradicted by average flight outcomes. It's the same with the equities market.
> Markets are not efficient and all data is not priced in.
This claim is obviously contradicted by the fact that people are willing to use equities to raise business capital. If the market wasn't efficient, they would think of another way to raise capital -- something more efficient. That's hardly controversial.
> If EMH were correct, the systematic risk that crashed the market could not have occurred.
The EMH isn't falsified by cheating. Does the fact that insider trading takes place contradict a hypothesis that a market without cheating is fair and efficient?
> Remember, it's a hypothesis, not a theory.
Yes, and it can never be a theory in the scientific sense -- there's no way to gather objective data in a controlled way. It will probably remain a hypothesis in a pseudoscientific twilight zone forever. But given all the alternatives, the fact that people invest in the equities market argues for the truth of the EMH -- in an unscientific and dubious way.
Efficient market hypothesis discussions are pointless imo, because it's obviously untrue, and obviously approximately true. It's trivial to create cases for both but there's not a credible test to how good the approximation is, what are it's failure modes and critical points, where the inefficiencies are amplified, and so on.
In the end it's just an assumption in economic papers so they can be correct in some sense.
In the end it's just an assumption in economic papers so they can be correct in some sense.
> Efficient market hypothesis discussions are pointless imo, because it's obviously untrue ...
So cash in. Since you think the EMH is "obviously untrue", you can drain the market of its capital based on your proven theory and your inside track on the truth.
In fact, plenty of evidence suggests that the EMH is true, but that the present equities market can't reliably demonstrate this fact because of widespread cheating.
I should add that, because of the nature of equities trading and markets in general, it's very doubtful that anyone will ever prove this issue one way or another in a scientific sense.
So cash in. Since you think the EMH is "obviously untrue", you can drain the market of its capital based on your proven theory and your inside track on the truth.
In fact, plenty of evidence suggests that the EMH is true, but that the present equities market can't reliably demonstrate this fact because of widespread cheating.
I should add that, because of the nature of equities trading and markets in general, it's very doubtful that anyone will ever prove this issue one way or another in a scientific sense.
you can drain the market of its capital based on your proven theory and your inside track on the truth
irrational != predictable ∴ markets cannot be accurately modelled
the EMH is true, but that the present equities market can't reliably demonstrate this fact because of widespread cheating
real markets != efficient and real markets != fair ∴ EMH is false for real markets
irrational != predictable ∴ markets cannot be accurately modelled
the EMH is true, but that the present equities market can't reliably demonstrate this fact because of widespread cheating
real markets != efficient and real markets != fair ∴ EMH is false for real markets
Side remark: Please don't use equality and inequality like that. How can a noun and an adjective possibly be equal? They aren't even the same word class. Just use english "is" and "is not" to connect your nouns and adjectives。
[deleted]
true? (is == equality)
don't think so.
don't think so.
The EMH isn't falsified by people engaging in widespread cheating, and it isn't falsified by big market reversals driven by public psychology. It could only be falsified by the market's inability to accurately set a price on average, across all equities, perpetually.
Apparently it can't be falsified or proved before the heat death of the universe, so not a very interesting or credible hypothesis.
This claim is obviously contradicted by the fact that people are willing to use equities to raise business capital.
jjarmusch, who is inexplicably hellbanned below as I post this, makes the point that people willing to use equities to raise money do not depend on an efficient market, in fact they profit from a broken or delusional market. So no, people using a market does not prove it is maximally efficient or even close.
The EMH isn't falsified by cheating.
No, it's falsified by comparisons with real markets which it attempts to model (which include cheating, stupidity and greed as well as occasional rational valuations).
Apparently it can't be falsified or proved before the heat death of the universe, so not a very interesting or credible hypothesis.
This claim is obviously contradicted by the fact that people are willing to use equities to raise business capital.
jjarmusch, who is inexplicably hellbanned below as I post this, makes the point that people willing to use equities to raise money do not depend on an efficient market, in fact they profit from a broken or delusional market. So no, people using a market does not prove it is maximally efficient or even close.
The EMH isn't falsified by cheating.
No, it's falsified by comparisons with real markets which it attempts to model (which include cheating, stupidity and greed as well as occasional rational valuations).
> Apparently it can't be falsified or proved before the heat death of the universe, so not a very interesting or credible hypothesis.
If that were true, if unprovable hypotheses had no practical value, psychology would collapse. Wait ... hold on ... nope, psychology isn't collapsing.
> So no, people using a market does not prove it is maximally efficient or even close.
You're missing the point that money flows to the most efficient of alternative capital raising methods -- which, if you think about it, also stands as evidence for the EMH. Given the freedom of businesses to choose any method to raise operating capital, and given that they prefer equities, this shows that the equities market is more efficient than existing alternatives.
If the equities market were less efficient than brand X, businesses would raise capital using brand X. How is that difficult to understand?
> No, it's falsified by comparisons with real markets which it attempts to model ...
You are apparently unaware that the EMH isn't compared to real markets, it's the other way around. And if businesses believed that equities were inefficient compared to anything else, they would change methods.
If that were true, if unprovable hypotheses had no practical value, psychology would collapse. Wait ... hold on ... nope, psychology isn't collapsing.
> So no, people using a market does not prove it is maximally efficient or even close.
You're missing the point that money flows to the most efficient of alternative capital raising methods -- which, if you think about it, also stands as evidence for the EMH. Given the freedom of businesses to choose any method to raise operating capital, and given that they prefer equities, this shows that the equities market is more efficient than existing alternatives.
If the equities market were less efficient than brand X, businesses would raise capital using brand X. How is that difficult to understand?
> No, it's falsified by comparisons with real markets which it attempts to model ...
You are apparently unaware that the EMH isn't compared to real markets, it's the other way around. And if businesses believed that equities were inefficient compared to anything else, they would change methods.
Your faith in EMH is looking religious. You're making assertion after assertion without evidence or sound reason. Because businesses choose to raise capital in equity markets does not show that EMH is true. It just shows that the markets are more efficient than the alternatives, it does not mean the market is efficient.
You are apparently unaware that the EMH isn't compared to real markets, it's the other way around.
So in the best of all possible worlds, the EMH is true, but we don't live there.
So in the best of all possible worlds, the EMH is true, but we don't live there.
The case that subprime mortgages were in a bubble was made very efficiently in public by several people - I was subscribed to Nouriel Roubini's widely followed blog at the time; Gillian Tett at the FT also did good work exposing what was going on with property prices - and believed by many people. The bubble continued for years after it had convincingly been called.
However, relatively few people figured out how to efficiently make use of this information. Shorting the stocks of banks like AIG is very high risk, and can cost you everything if you don't know when the bubble is going to burst. In the end, the only shorting strategy that was effective was to use CDS, a newly popular derivative (which was a very smart idea). Jon Paulson's hedge fund made $15 billion from this, but Kyle Bass, another person who made money from this, says only 15 people figured out the CDS trade and made money from it. The amount of shorting that happened was tiny in proportion to the size of the bubble.
The EMH is a hypothesis that was contrived in an analysis that does not take account the actual way that information gets incorporated into prices, the actual trades available to market participants, or the asymmetry between upside and downside risks (aka. the Keynesian risks of opposing the market's animal spirits).
The EMH states that market prices correctly price in all public information. The weaker proposition, that it is hard to get rich quick from spotting market mispricing using just public information, does seem to be true.
However, relatively few people figured out how to efficiently make use of this information. Shorting the stocks of banks like AIG is very high risk, and can cost you everything if you don't know when the bubble is going to burst. In the end, the only shorting strategy that was effective was to use CDS, a newly popular derivative (which was a very smart idea). Jon Paulson's hedge fund made $15 billion from this, but Kyle Bass, another person who made money from this, says only 15 people figured out the CDS trade and made money from it. The amount of shorting that happened was tiny in proportion to the size of the bubble.
The EMH is a hypothesis that was contrived in an analysis that does not take account the actual way that information gets incorporated into prices, the actual trades available to market participants, or the asymmetry between upside and downside risks (aka. the Keynesian risks of opposing the market's animal spirits).
The EMH states that market prices correctly price in all public information. The weaker proposition, that it is hard to get rich quick from spotting market mispricing using just public information, does seem to be true.
> Markets are not efficient and all data is not priced in
EMH says that all publicly available data is priced in. The strongest form claims all data, but no one believes that (insider trading alone debunks it).
I would postulate that given the data available right before the crash, you would probably end up with similar results to what the pricing was at the time.
Note that one thing people have a hard time understanding with EMH is externalities, such as enthusiasm or whatever the opposite is called for a product or class of product.
> those are the profit opportunities that traders make a living off of
EMH relies on people making a living to function, the price setters need to do well or else enough won't be attracted to make the market efficient. You are correct that this kind of problem makes anything but the weakest form difficult to prove.
> the systematic risk that crashed the market could not have occurred
Why? One major factor in the crash was that people started walking away from their homes. Historically this didn't happen. If all of the data points to stability in mortgages, you cannot expect people to divine that problem, nor does the EMH require them too.
EMH says that all publicly available data is priced in. The strongest form claims all data, but no one believes that (insider trading alone debunks it).
I would postulate that given the data available right before the crash, you would probably end up with similar results to what the pricing was at the time.
Note that one thing people have a hard time understanding with EMH is externalities, such as enthusiasm or whatever the opposite is called for a product or class of product.
> those are the profit opportunities that traders make a living off of
EMH relies on people making a living to function, the price setters need to do well or else enough won't be attracted to make the market efficient. You are correct that this kind of problem makes anything but the weakest form difficult to prove.
> the systematic risk that crashed the market could not have occurred
Why? One major factor in the crash was that people started walking away from their homes. Historically this didn't happen. If all of the data points to stability in mortgages, you cannot expect people to divine that problem, nor does the EMH require them too.
> The strongest form claims all data, but no one believes that (insider trading alone debunks it).
Insider trading doesn't "debunk" the EMH, because the EMH is predicated on fair dealing. There's a lot of distorted ideas about the EMH, for example that a market crash, or cheating, disproves it. These events don't disprove the hypothesis.
Insider trading doesn't "debunk" the EMH, because the EMH is predicated on fair dealing. There's a lot of distorted ideas about the EMH, for example that a market crash, or cheating, disproves it. These events don't disprove the hypothesis.
> These events don't disprove the hypothesis.
Then the hypothesis is useless and false. You can't claim markets are efficient and then exclude all evidence to the contrary.
> because the EMH is predicated on fair dealing
Yes, which makes EMH false in the real world. A hypothesis that's only true in an ideal world isn't useful nor is it meaningful in the real world.
Then the hypothesis is useless and false. You can't claim markets are efficient and then exclude all evidence to the contrary.
> because the EMH is predicated on fair dealing
Yes, which makes EMH false in the real world. A hypothesis that's only true in an ideal world isn't useful nor is it meaningful in the real world.
> A hypothesis that's only true in an ideal world isn't useful nor is it meaningful in the real world.
You are not treating EMH like you should, think of it as a guideline not a hard and always true rule of life.
For instance lets say Telsa is trading for $100 and you think it is worth $50, there are two reasons for this, either:
* The market is not efficient and you should take advantage. * The market is efficient and you aren't considering all of the variables.
In almost every conceivable case it is the latter. The answer for Tesla is probably optimism for the technology when optimism often drives prices above the otherwise expected value.
EMH makes you consider the market as the baseline, which greatly helps steer you towards what is going on.
You are not treating EMH like you should, think of it as a guideline not a hard and always true rule of life.
For instance lets say Telsa is trading for $100 and you think it is worth $50, there are two reasons for this, either:
* The market is not efficient and you should take advantage. * The market is efficient and you aren't considering all of the variables.
In almost every conceivable case it is the latter. The answer for Tesla is probably optimism for the technology when optimism often drives prices above the otherwise expected value.
EMH makes you consider the market as the baseline, which greatly helps steer you towards what is going on.
Your understanding of the claims of EMH are flawed. On the other hand, the true claim is not easily tested. The best test exists in the form of looking at how easy it is to turn a consistent profit from transacting in open and liquid financial markets. Answer: not easy
Not easy isn't equal to not possible. Math isn't easy for most people, that doesn't disprove math. If the claims of EHM cannot be tested, then EMH is obviously nonsense.
Agreed. In fact markets can from time to time exhibit systemic inefficiencies. What this means is that the very fabric of the market itself can give rise to bubbles or crashes. A brilliant book that I found an eye-opening read is by Didier Sornette, a physicist by training, "Why stock markets crash," circa 2003.
"(a) business leaders aren't stupid and (b) if the equities market was really significantly rigged or inefficient, these people would refuse to raise capital using equities."
Wouldn't that depend on who the market is stacked for/against, by how much, and what their alternatives are?
Wouldn't that depend on who the market is stacked for/against, by how much, and what their alternatives are?
But the most basic premise of a fair market is that it's not "stacked" at all -- that everyone has an equal chance to react to market moves and price changes.
For example, a business cannot possibly get one cent more than the purchasers of their stock offer in the marketplace, and because of transaction fees, somewhat less. And an investor has the same experience -- he can't possibly get more than the market value of his shares on the day of sale.
Brokerage houses are in competition to offer the lowest transaction costs and the most reliable environment for trading. That competition reduces transaction prices for everyone.
Conclusion? Not stacked in general. Insider trading is a crime, betting against your own customers is a crime, and these things happen, but they're prosecuted.
For example, a business cannot possibly get one cent more than the purchasers of their stock offer in the marketplace, and because of transaction fees, somewhat less. And an investor has the same experience -- he can't possibly get more than the market value of his shares on the day of sale.
Brokerage houses are in competition to offer the lowest transaction costs and the most reliable environment for trading. That competition reduces transaction prices for everyone.
Conclusion? Not stacked in general. Insider trading is a crime, betting against your own customers is a crime, and these things happen, but they're prosecuted.
>that everyone has an equal chance to react to market moves and price changes.
A rich person or big trading house can buy rack space close to New York or whatever city their market is based on, a regular person cannot. This ability gives a significant advantage to rich people/business owners, therefore the market is not fair at all.
A rich person or big trading house can buy rack space close to New York or whatever city their market is based on, a regular person cannot. This ability gives a significant advantage to rich people/business owners, therefore the market is not fair at all.
I'm not sure just how much I can say, in the context of my employment, so I'm going to drop out here.
No, whether people finance with equity capital has no bearing on whether markets are fair or efficient.
People finance with VC money when there is not even usually a market.
People who want capital have little choice, they have to pay the ridiculous fees and have it traded in the existing market. But they don;t really care about the non competitive cartel of investment banks anyway, as running a public company means you can pay your self almost unlimited amounts of the shareholders money anyway. Whats not to like? Who cares if the market is efficient?
People finance with VC money when there is not even usually a market.
People who want capital have little choice, they have to pay the ridiculous fees and have it traded in the existing market. But they don;t really care about the non competitive cartel of investment banks anyway, as running a public company means you can pay your self almost unlimited amounts of the shareholders money anyway. Whats not to like? Who cares if the market is efficient?
> You really have very little chance of success unless you're doing this full time
Heh. I also wanted to play with algorithmic trading, but before doing so, I got a book "Trading and exchanges" by Harris, which was recommended by somebody on reddit.
I have read it only halfways and realized two things:
1. I'd have to do trading (and related activities) as a full-time job in order to be successful (i.e., earn money in the long run).
2. I'd need to spend at least a year's equivalent of a full-time job on learning the necessary background math and algorithms.
I.e., I'd need to quit my job (which I quite like), only to replace it with another set of daily duties.
Heh. I also wanted to play with algorithmic trading, but before doing so, I got a book "Trading and exchanges" by Harris, which was recommended by somebody on reddit.
I have read it only halfways and realized two things:
1. I'd have to do trading (and related activities) as a full-time job in order to be successful (i.e., earn money in the long run).
2. I'd need to spend at least a year's equivalent of a full-time job on learning the necessary background math and algorithms.
I.e., I'd need to quit my job (which I quite like), only to replace it with another set of daily duties.
This is true. I think that people should have the capability of being successful, but only if they do approach it like a full-time job. That doesn't, however, guarantee success, even a large operation like the last one I worked for can fail.
If it were easy, and everyone could do it and make as much money as working full-time, without actually working full-time; then everyone would.
If it were easy, and everyone could do it and make as much money as working full-time, without actually working full-time; then everyone would.
Everything you said could practically be applied to any skilled profession. Being an X is hard, 99% of people fail at X. What makes trading any harder than programming or engineering or abstract mathematics or physics? Successful traders exist, plenty of people trade for a living. People have to try something to see if it's something they're good at and most people aren't good at most things. Pointing that out X is hard and most people fail is a tautology, it's true of all difficult professions.
> What makes trading any harder than programming or engineering or abstract mathematics or physics?
That's easy to answer -- if you're very smart, you will do better in math and physics, and the positive correlation between intelligence and success actually has meaning. But in equities trading, being very smart does no good at all, and might even represent a handicap. The reason? Equity pricing is mostly random noise, very high entropy.
Many very smart people have tried to model and then beat the market, and (apart from chance outcomes) no one has succeeded in the sense that they located a reliable, describable "method" for it.
Also, if in principle someone discovered an actual winning system, by publishing the system he would destroy its effectiveness (because once a "system" is put into practice, once everyone uses it, it loses its effectiveness).
Obviously for a random collection of investment schemes, half will do better, and half worse, than the average market, but the half that do better will naturally enough claim the outcome resulted from their investment genius -- even if they're smart enough that they should know better.
I can use a computer to model a random market and random investors, using random price changes and random trades, and not surprisingly, half the investors will do better than the average outcome. What is surprising is that, when the above-average investors are humans, they invariably try to claim this chance outcome resulted from a winning system that they will sell to you for some princely sum.
More here: http://arachnoid.com/equities_myths
> Pointing that out X is hard and most people fail is a tautology, it's true of all difficult professions.
Yes, but there's a qualitative difference between "hard" and "impossible".
That's easy to answer -- if you're very smart, you will do better in math and physics, and the positive correlation between intelligence and success actually has meaning. But in equities trading, being very smart does no good at all, and might even represent a handicap. The reason? Equity pricing is mostly random noise, very high entropy.
Many very smart people have tried to model and then beat the market, and (apart from chance outcomes) no one has succeeded in the sense that they located a reliable, describable "method" for it.
Also, if in principle someone discovered an actual winning system, by publishing the system he would destroy its effectiveness (because once a "system" is put into practice, once everyone uses it, it loses its effectiveness).
Obviously for a random collection of investment schemes, half will do better, and half worse, than the average market, but the half that do better will naturally enough claim the outcome resulted from their investment genius -- even if they're smart enough that they should know better.
I can use a computer to model a random market and random investors, using random price changes and random trades, and not surprisingly, half the investors will do better than the average outcome. What is surprising is that, when the above-average investors are humans, they invariably try to claim this chance outcome resulted from a winning system that they will sell to you for some princely sum.
More here: http://arachnoid.com/equities_myths
> Pointing that out X is hard and most people fail is a tautology, it's true of all difficult professions.
Yes, but there's a qualitative difference between "hard" and "impossible".
You're basically saying investing is random chance. Ok.
What is the p-value that Warren Buffett's random investment plan outperforms the market 39/47 years?
http://finance.fortune.cnn.com/2012/02/25/buffett-berkshire-...
If a drug beat a placebo in 39/47 experiments, would it be approved by the FDA? :-)
There are investors who get lucky, and there are investors with sound strategies. The notion that every winning strategy gets immediately and flawlessly applied by every actor in the market is a fantasy world populated by homo economicus.
What is the p-value that Warren Buffett's random investment plan outperforms the market 39/47 years?
http://finance.fortune.cnn.com/2012/02/25/buffett-berkshire-...
If a drug beat a placebo in 39/47 experiments, would it be approved by the FDA? :-)
There are investors who get lucky, and there are investors with sound strategies. The notion that every winning strategy gets immediately and flawlessly applied by every actor in the market is a fantasy world populated by homo economicus.
> You're basically saying investing is random chance.
No, what I am saying is that random choices can produce the illusion of successful investment strategies for 1/2 the practitioners. This isn't at all controversial -- it's equivalent to saying that half of people are above average in intelligence.
> What is the p-value that Warren Buffett's random investment plan outperforms the market 39/47 years?
That's not a particularly interesting question (and it's not an improbable unbroken streak of 39 years, but a mixture of successes and failures, much more likely to result from chance). What is more interesting is to ask what Buffett's market performance would have been without the announcement and herd effects? Everyone wants to invest in the same stocks Buffett invests in, and Buffet's investments are public knowledge. The herd can be relied on to make Buffett's choices after he has established his position, which means he's positioned to benefit from the public's responses to his choices.
Again, I never claimed that investing is random choice. I only said that random choices work for 1/2 the investors, and those investors are likely to attribute their success to genius instead of chance.
No, what I am saying is that random choices can produce the illusion of successful investment strategies for 1/2 the practitioners. This isn't at all controversial -- it's equivalent to saying that half of people are above average in intelligence.
> What is the p-value that Warren Buffett's random investment plan outperforms the market 39/47 years?
That's not a particularly interesting question (and it's not an improbable unbroken streak of 39 years, but a mixture of successes and failures, much more likely to result from chance). What is more interesting is to ask what Buffett's market performance would have been without the announcement and herd effects? Everyone wants to invest in the same stocks Buffett invests in, and Buffet's investments are public knowledge. The herd can be relied on to make Buffett's choices after he has established his position, which means he's positioned to benefit from the public's responses to his choices.
Again, I never claimed that investing is random choice. I only said that random choices work for 1/2 the investors, and those investors are likely to attribute their success to genius instead of chance.
Perhaps more to the point: Buffet's results are unlikely (though likely the result of many different strategies, some successful, others not...), but in a market with so many investors, it is perhaps not so surprising that one so successful arose?
> Buffet's results are unlikely (though likely the result of many different strategies, some successful, others not...), but in a market with so many investors, it is perhaps not so surprising that one so successful arose?
Quite correct, in fact, in an equities trading community the size of the present one, successes like Buffet's can and do result from chance. This is not to claim that any particular one does result from chance, only that this possibility is the first one that a scientist or statistician would need to consider.
Quite correct, in fact, in an equities trading community the size of the present one, successes like Buffet's can and do result from chance. This is not to claim that any particular one does result from chance, only that this possibility is the first one that a scientist or statistician would need to consider.
> Buffet's results are unlikely
That wasn't Buffets point; his point is that those who followed certain strategies of value investing consistently win and he gave examples. That's more than a random outcome.
That wasn't Buffets point; his point is that those who followed certain strategies of value investing consistently win and he gave examples. That's more than a random outcome.
> That wasn't Buffets point; his point is that those who followed certain strategies of value investing consistently win and he gave examples.
"Consistently win" is obviously false. If someone had an actual method (not a random unexplained event) with a description, that could be tested and that could consistently beat market averages, it would surely be applied and the market would collapse. The market didn't collapse, so there is no such method.
How is that so hard to figure out? There is no winning strategy, no secrets of the winners. There are people who make more than others in equities, but not because of a describable, scientifically testable system.
When a scientist encounters a description like this, he always assumes a priori that it's chance. That agrees with the null hypothesis and Occam's razor. It also keeps people from selling him worthless investment books.
"Consistently win" is obviously false. If someone had an actual method (not a random unexplained event) with a description, that could be tested and that could consistently beat market averages, it would surely be applied and the market would collapse. The market didn't collapse, so there is no such method.
How is that so hard to figure out? There is no winning strategy, no secrets of the winners. There are people who make more than others in equities, but not because of a describable, scientifically testable system.
When a scientist encounters a description like this, he always assumes a priori that it's chance. That agrees with the null hypothesis and Occam's razor. It also keeps people from selling him worthless investment books.
You're restricting yourself with the giant caveat that the system must be perfectly deterministic (i.e. capable of being executed by code). Why? For the convenience of your testing?
An analogy: humans have beaten computers at Go for decades. Is the human strategy random chance? It must be, since they cannot write down their algorithm so well that anyone else can be a Go master!
Go strategy (similar to investment strategy) cannot be perfectly laid down (though the general principles can), and yet the results are clear. Would you argue there is nobody inherently better at Go than anyone else? Even if they are the best performing Go-perfomer ever, who has beaten every computer system for decades and decades?
I'm not sure how much of a butt-kicking Buffett needs to perform against the market for you to believe there may be an element of skill at play.
An analogy: humans have beaten computers at Go for decades. Is the human strategy random chance? It must be, since they cannot write down their algorithm so well that anyone else can be a Go master!
Go strategy (similar to investment strategy) cannot be perfectly laid down (though the general principles can), and yet the results are clear. Would you argue there is nobody inherently better at Go than anyone else? Even if they are the best performing Go-perfomer ever, who has beaten every computer system for decades and decades?
I'm not sure how much of a butt-kicking Buffett needs to perform against the market for you to believe there may be an element of skill at play.
> it would surely be applied and the market would collapse. The market didn't collapse, so there is no such method.
You keep saying that, but it's full of assumptions that simply aren't true. Just because a successful method exists does not mean everyone can or will apply it nor does it mean the market will collapse.
> How is that so hard to figure out?
How it it so hard for you to see the absurdity of what you're saying?
> When a scientist encounters a description like this, he always assumes a priori that it's chance. That agrees with the null hypothesis and Occam's razor.
There's a reason scientists are in general not rich people. And I have much respect for scientists, but you're just being absurd.
Investigate a bit before just claiming it's nonsense because you're not giving it the thought you should, you're just presuming you're right and making absurd claims based on absurd assumptions that don't hold water.
http://www.businessinsider.com/warren-buffett-on-efficient-m...
You keep saying that, but it's full of assumptions that simply aren't true. Just because a successful method exists does not mean everyone can or will apply it nor does it mean the market will collapse.
> How is that so hard to figure out?
How it it so hard for you to see the absurdity of what you're saying?
> When a scientist encounters a description like this, he always assumes a priori that it's chance. That agrees with the null hypothesis and Occam's razor.
There's a reason scientists are in general not rich people. And I have much respect for scientists, but you're just being absurd.
Investigate a bit before just claiming it's nonsense because you're not giving it the thought you should, you're just presuming you're right and making absurd claims based on absurd assumptions that don't hold water.
http://www.businessinsider.com/warren-buffett-on-efficient-m...
> That's not a particularly interesting question (and it's not an improbable unbroken streak of 39 years, but a mixture of successes and failures, much more likely to result from chance).
Why is any particular permutation of 39 wins and 8 losses more improbable than another? That's like saying HHHTT is more likely than HTHTH.
> The herd can be relied on to make Buffett's choices after he has established his position, which means he's positioned to benefit from the public's responses to his choices.
But now you're playing it both ways. Are markets efficient or not? Can't every rational investor realize the herd will do this and just ape Buffett?
I may have misunderstood, but your thesis seems to be that markets are efficient, and no strategy can consistently beat the market over time. My thesis is that we have an existence proof that isn't the case.
Why is any particular permutation of 39 wins and 8 losses more improbable than another? That's like saying HHHTT is more likely than HTHTH.
> The herd can be relied on to make Buffett's choices after he has established his position, which means he's positioned to benefit from the public's responses to his choices.
But now you're playing it both ways. Are markets efficient or not? Can't every rational investor realize the herd will do this and just ape Buffett?
I may have misunderstood, but your thesis seems to be that markets are efficient, and no strategy can consistently beat the market over time. My thesis is that we have an existence proof that isn't the case.
>> That's not a particularly interesting question (and it's not an improbable unbroken streak of 39 years, but a mixture of successes and failures, much more likely to result from chance).
> Why is any particular permutation of 39 wins and 8 losses more improbable than another? That's like saying HHHTT is more likely than HTHTH.
Again, that is not what I said. Are you trying to misinterpret, or is this all inadvertent? Had I anticipated your effort to misinterpret, I would have said that a random sequence of 39 heads is less probable than three sequences of 13 heads, each separated by some other unidentified, random sequences or, as I put it originally, "a mixture of successes and failures".
>> The herd can be relied on to make Buffett's choices after he has established his position, which means he's positioned to benefit from the public's responses to his choices.
> But now you're playing it both ways.
Nonsense. I'm explaining how any outcome can be attributed to chance, not that they are explained by chance.
> Are markets efficient or not?
That's not even a topic, and if it were, it wouldn't be a binary choice.
> I may have misunderstood,
You thoroughly misunderstood, no ambiguity. And no offense meant.
> ... but your thesis seems to be that markets are efficient, and no strategy can consistently beat the market over time.
No to the first (no one knows) but absolutely yes to the second -- no strategy can consistently beat the market over time. Isn't that obvious? Any strategy that consistently beat the market, and that could be expressed as a deterministic algorithm, and that was something other than a transient effect of no real value, could be used to drain the market of its capital in a matter of months -- and therefore it would be. The market would collapse and businesses would refuse to trust equities.
My point? If there really was such a strategy, it would demolish the market, meaning there would be no remaining market to beat consistently. The golden goose would lie dead.
Just think a bit more deeply.
> Why is any particular permutation of 39 wins and 8 losses more improbable than another? That's like saying HHHTT is more likely than HTHTH.
Again, that is not what I said. Are you trying to misinterpret, or is this all inadvertent? Had I anticipated your effort to misinterpret, I would have said that a random sequence of 39 heads is less probable than three sequences of 13 heads, each separated by some other unidentified, random sequences or, as I put it originally, "a mixture of successes and failures".
>> The herd can be relied on to make Buffett's choices after he has established his position, which means he's positioned to benefit from the public's responses to his choices.
> But now you're playing it both ways.
Nonsense. I'm explaining how any outcome can be attributed to chance, not that they are explained by chance.
> Are markets efficient or not?
That's not even a topic, and if it were, it wouldn't be a binary choice.
> I may have misunderstood,
You thoroughly misunderstood, no ambiguity. And no offense meant.
> ... but your thesis seems to be that markets are efficient, and no strategy can consistently beat the market over time.
No to the first (no one knows) but absolutely yes to the second -- no strategy can consistently beat the market over time. Isn't that obvious? Any strategy that consistently beat the market, and that could be expressed as a deterministic algorithm, and that was something other than a transient effect of no real value, could be used to drain the market of its capital in a matter of months -- and therefore it would be. The market would collapse and businesses would refuse to trust equities.
My point? If there really was such a strategy, it would demolish the market, meaning there would be no remaining market to beat consistently. The golden goose would lie dead.
Just think a bit more deeply.
I appreciate the discussion, let me clarify.
> a random sequence of 39 heads is less probable than three sequences of 13 heads, each separated by some other unidentified, random sequences
Investors are not going for streaks. They are going for overall return. A better way to put it: what are the chances that over a 48-year timeframe, a random strategy will return ~20% annualized return compared to the market's ~10%? Ignore an arbitrary year limitation (why not measure monthly or daily return?). What is the chance a random strategy would outperform so long?
Or, a better question: what is a chance that a random strategy will be the best-performing investment over a 30-year period (1926-2011), and the subject of academic studies:
http://www.econ.yale.edu/~af227/pdf/Buffett's%20Alpha%20-%20...
Buffett is an existence proof that investment is skill, not chance. He created the 9th most valuable company in the world. You can believe that was entirely due to chance if you wish.
(On beating the market consistently: you don't have to crush the market to extract value. Certain strategies only work for certain amounts of capital. You can be an Apex predator and not drive the entire ecosystem into extinction.)
> a random sequence of 39 heads is less probable than three sequences of 13 heads, each separated by some other unidentified, random sequences
Investors are not going for streaks. They are going for overall return. A better way to put it: what are the chances that over a 48-year timeframe, a random strategy will return ~20% annualized return compared to the market's ~10%? Ignore an arbitrary year limitation (why not measure monthly or daily return?). What is the chance a random strategy would outperform so long?
Or, a better question: what is a chance that a random strategy will be the best-performing investment over a 30-year period (1926-2011), and the subject of academic studies:
http://www.econ.yale.edu/~af227/pdf/Buffett's%20Alpha%20-%20...
Buffett is an existence proof that investment is skill, not chance. He created the 9th most valuable company in the world. You can believe that was entirely due to chance if you wish.
(On beating the market consistently: you don't have to crush the market to extract value. Certain strategies only work for certain amounts of capital. You can be an Apex predator and not drive the entire ecosystem into extinction.)
> A better way to put it: what are the chances that over a 48-year timeframe, a random strategy will return ~20% annualized return compared to the market's ~10%?
Easily answered. Let's say that a 10% return is the mean return, and one standard deviation is 5% -- just an example, and these numbers aren't real (although they could be established by asking everyone what their returns are).
So a return of 20% or better represents two standard deviations above the norm, or 2.2% of the investing population (this is a one-tailed distribution). How many investors will achieve that result in a large population? 2.2%. In a pool of a million investors, that's 22,000 people.
> Buffett is an existence proof that investment is skill, not chance.
With all respect, it's more accurate to say that your view of probability is an existence proof that many people don't understand statistics. Let me ask you -- do you understand how science works? Scientists don't say what you just said, ever. They say that the probability that this outcome resulted from chance is p ("p-value"), referring to the probability that the result arose because of chance. (My 2.2% probability above is a p-value.)
When the LHC scientists announced that they believed they might have detected the Higgs boson, did they say that their measurement constituted an "existence proof" that the Higgs was real? No, because they were scientists addressing educated people, they expressed their result in terms of a p-value -- p was the probability that their result came about because of chance, not the hypothesized particle.
A chance result isn't the last possibility that a scientist considers, it's the first. And no one who has been educated claims that a result that might have arisen by chance constitutes an "existence proof".
Easily answered. Let's say that a 10% return is the mean return, and one standard deviation is 5% -- just an example, and these numbers aren't real (although they could be established by asking everyone what their returns are).
So a return of 20% or better represents two standard deviations above the norm, or 2.2% of the investing population (this is a one-tailed distribution). How many investors will achieve that result in a large population? 2.2%. In a pool of a million investors, that's 22,000 people.
> Buffett is an existence proof that investment is skill, not chance.
With all respect, it's more accurate to say that your view of probability is an existence proof that many people don't understand statistics. Let me ask you -- do you understand how science works? Scientists don't say what you just said, ever. They say that the probability that this outcome resulted from chance is p ("p-value"), referring to the probability that the result arose because of chance. (My 2.2% probability above is a p-value.)
When the LHC scientists announced that they believed they might have detected the Higgs boson, did they say that their measurement constituted an "existence proof" that the Higgs was real? No, because they were scientists addressing educated people, they expressed their result in terms of a p-value -- p was the probability that their result came about because of chance, not the hypothesized particle.
A chance result isn't the last possibility that a scientist considers, it's the first. And no one who has been educated claims that a result that might have arisen by chance constitutes an "existence proof".
I am not an expert in the scientific method, but know a P value of < .05 (in this case, 39/47 is more like .000001) is pretty strong evidence of the conclusion. Of course things can always be due to chance; I may not be a person, but a chimp randomly hitting keys.
At what point do you say "The hypothesis that a professional investor with a published strategy who returns the best-performing fund in history appears to not be based on chance?".
Do you really think 2% of investors (1 in 50!) achieve 20% compound growth over 48 years? Do you know how many billions that is? (Buffet started with $100k and grew it to the 9th biggest company in the world. Where are the thousands of other investing billionaires?)
Do you really think beating the market long-term is a simple 1-time standard deviation computation? (I thought it was impossible, now 50% of people will consistently beat the market long term by any margin?)
Per the cited article, from trained economists:
"Buffett’s success has become the focal point of the debate on market efficiency that continues to be at the heart of financial economics. Efficient market academics suggest that his success may simply be luck, the happy winner of a coin-flipping contest as articulated by Michael Jensen at a famous 1984 conference at Columbia Business School celebrating the 50th anniversary of the book by Graham and Dodd (1934). Tests of this argument via a statistical analysis of the extremity of Buffett’s performance cannot fully resolve the issue."
At what point do you say "The hypothesis that a professional investor with a published strategy who returns the best-performing fund in history appears to not be based on chance?".
Do you really think 2% of investors (1 in 50!) achieve 20% compound growth over 48 years? Do you know how many billions that is? (Buffet started with $100k and grew it to the 9th biggest company in the world. Where are the thousands of other investing billionaires?)
Do you really think beating the market long-term is a simple 1-time standard deviation computation? (I thought it was impossible, now 50% of people will consistently beat the market long term by any margin?)
Per the cited article, from trained economists:
"Buffett’s success has become the focal point of the debate on market efficiency that continues to be at the heart of financial economics. Efficient market academics suggest that his success may simply be luck, the happy winner of a coin-flipping contest as articulated by Michael Jensen at a famous 1984 conference at Columbia Business School celebrating the 50th anniversary of the book by Graham and Dodd (1934). Tests of this argument via a statistical analysis of the extremity of Buffett’s performance cannot fully resolve the issue."
> ... but know a P value of < .05 (in this case, 39/47 is more like .000001) is pretty strong evidence of the conclusion.
This is false, and the fact that it's false has been proven and accepted for years.
http://www.nature.com/news/scientific-method-statistical-err...
Quote: "P values, the 'gold standard' of statistical validity, are not as reliable as many scientists assume."
http://bigthink.com/neurobonkers/the-statistical-significanc...
Quote: "P<0.05 is the figure you will often find printed on an academic paper, that is commonly (mis)understood as indicating that the findings have a one in twenty chance of being incorrect."
Also, your effort to estimate the probability of 39/47 is seriously flawed -- it depends on the distribution of successes and failures within the list.
> At what point do you say "The hypothesis that a professional investor with a published strategy who returns the best-performing fund in history appears to not be based on chance?".
Have you been reading my replies? The answer is never. The scientists at the LHC have acquired a p-value of 5-sigma for their observation of the Higgs, meaning the probability that their apparent detection of the Higgs arose from chance is 2.8 * 10^-7 (the "p-value"), but guess what? They are never going to say, "We detected the Higgs." Only ignorant science journalists say things like that, in the same way that ignorant stock market journalists say, "So-and-so has a reliable method for beating the market."
> Do you really think beating the market long-term is a simple 1-time standard deviation computation?
Yes, that's exactly what it is, given adequate data. And if that cannot be done, then economics has earned its reputation as a non-science.
> Do you really think 2% of investors (1 in 50!) achieve 20% compound growth over 48 years?
That was a hypothetical example, don't try to use it as an argument.
> I thought it was impossible, now 50% of people will consistently beat the market long term by any margin?
You will need to try to edit your posts to make them comprehensible. If you're asserting this viewpoint, I can prove that, for a random market with random buys and sells, with millions of investors, (a) 50% will do better than the average (uncontroversial), and (b) many of them will become rich by chance alone -- not just do better than the average, but become very rich with a small initial stake. Details here:
http://arachnoid.com/equities_myths/index.html#Market_Model
> Per the cited article, from trained economists ...
What? You're invoking the authority of "trained economists" to bolster your viewpoint? First, economists aren't scientists, second, an appeal to authority is a logical error, and third, your choice of quotation supports my view, not yours.
But I ask you to examine your position, and I shall help you do this by way of a reductio ad absurdum. Let's say there's a system for improving on average market returns, that it's something other than a chance outcome, and that it can be defined, tested, and thereby proven to exist in a scientific sense -- that it can be raised above the level of cocktail chatter.
1: Thesis: There is a way to trade securities that can be assured to reliably improve on average market returns, by means other than chance. The method can be written down, and therefore it can be tested objectively.
2: If (1) is correct, then anyone could take the proven, defined method, apply it to the market on a large scale, and drain it of all its capital in a short time.
3: But, notwithstanding the certainty of human greed and simple curiosity, (2) has not happened, anywhere, ever.
4: Therefore (1) is false.
Why is that definitive? Because science cannot be just descriptions ("Warren Buffett has an equities track record"), it requires explanations ("This is why Warren Buffett has an equities track record"). If there is no explanation, there's no science, and there's no point in conversations like this one.
Anyone can describe something in the environment, but the only interesting ideas are those accompanies by a testable explanation, a theory that purports to explain some aspect of nature, a theory subject to empirical test. And the outcome is interesting whether the theory turns out to be true or false -- we learn just as much from false scientific theories as true scientific theories.
And this is a classic debate between someone who understands science, and someone who doesn't.
This is false, and the fact that it's false has been proven and accepted for years.
http://www.nature.com/news/scientific-method-statistical-err...
Quote: "P values, the 'gold standard' of statistical validity, are not as reliable as many scientists assume."
http://bigthink.com/neurobonkers/the-statistical-significanc...
Quote: "P<0.05 is the figure you will often find printed on an academic paper, that is commonly (mis)understood as indicating that the findings have a one in twenty chance of being incorrect."
Also, your effort to estimate the probability of 39/47 is seriously flawed -- it depends on the distribution of successes and failures within the list.
> At what point do you say "The hypothesis that a professional investor with a published strategy who returns the best-performing fund in history appears to not be based on chance?".
Have you been reading my replies? The answer is never. The scientists at the LHC have acquired a p-value of 5-sigma for their observation of the Higgs, meaning the probability that their apparent detection of the Higgs arose from chance is 2.8 * 10^-7 (the "p-value"), but guess what? They are never going to say, "We detected the Higgs." Only ignorant science journalists say things like that, in the same way that ignorant stock market journalists say, "So-and-so has a reliable method for beating the market."
> Do you really think beating the market long-term is a simple 1-time standard deviation computation?
Yes, that's exactly what it is, given adequate data. And if that cannot be done, then economics has earned its reputation as a non-science.
> Do you really think 2% of investors (1 in 50!) achieve 20% compound growth over 48 years?
That was a hypothetical example, don't try to use it as an argument.
> I thought it was impossible, now 50% of people will consistently beat the market long term by any margin?
You will need to try to edit your posts to make them comprehensible. If you're asserting this viewpoint, I can prove that, for a random market with random buys and sells, with millions of investors, (a) 50% will do better than the average (uncontroversial), and (b) many of them will become rich by chance alone -- not just do better than the average, but become very rich with a small initial stake. Details here:
http://arachnoid.com/equities_myths/index.html#Market_Model
> Per the cited article, from trained economists ...
What? You're invoking the authority of "trained economists" to bolster your viewpoint? First, economists aren't scientists, second, an appeal to authority is a logical error, and third, your choice of quotation supports my view, not yours.
But I ask you to examine your position, and I shall help you do this by way of a reductio ad absurdum. Let's say there's a system for improving on average market returns, that it's something other than a chance outcome, and that it can be defined, tested, and thereby proven to exist in a scientific sense -- that it can be raised above the level of cocktail chatter.
1: Thesis: There is a way to trade securities that can be assured to reliably improve on average market returns, by means other than chance. The method can be written down, and therefore it can be tested objectively.
2: If (1) is correct, then anyone could take the proven, defined method, apply it to the market on a large scale, and drain it of all its capital in a short time.
3: But, notwithstanding the certainty of human greed and simple curiosity, (2) has not happened, anywhere, ever.
4: Therefore (1) is false.
Why is that definitive? Because science cannot be just descriptions ("Warren Buffett has an equities track record"), it requires explanations ("This is why Warren Buffett has an equities track record"). If there is no explanation, there's no science, and there's no point in conversations like this one.
Anyone can describe something in the environment, but the only interesting ideas are those accompanies by a testable explanation, a theory that purports to explain some aspect of nature, a theory subject to empirical test. And the outcome is interesting whether the theory turns out to be true or false -- we learn just as much from false scientific theories as true scientific theories.
And this is a classic debate between someone who understands science, and someone who doesn't.
Jumping into the middle of this, while you're clearly up on your science and math, you're arguing against a straw man of your own making. No one is arguing that there's a defined testable method of consistently beating the market. They're arguing that what traders do is consistently find temporary inefficiencies in the market and arb them away and then move on looking for an advantage somewhere else. So no that doesn't mean believing profit can be consistently had means the market can be drained of equity. And obviously any published strategy will quickly stop working, this is well known and accepted by pretty much everyone.
The argument you need to address is whether it's possible for a person to consistently find new opportunities to arbitrage some inefficiency out of the market without relying on arguments that assume there's only one method and that it always works.
The argument you need to address is whether it's possible for a person to consistently find new opportunities to arbitrage some inefficiency out of the market without relying on arguments that assume there's only one method and that it always works.
This is a debate between someone who is trying to force-fit reality into a scientific model and someone who isn't. You would like economists, they make many wonderfully simplifying assumptions to make the reasoning easier (like the efficient market hypothesis, perfectly well-informed, rational actors), which, unfortunately, are incorrect.
There's many things to say, but just consider that you believe beating the market over a 47-year time period comes down to a standard-deviation calculation where half will, half won't. Just ruminate on that. If you can't see the absurdity of it, I'm not sure what to say.
There's many things to say, but just consider that you believe beating the market over a 47-year time period comes down to a standard-deviation calculation where half will, half won't. Just ruminate on that. If you can't see the absurdity of it, I'm not sure what to say.
That's a fundamental misunderstanding of random.
Investing being random chance doesn't mean everyone will do about average. It means that some people will make quite a bit of money and some people will lose quite a bit of money. It just happens that Warren Buffet is an outlier.
Investing being random chance doesn't mean everyone will do about average. It means that some people will make quite a bit of money and some people will lose quite a bit of money. It just happens that Warren Buffet is an outlier.
I'm not sure I understand. Buffett has a strategy X, which has defeated strategy Y (buy a market index fund) 39/47 times.
We can say any return in strategy X is
1) Due to random chance (i.e., it defeated the market 39 times "randomly") 2) Not due to random chance, and there is an inherent advantage
I'm saying the probability of 1) is sufficiently low so as to believe 2).
We can say any return in strategy X is
1) Due to random chance (i.e., it defeated the market 39 times "randomly") 2) Not due to random chance, and there is an inherent advantage
I'm saying the probability of 1) is sufficiently low so as to believe 2).
You're missing the point. What is the probability that someone flipping a fair coin will flip 20 heads in an unbroken sequence? The answer is 2^-20 = about 9.5 * 10^-7.
Next question. How many people need to be flipping coins for one of them to have a better than even chance to flip 20 heads in a row? Answer: about 3/4 million.
Next question. How many investors are there in the world? Answer: many more than 3/4 million.
Next question. If someone among millions of investors makes 20 successful market picks in an unbroken sequence, what's the probability that he will attribute that outcome to blind chance, and what is the probability he will start selling a book titled "Secrets of the Winners" on late night TV?
My point? When confronted by an unexplained occurrence, it's wise to consider the possibility that it's a random outcome. This is called the "null hypothesis" and it's the first possibility a scientist considers.
Next question. How many people need to be flipping coins for one of them to have a better than even chance to flip 20 heads in a row? Answer: about 3/4 million.
Next question. How many investors are there in the world? Answer: many more than 3/4 million.
Next question. If someone among millions of investors makes 20 successful market picks in an unbroken sequence, what's the probability that he will attribute that outcome to blind chance, and what is the probability he will start selling a book titled "Secrets of the Winners" on late night TV?
My point? When confronted by an unexplained occurrence, it's wise to consider the possibility that it's a random outcome. This is called the "null hypothesis" and it's the first possibility a scientist considers.
The question is, how many potential Warren Buffets were there in the pool to start with? You can't ask the question "what are the chances that this particular outcome would happen?" with a sample of one, you have to use the population it is drawn from. Otherwise it would be like saying "That's amazing! The lottery numbers today were 56 23 45 12 27 91! What are the chances!" Exactly the same as any other combination (very, very low). The trick is picking them in advance.
While the probability of any individual defeating the market is 0.5 in any given year, the probability of someone from the pool of N investors getting beating the market 39/47 years is N * 0.5 ^ 47. (It's been a while since I've done probability, so correct me if I'm wrong.)
(47 choose 39) * 0.5 ^ 39 * 0.5^8 = 0.0000022, so that'd be about 1 in 500000 of all who traded for this long.
> the probability of someone from the pool of N investors getting beating the market 39/47 years is N * 0.5 ^ 47.
No, that's wrong. If the original performance had been an unbroken sequence of successful years, say, 39, it would be possible to apply the binomial theorem to it, but the binomial theorem would need to be applied both to the original probability and to the calculation of how many investors would be needed to create a better-than-even prospect of that outcome.
But the 39 successful years were randomly distributed among years where the performance wasn't better than market indices, which makes the probability much higher for it being explainable by chance -- how much higher depends on the actual pattern, which I wasn't able to find.
No, that's wrong. If the original performance had been an unbroken sequence of successful years, say, 39, it would be possible to apply the binomial theorem to it, but the binomial theorem would need to be applied both to the original probability and to the calculation of how many investors would be needed to create a better-than-even prospect of that outcome.
But the 39 successful years were randomly distributed among years where the performance wasn't better than market indices, which makes the probability much higher for it being explainable by chance -- how much higher depends on the actual pattern, which I wasn't able to find.
> Many very smart people have tried to model and then beat the market, and (apart from chance outcomes) no one has succeeded in the sense that they located a reliable, describable "method" for it.
That's an easy claim to make when you simply dismiss anyone who succeeds as random chance.
> by publishing the system he would destroy its effectiveness
Obviously and not disputed.
> I can use a computer to model a random market and random investors, using random price changes and random trades, and not surprisingly, half the investors will do better than the average outcome. What is surprising is that, when the above-average investors are humans, they invariably try to claim this chance outcome resulted from a winning system that they will sell to you for some princely sum.
That's not damning evidence, merely circumstantial.
> Yes, but there's a qualitative difference between "hard" and "impossible".
Yes there is, however I don't believe it's been adequately proven to be impossible.
That's an easy claim to make when you simply dismiss anyone who succeeds as random chance.
> by publishing the system he would destroy its effectiveness
Obviously and not disputed.
> I can use a computer to model a random market and random investors, using random price changes and random trades, and not surprisingly, half the investors will do better than the average outcome. What is surprising is that, when the above-average investors are humans, they invariably try to claim this chance outcome resulted from a winning system that they will sell to you for some princely sum.
That's not damning evidence, merely circumstantial.
> Yes, but there's a qualitative difference between "hard" and "impossible".
Yes there is, however I don't believe it's been adequately proven to be impossible.
> That's an easy claim to make when you simply dismiss anyone who succeeds as random chance.
I never said this anywhere. You're confusing the practice of science with a blanket statement I never made. Pretend to be a scientist -- prove me wrong. Locate where I said what you claim I said.
> Yes there is, however I don't believe it's been adequately proven to be impossible.
Oh, great -- "proven to be impossible". Before you go on, learn about the impossibility of proving a negative:
http://en.wikipedia.org/wiki/Russell's_teapot
Quote: "Russell's teapot, sometimes called the celestial teapot or cosmic teapot, is an analogy first coined by the philosopher Bertrand Russell (1872–1970) to illustrate that the philosophic burden of proof lies upon a person making scientifically unfalsifiable claims rather than shifting the burden of proof to others ..."
Remember -- all I have ever said is that the possibility of a chance outcome cannot, must not, be dismissed, and to a scientist, it is the first possibility to be considered, not the last.
> That's not damning evidence, merely circumstantial.
What? According to the scientific rules of evidence, the burden of evidence is not mine to prove that a winning stock picking strategy does not exist (i.e. prove a negative), the burden is on others to prove that it does.
I never said this anywhere. You're confusing the practice of science with a blanket statement I never made. Pretend to be a scientist -- prove me wrong. Locate where I said what you claim I said.
> Yes there is, however I don't believe it's been adequately proven to be impossible.
Oh, great -- "proven to be impossible". Before you go on, learn about the impossibility of proving a negative:
http://en.wikipedia.org/wiki/Russell's_teapot
Quote: "Russell's teapot, sometimes called the celestial teapot or cosmic teapot, is an analogy first coined by the philosopher Bertrand Russell (1872–1970) to illustrate that the philosophic burden of proof lies upon a person making scientifically unfalsifiable claims rather than shifting the burden of proof to others ..."
Remember -- all I have ever said is that the possibility of a chance outcome cannot, must not, be dismissed, and to a scientist, it is the first possibility to be considered, not the last.
> That's not damning evidence, merely circumstantial.
What? According to the scientific rules of evidence, the burden of evidence is not mine to prove that a winning stock picking strategy does not exist (i.e. prove a negative), the burden is on others to prove that it does.
> I never said this anywhere. You're confusing the practice of science with a blanket statement I never made.
Poorly worded, I wasn't claiming you said it, I was saying you could claim it and you would because you consider it the null hypothesis.
> Before you go on, learn about the impossibility of proving a negative:
I don't need to research Russell's teapot, I'm well aware, that was a poorly thought out statement on my part is all. I didn't mean to ask you to prove a negative.
> Remember -- all I have ever said is that the possibility of a chance outcome cannot, must not, be dismissed, and to a scientist, it is the first possibility to be considered, not the last.
It's not being dismissed. The issue is whether it's just chance; that Buffet's disciples have also done consistently well disputes the null hypothesis. Please actually read what Buffet has to say[1], he's not stupid and he's not ignoring the null hypothesis.
[1] http://www.businessinsider.com/warren-buffett-on-efficient-m...
Poorly worded, I wasn't claiming you said it, I was saying you could claim it and you would because you consider it the null hypothesis.
> Before you go on, learn about the impossibility of proving a negative:
I don't need to research Russell's teapot, I'm well aware, that was a poorly thought out statement on my part is all. I didn't mean to ask you to prove a negative.
> Remember -- all I have ever said is that the possibility of a chance outcome cannot, must not, be dismissed, and to a scientist, it is the first possibility to be considered, not the last.
It's not being dismissed. The issue is whether it's just chance; that Buffet's disciples have also done consistently well disputes the null hypothesis. Please actually read what Buffet has to say[1], he's not stupid and he's not ignoring the null hypothesis.
[1] http://www.businessinsider.com/warren-buffett-on-efficient-m...
Excellent reply.
All true, but on top of being hard, trading is a competitive game. It's not just how good you are -- it's how much better or worse you are than everyone else you're trading against. Furthermore, every successful trade puts you in a better position, and every unsuccessful trade in a worse one.
I wouldn't say no one should trade -- obviously some people do very well. But most of us, I think, should be very careful, and limit our exposure.
I wouldn't say no one should trade -- obviously some people do very well. But most of us, I think, should be very careful, and limit our exposure.
Trading liquid and open markets is much harder precisely because its way more competitive and the barriers to entry to compete are non-existent. And the financial rewards for those who compete well are incomparable.
There appears to be no real evidence that objectively successful traders do exist, other than ones who have non public information (insider dealing, order flow).
I agree that day trading is a terrible idea, but how do you feel about laymen investing with a buy-and-hold strategy? If you buy large cap stocks and avoid making trades there is not much risk.
This kind of strategy enables laymen like myself to do about as well as the market, without paying a middleman such as a mutual fund.
This kind of strategy enables laymen like myself to do about as well as the market, without paying a middleman such as a mutual fund.
> ... how do you feel about laymen investing with a buy-and-hold strategy?
The answer is that buy & hold is a very reliable way to grow your capital, and it does without the services of investment advisors and brokers, which is why they don't want you to find out about it.
On average, buy & hold investors do better than those who enlist the help of brokers or who become day traders (who on average do worse than anyone else).
For actual evidence, one need only examine the outcome of the Wall Street Journal dartboard contest, which compared expert equity picks to random picks. The expert picks weren't sufficiently better than random picks to justify the broker fees, and this outcome went on for years.
http://www.investorhome.com/darts.htm
Quote: "... the performance of the pros versus the Dow Jones Industrial Average was less impressive. The pros barely edged the DJIA by a margin of 51 to 49 contests. In other words, simply investing passively in the Dow, an investor would have beaten the picks of the pros in roughly half the contests (that is, without even considering transactions costs or taxes for taxable investors)."
The answer is that buy & hold is a very reliable way to grow your capital, and it does without the services of investment advisors and brokers, which is why they don't want you to find out about it.
On average, buy & hold investors do better than those who enlist the help of brokers or who become day traders (who on average do worse than anyone else).
For actual evidence, one need only examine the outcome of the Wall Street Journal dartboard contest, which compared expert equity picks to random picks. The expert picks weren't sufficiently better than random picks to justify the broker fees, and this outcome went on for years.
http://www.investorhome.com/darts.htm
Quote: "... the performance of the pros versus the Dow Jones Industrial Average was less impressive. The pros barely edged the DJIA by a margin of 51 to 49 contests. In other words, simply investing passively in the Dow, an investor would have beaten the picks of the pros in roughly half the contests (that is, without even considering transactions costs or taxes for taxable investors)."
I read these sorts of outcomes periodically and it always makes me wonder how on earth managed funds stay in business? How can an industry that does no better than chance on average continue to exist?
> How can an industry that does no better than chance on average continue to exist?
Are you asking why people invest in equities? It's because the market as a whole grows over time, sometimes as high as 15% but at present about 4% after taxes, fees and inflation. This means a buy & hold investment returns a reliable long-term averaged 4%, at a time when virtually no other investment can offer that kind of return (when expressed in the same way).
The above is true if one simply invests in an index fund that tracks (for example) the DJIA, and no trades are made -- no brokerage fees, no taxes until a redemption takes place, and lower taxes than for regular income.
Or are you asking why people have managed portfolios? It's because of widespread ignorance and a bit of the cowboy mentality -- a real cowboy would throw the dice instead of playing it safe. But "throwing the dice" in equities is easily shown to be a foolish act, historically, statistically, and reliably.
Are you asking why people invest in equities? It's because the market as a whole grows over time, sometimes as high as 15% but at present about 4% after taxes, fees and inflation. This means a buy & hold investment returns a reliable long-term averaged 4%, at a time when virtually no other investment can offer that kind of return (when expressed in the same way).
The above is true if one simply invests in an index fund that tracks (for example) the DJIA, and no trades are made -- no brokerage fees, no taxes until a redemption takes place, and lower taxes than for regular income.
Or are you asking why people have managed portfolios? It's because of widespread ignorance and a bit of the cowboy mentality -- a real cowboy would throw the dice instead of playing it safe. But "throwing the dice" in equities is easily shown to be a foolish act, historically, statistically, and reliably.
You are correct in your second guess: why managed funds. And I am tempted to agree with you. It just seems like eventually the truth would out and, what essentially amounts to a scam would stop being able to attract customers.
> How can an industry that does no better than chance on average continue to exist?
The average Joe does not know that.
The average Joe does not know that.
Not just average Joes. Lots of extremely wealthy people put their money in actively-managed hedge funds that also underperform the market: http://www.bloomberg.com/news/2013-12-06/hedge-funds-trail-s...
Their selling point isn't profit, it's convenience. The average worker just wants to check a box on his 401(k) form and be done with it. Mutual funds fill that niche.
Yes, but there are mutual funds that track market indices -- they're not all managed portfolios. So you have the relative stability and security of an index fund, and the convenience of a mutual fund to handle the transactions.
I want to clarify that there's no real security in equities, my use of that term is only relative to other kinds of investments. Everyone should understand that you could lose everything, and there's no deposit insurance or anything like that.
I want to clarify that there's no real security in equities, my use of that term is only relative to other kinds of investments. Everyone should understand that you could lose everything, and there's no deposit insurance or anything like that.
There is nothing wrong with buy and hold executed well...if that fits your investor risk profile and personality.
But you're dead wrong if you think large-cap stocks provide safety from risk. Plenty of large-cap stocks have disappeared completely, another large cohort have had terrible returns.
Any strategy is only going to work if you put disciplined position sizing/money management rules in place. Rules that allow you to trim the dogs and keep the winners, whilst making sure that any large losses don't fatally wound your portfolio.
But you're dead wrong if you think large-cap stocks provide safety from risk. Plenty of large-cap stocks have disappeared completely, another large cohort have had terrible returns.
Any strategy is only going to work if you put disciplined position sizing/money management rules in place. Rules that allow you to trim the dogs and keep the winners, whilst making sure that any large losses don't fatally wound your portfolio.
Buy and hold is great. These days though you can build a very inexpensive diversified portfolio of ETFs (or even index mutual funds with something like a Vanguard Admiral account if you're in the US). For example, you can get a globally diversified equity portfolio using just VT (expense ratio of 0.06%) and VXUS (0.16%). So you are paying a middleman, but only around 0.1%, and you end up with far better diversification, and far fewer securities to manage.
You're probably leaving most of the diversification benefit (which is the only edge you really have, unless you're assuming a fat equity risk premium haha) on the table unless you're a pro at that, as well. I think some of the newer automated-investment services are OK at building portfolios, although I would personally want a lot more exposure to other currencies/markets than my home (see: Home Country Bias)
In other words, there's no easy way to get rich. There is always pain involved, and people looking to get rich quick in financial markets are probably better off learning how to play poker.
> In other words, there's no easy way to get rich.
Of course there are -- very easy ways. But there's no reliable easy way to get rich.
> people looking to get rich quick in financial markets are probably better off learning how to play poker.
If we eliminate "quick", then not true. All one needs to do is invest in an equity index fund and patiently wait decades while ignoring the dubious claims of investment counselors. It's very reliable.
With respect to "quick", that also happens but with much less reliability. Consider that a blind computer program randomly flipping coins, with no strategy at all, will enrich a small percentage of its accounts by chance alone. This works for people too, but it's a rare, chance outcome, not any kind of strategy. More here:
http://arachnoid.com/equities_myths
Of course there are -- very easy ways. But there's no reliable easy way to get rich.
> people looking to get rich quick in financial markets are probably better off learning how to play poker.
If we eliminate "quick", then not true. All one needs to do is invest in an equity index fund and patiently wait decades while ignoring the dubious claims of investment counselors. It's very reliable.
With respect to "quick", that also happens but with much less reliability. Consider that a blind computer program randomly flipping coins, with no strategy at all, will enrich a small percentage of its accounts by chance alone. This works for people too, but it's a rare, chance outcome, not any kind of strategy. More here:
http://arachnoid.com/equities_myths
The only case where a completely random strategy works is if there are black swan events occurring at equally random intervals. I agree on the buy-and-hold index fund advice.
> The only case where a completely random strategy works is if there are black swan events occurring at equally random intervals.
It depends on how you define "works". A random strategy doesn't work at all on average, it can only be said to "work" for the 1/2 of the investors that by chance fall above the average.
And a random sequence is by definition a series of black swans. Unless you're saying that a black swan can only be an extraordinarily unlikely occurrence, in which case all we need do is wait a little longer for the improbable but eventually certain outcome that will meet anyone's definition of black swan.
There are 23 people in a room talking. Pretty quickly two of them realize they have the same birthday. Is that a black swan? After all, there are 365 1/4 days in a year -- how unlikely is it for two of 23 random people to have the same birthday? Is it 23 / 365.25 (6.2%) or is it more or less likely?
My point? Not all apparent black swans are really black swans.
It depends on how you define "works". A random strategy doesn't work at all on average, it can only be said to "work" for the 1/2 of the investors that by chance fall above the average.
And a random sequence is by definition a series of black swans. Unless you're saying that a black swan can only be an extraordinarily unlikely occurrence, in which case all we need do is wait a little longer for the improbable but eventually certain outcome that will meet anyone's definition of black swan.
There are 23 people in a room talking. Pretty quickly two of them realize they have the same birthday. Is that a black swan? After all, there are 365 1/4 days in a year -- how unlikely is it for two of 23 random people to have the same birthday? Is it 23 / 365.25 (6.2%) or is it more or less likely?
My point? Not all apparent black swans are really black swans.
Because playing poker is a much easier way to make a sustainable income!
Hm... actually, it might be. At the very least you certainly don't have to shell out as much for software.
Hm... actually, it might be. At the very least you certainly don't have to shell out as much for software.
Also, in zero sum games like poker and investing, the real question is how much better then the competition you are. In investing, you are up against some really good competition. In poker, you can play against a lot of recreational players, who are could be far less skilled then you. With online poker, you can play many low value hands in parallel to get even easier competition.
That depends on where you invest, all markets are not equally competitive.
I'd love to talk to you about your algorithmic trading experience. Can we get in touch sometime?
I actually just launched http://fxmachine.com with my co-founder, and so far, it looks like our system will enable anybody to achieve a sustainable income without any active involvement.
We're trading on live data (not just making claims based on forward tests) without any human involvement, though human intervention and the ability to recognize trends could have saved some of the trades that ended up being losers. Still, we're looking at sizable gains with a hands-off approach (maybe you'll need to re-train the system every year, but even that can be automated).
We've now run into the problem of telling the world about it; you'd think that 40% annual returns would have large institutions knocking down the door to give us our millions. So it goes!
We're trading on live data (not just making claims based on forward tests) without any human involvement, though human intervention and the ability to recognize trends could have saved some of the trades that ended up being losers. Still, we're looking at sizable gains with a hands-off approach (maybe you'll need to re-train the system every year, but even that can be automated).
We've now run into the problem of telling the world about it; you'd think that 40% annual returns would have large institutions knocking down the door to give us our millions. So it goes!
40% returns seems like a lot. Your quote, "FX.Machine builds profitable trading strategies by learning from historical data", is true for most of the strategies. What makes your strategy different? I would think that many people have tried both statistical and machine learning methods. I do not hear about any 40% returns from them. Your claim of 40% returns makes me doubt your testing strategy. Your forward test shows only 7.9% increase till now. There are a whole set of backward tests missing and a lot of information missing to consider your strategy seriously. For example, how is the risk being measured? Is it drawdown, Sharpe ratio etc?
Thanks for your questions! Feedback like this really helps us.
To your first point about our development strategy: we used genetic programming, and tried to apply novel strategies for which there wasn't much of a precedent in financial applications.
To your second point, re: our testing strategy:
I will concede that we only have 2 months of live trading data, but so far it corresponds well to the behavior we were seeing in our forward tests (what we call the data that the system was not trained on).
We trained our system on 10 years of data (2001 - 2011) with two years of forward tests (2012 and 2013) for evaluating its effectiveness (and to verify that our algorithms have not been overtrained to fit the testing data set). Our forward testing involved taking the same data we would have received from our financial institution and feeding it through the algorithm to see what trades it makes. Having proven our algorithm worked with forward tests, we hooked it up to live data to get real results. So far, so good!
To you third point--risk--we need to explain how our algorithm trades.
We've trained our system to recognize patterns such that it will open a trade with a stop-loss and a take-profit. Because the take-profit and stop-loss are preconfigured, our system doesn't need to monitor data feeds with a time resolution of “real time” to make decisions; the onus is on the brokerage to close winning and losing trades.
The lot size of the trade is calculated such that a losing trade (the price of the currency pair reaching the stop-loss) will incur a loss equal to the risk. So at a risk of 2%, a losing trade on a $1000 account will bring the account value down to $980. Winning trades earn double the risk, so a winning trade on an account of $1000 at 2% risk will increase the account value to $1040.
With all this in mind, we claim 40% annual profit because our live trading exhibits patterns similar to our forward tests, and while our forward tests exceed 40% annual profit, we're extrapolating our two months' worth of live testing performance out to the rest of 2014. There's a good chance that this number will change in the coming months. We will revise it up or down accordingly.
It's possible that our system will need to be used "in the wild" for more than just two months in order for people to take us seriously. If that's the case, I am happy to continue dogfooding FX Machine; so far, it’s been a profitable experience :)
Thanks again for your great questions. We’ll use questions like these to build an FAQ section on the site!
To your first point about our development strategy: we used genetic programming, and tried to apply novel strategies for which there wasn't much of a precedent in financial applications.
To your second point, re: our testing strategy:
I will concede that we only have 2 months of live trading data, but so far it corresponds well to the behavior we were seeing in our forward tests (what we call the data that the system was not trained on).
We trained our system on 10 years of data (2001 - 2011) with two years of forward tests (2012 and 2013) for evaluating its effectiveness (and to verify that our algorithms have not been overtrained to fit the testing data set). Our forward testing involved taking the same data we would have received from our financial institution and feeding it through the algorithm to see what trades it makes. Having proven our algorithm worked with forward tests, we hooked it up to live data to get real results. So far, so good!
To you third point--risk--we need to explain how our algorithm trades.
We've trained our system to recognize patterns such that it will open a trade with a stop-loss and a take-profit. Because the take-profit and stop-loss are preconfigured, our system doesn't need to monitor data feeds with a time resolution of “real time” to make decisions; the onus is on the brokerage to close winning and losing trades.
The lot size of the trade is calculated such that a losing trade (the price of the currency pair reaching the stop-loss) will incur a loss equal to the risk. So at a risk of 2%, a losing trade on a $1000 account will bring the account value down to $980. Winning trades earn double the risk, so a winning trade on an account of $1000 at 2% risk will increase the account value to $1040.
With all this in mind, we claim 40% annual profit because our live trading exhibits patterns similar to our forward tests, and while our forward tests exceed 40% annual profit, we're extrapolating our two months' worth of live testing performance out to the rest of 2014. There's a good chance that this number will change in the coming months. We will revise it up or down accordingly.
It's possible that our system will need to be used "in the wild" for more than just two months in order for people to take us seriously. If that's the case, I am happy to continue dogfooding FX Machine; so far, it’s been a profitable experience :)
Thanks again for your great questions. We’ll use questions like these to build an FAQ section on the site!
If your stops are limit orders, then they can get blown through without execution. If your stops are market orders then your limit trigger price can be quite distant from your fill price. You need to do some serious thinking about your risk management and the long tails associated with extraordinary market moves.
If that 40% is real and you're entirely confident about it as your post implies, just go to the bank and get a personal loan for as much as you can borrow, and invest that. Within a few years you'll have your millions. If you don't do that then you're not willing to gamble your own financial stability on your system, which begs the question why should anyone else?
While this idea is mathematically sound in that our returns so far exceed the interest we would pay on a loan, there are numerous examples against investing money you don't have (the subprime mortgage crisis comes to mind). As well, there are many precedents for investing in technologies or ideas that are not yet proven. You are reading a Y-Combinator site, after all.
I am not purporting to claim that FX Machine will always return 40%+ annual profit forever. Rather, I'm presenting a technology we've developed (as a startup on a startup site!) that trades automatically and, so far, has yielded returns that average (across all currency pairs) in excess of 40% annually. This number was higher on our forward tests, but we have two years of historical data for those and only two months' worth of live testing.
As I said in my reply to ep25, there’s a good chance that our 40% claim will need to be revised either up or down as we continue to dogfood our product. Investing is never a sure bet, even (or especially) when computers are involved. For that reason, I will continue to invest only money that I already have. I'm dogfooding our product with my own hard-earned cash--not somebody else's. What greater measure of confidence is there than this?
I am not purporting to claim that FX Machine will always return 40%+ annual profit forever. Rather, I'm presenting a technology we've developed (as a startup on a startup site!) that trades automatically and, so far, has yielded returns that average (across all currency pairs) in excess of 40% annually. This number was higher on our forward tests, but we have two years of historical data for those and only two months' worth of live testing.
As I said in my reply to ep25, there’s a good chance that our 40% claim will need to be revised either up or down as we continue to dogfood our product. Investing is never a sure bet, even (or especially) when computers are involved. For that reason, I will continue to invest only money that I already have. I'm dogfooding our product with my own hard-earned cash--not somebody else's. What greater measure of confidence is there than this?
> there’s a good chance that our 40% claim will need to be revised either up or down as we continue to dogfood our product.
The question is not about 40% or 100%, if you are beating the market by a good margin consistently, why would you share that secret with everyone else. You trade/invest for your own account and get rich.
If your system cannot beat the market consistently why bother then, just close your shop and invest in an index fund.
The question is not about 40% or 100%, if you are beating the market by a good margin consistently, why would you share that secret with everyone else. You trade/invest for your own account and get rich.
If your system cannot beat the market consistently why bother then, just close your shop and invest in an index fund.
You are correct: we could continue to trade while keeping the tech a secret, and eventually amass millions of dollars. Doing so would take 5-10 years.
If, however, we find an institution who understands the potential of FX Machine and is willing to invest in it, we can amass our millions much sooner than we would on our own. The opportunity cost for us associated with selling FX Machine versus using it personally is not measured in dollars; it's measured in years.
If, however, we find an institution who understands the potential of FX Machine and is willing to invest in it, we can amass our millions much sooner than we would on our own. The opportunity cost for us associated with selling FX Machine versus using it personally is not measured in dollars; it's measured in years.
Congratulations to the quantopian team.
Trading on real data is very different from simulated trading on historic data. There are a lot of leaky abstractions and assumptions that happen during simulations that cause problems once you put real money to work. In the past, here on HN, I have criticized quantopian several times because no one has actually used quantopian to trade real money. This is no longer true.
It will take a lot of time and a lot more capital to figure out all the holes, but they are finally on the right track and I am glad they have 20 people already trading. That is 20 sets of eyes balls to look for lack of fidelity in the simulations.
Personally I am interested to see how good their transaction cost model is when trading hundreds to thousands of shares at a clip.
Trading on real data is very different from simulated trading on historic data. There are a lot of leaky abstractions and assumptions that happen during simulations that cause problems once you put real money to work. In the past, here on HN, I have criticized quantopian several times because no one has actually used quantopian to trade real money. This is no longer true.
It will take a lot of time and a lot more capital to figure out all the holes, but they are finally on the right track and I am glad they have 20 people already trading. That is 20 sets of eyes balls to look for lack of fidelity in the simulations.
Personally I am interested to see how good their transaction cost model is when trading hundreds to thousands of shares at a clip.
Yes, 1000x! I'm waiting to see Quantopian's forward testing with results.
Personally, I couldn't wait for their roll-out. So I wrote my own IB client and traded my Quantopian backtested strategies myself on my IB test account. Race condition, deadlocks much at 3:59PM ET at 2000+/msg burst from quote ticks (esp. when you are doing some kind of stat arb, pairs trading where one order submission is dependent on another order's fill).
Anyways interesting technical problems abound. Gl to the Quantopian team!
Personally, I couldn't wait for their roll-out. So I wrote my own IB client and traded my Quantopian backtested strategies myself on my IB test account. Race condition, deadlocks much at 3:59PM ET at 2000+/msg burst from quote ticks (esp. when you are doing some kind of stat arb, pairs trading where one order submission is dependent on another order's fill).
Anyways interesting technical problems abound. Gl to the Quantopian team!
very interesting!!! but I'd like to try writing my own algorithm offline, in my own prefered language. (javascript)
does anyone know where I can get price history for the last few years for something like an index fund, like DIA?
does anyone know where I can get price history for the last few years for something like an index fund, like DIA?
Sweet. This is perfect for those of us who still have some money we didn't put into MtGox left to lose!
I'd love to have a go at this but I haven't a clue where to even start. Any tips as to where I should start reading?
To get started on Quantopian, take a look at the FAQ (https://www.quantopian.com/faq) and help documentation (https://www.quantopian.com/help). The platform runs entirely on Python and supports US stocks and equities.
To begin coding, clone the sample algorithm on the front page to get familiarized with the IDE. You can then backtest the strategy, for free, using historical data from 2002 - Now. When you run a full backtest, you can see how your algorithm would have performed in the past - you can see your returns, Sharpe ratio, and other metrics.
Once you're comfortable with the syntax and mechanisms, the community is a great place to look for trading ideas to try on your own. Try cloning an algorithm posted in a thread to begin experimenting. If you get stuck, post to the forums - they're a friendly bunch and will be happy to help out.
Of course, past performance is not indicative of future results. When you're ready, you can paper-trade your algo (trade using fake money) on the live market. It's free on Quantopian and a risk-free method to see how your algo performs in the market. And then, the last step is to connect your account to Interactive Brokers and begin live trading with real money. This blog post was an update of the live trading pilot program.
(Disclaimer: I work for Quantopian. But I'd be happy to help answer any questions! Feel free to reach out to us directly as well, [email protected])
To begin coding, clone the sample algorithm on the front page to get familiarized with the IDE. You can then backtest the strategy, for free, using historical data from 2002 - Now. When you run a full backtest, you can see how your algorithm would have performed in the past - you can see your returns, Sharpe ratio, and other metrics.
Once you're comfortable with the syntax and mechanisms, the community is a great place to look for trading ideas to try on your own. Try cloning an algorithm posted in a thread to begin experimenting. If you get stuck, post to the forums - they're a friendly bunch and will be happy to help out.
Of course, past performance is not indicative of future results. When you're ready, you can paper-trade your algo (trade using fake money) on the live market. It's free on Quantopian and a risk-free method to see how your algo performs in the market. And then, the last step is to connect your account to Interactive Brokers and begin live trading with real money. This blog post was an update of the live trading pilot program.
(Disclaimer: I work for Quantopian. But I'd be happy to help answer any questions! Feel free to reach out to us directly as well, [email protected])
How does Quantopian connect to IB?
Doesn't IB require a local TWS connection as well as credentials, e.g. 2-factor authentication?
Doesn't IB require a local TWS connection as well as credentials, e.g. 2-factor authentication?
We run an IB gateway and in the Quantopian UI you enter your IB credentials - but we do not store your password. You then deploy your algo and see a dashboard with updates from the market.
Fore more info see: https://www.youtube.com/watch?v=hye1L86s43M
Fore more info see: https://www.youtube.com/watch?v=hye1L86s43M
Great site. Do you know of anything similar for the UK? It looks like a lot of fun to play with.
Don't.
If he doesn't, who will line my HFT pockets?
That's quite harsh, would you mind to elaborate?
The guy "hasn't got a clue". Basically, inventing random algorithms (even if they back test ok) is a good way of losing all your money. Fundamentally, there are people who write algorithms based on past prices, largely trend following. They generally think they are doing well, without realising that they are making money by being generally long a rising market. Then they lose it all. Then there are the other people who have inside information, largely details of other peoples trades they can front run. They generally make money over the long run. You are not them.
I know very few traders who have either gone mechanical or semi with their own money. It's less about the algorithms, but more about discipline, intuition and knowledge. All of the traders that I mention knowing, are actuaries, who still spend a lot of time working on their models, always backtesting and most importantly, spending a lot of time on risk management, because futures are unpredictable like the future. I tried trading.
Reality is that if I have $10'000 to trade, unless I am successful, which is likely a low probability; 10% of that goes to brokers for all the spreads I pay over time, and the other 90% simply goes to another man/woman/company's pockets.
The one thing that I discounted was the adrenaline rush in seeing a certain currency pair or index go up by a few bips in a matter of milliseconds and that translating into gains/losses in my account. If you do it to 'try' it out, stay away. If you don't understand the effect of a certain EMA converging/whatevering with another EMA, stay out. If Bollinger rings of a friend's surname more than a band, stay out. etc. To even be extreme, I would equate high-leverage trading to gambling for us folk who don't have a lot of knowledge, and I'd go further and equate it to drugs for those who have less of a clue.
As the parent post said, stick to value investing.
Disclaimer: I've spent about 6 years trading in my spare time, learnt lots etc. but called it quits 3 years back to focus on something better with my time. I'm currently 10e10/4-.5 years old.
Reality is that if I have $10'000 to trade, unless I am successful, which is likely a low probability; 10% of that goes to brokers for all the spreads I pay over time, and the other 90% simply goes to another man/woman/company's pockets.
The one thing that I discounted was the adrenaline rush in seeing a certain currency pair or index go up by a few bips in a matter of milliseconds and that translating into gains/losses in my account. If you do it to 'try' it out, stay away. If you don't understand the effect of a certain EMA converging/whatevering with another EMA, stay out. If Bollinger rings of a friend's surname more than a band, stay out. etc. To even be extreme, I would equate high-leverage trading to gambling for us folk who don't have a lot of knowledge, and I'd go further and equate it to drugs for those who have less of a clue.
As the parent post said, stick to value investing.
Disclaimer: I've spent about 6 years trading in my spare time, learnt lots etc. but called it quits 3 years back to focus on something better with my time. I'm currently 10e10/4-.5 years old.
At best, your naive algorithms will under-perform market indices (especially after trading fees). At worst, you'll lose all your money.
Stick with value investing (or index investing if you don't have the skill/discipline/time for it).
In other news, the sky is blue. C'mon man, make a useful comment about the site, not just some generic and completely unrelated investment advice.
You could try and write an algo that would carry out your value investing strategy. You could trawl Yahoo for P/E or whatever value metrics you want to look at buy or sell based on certain thresholds.
Successfully implementing a value strategy cannot be codified into an algo
EDIT: There are just too many exceptions and nuances. Every single investment involves more exceptions and nuances that would need to be coded. And if you don't think so, I would bet you don't have alpha :)
EDIT: There are just too many exceptions and nuances. Every single investment involves more exceptions and nuances that would need to be coded. And if you don't think so, I would bet you don't have alpha :)
I guess I'm agreeing with your statement "Successfully," but it doesn't mean it hasn't been tried:
http://www.jatit.org/volumes/Vol51No1/24Vol51No1.pdf
Also, weren't the quant funds that were involved in the summer 2007 blowup all running automated value strategies based on Fama+French factor models?
It works really great until it doesn't.
http://www.jatit.org/volumes/Vol51No1/24Vol51No1.pdf
Also, weren't the quant funds that were involved in the summer 2007 blowup all running automated value strategies based on Fama+French factor models?
It works really great until it doesn't.
Oh plenty of tryers. Value investing is a lot harder than it looks. And what you're talking about isn't really the definition of value investing (like graham-dodd + infinite details on the individual companies). Those quant funds were running factor models and mid-frequency stat arb
What makes value investing better than index investing? Surely if it is statistically impossible to beat the market on a risk-adjusted basis over time, this also applies to value investing.
[deleted]
I've really enjoyed playing with Quantopian, and I think that I will continue to play with it as an educational tool. However my algorithms so far have been very good at losing an awful lot of money, so I think I will be sticking with simulations for now :)
> my algorithms so far have been very good at losing an awful lot of money
But... the solution is just so obvious, just make the algorithm trade the opposite of what it would have done.
Pure profit!
But... the solution is just so obvious, just make the algorithm trade the opposite of what it would have done.
Pure profit!
A few years ago I heard a story from an acquaintance working at an online spread betting company. One senior figure suggested that they should actively bet against their worst performing customers rather than hedging, they discovered pretty quickly that this wasn't a good idea!
What happened when they tried it?
haha! So... if the algorithm thinks a stock will rise, go short instead and vice versa?
This is quite interesting. I had just built a predictive rebalancing bot to trade on MtGox a few hours before it collapsed. I was rebalancing every 60 seconds and making lots of trades. I only got a couple of hours of run time before it stopped working :(
Here's the bot I built: https://github.com/pontifier/rebalancer
Quantopian looks interesting, and hopefully the exchanges it trades on won't disappear.
Here's the bot I built: https://github.com/pontifier/rebalancer
Quantopian looks interesting, and hopefully the exchanges it trades on won't disappear.
Doubt that. NYSE is definitely a scam. I have heard they pivoted from creating a new social network, like Facebook just better.
It seems the natural next step would be to allow non-algorithm writers to invest in algorithms (or a bundle of algorithms), with some fee going to either or both Quantopian or the writer of the algorithm(s). Any chance that's on the roadmap, and if so, when can I expect to be able to crowdsource my investing?
If only there was some way to invest in an index of some sort of the best algorithms...
But what you don't get with an index (at least, I don't think you do? I admit my ignorance on this) is the sort of transparency and ability to dig deep into the algorithm(s) that it appears you'd get with something like Quantopian. And possibly the ability to tweak that algorithm. Basically, a market for starter Quantopian algorithms that are more advanced than the samples on the site - I think that's vastly different than an index, and I could see paying for something like that.
Sure you do. All index funds disclose their holdings, and index ETFs show their constituent equities, along with any tracking error from the index being tracked. A long time ago, "proprietary" traders would perform index arbitrage in this fashion.
If there were decent algorithms that third parties could dig into, then the opportunity would go away.
You mean something like http://www.collective2.com/ ?
One of the difference is that on c2 you can't see the internals of the algo (because it could also be not an algo but somebody deciding how to invest, actually the algo is not implemented on the site but outside, the site only receives buy/sell commands).
on Techzing podcast can find and interview to the founder (a bit old, 2011): http://techzinglive.com/page/868/155-tz-interview-mathew-kle...
on Techzing podcast can find and interview to the founder (a bit old, 2011): http://techzinglive.com/page/868/155-tz-interview-mathew-kle...
Can anyone in the know tell me why traders are not interested more in sports betting? Seems like you have a lot more people betting with heart then brain which should result in a profit if you employ all your fancy algorithms no?
I've always found https://www.quantconnect.com/ more interesting. They're just not as good as Quantopian at generating some buzz.
For those interested in doing some algo testing I found this site quite interesting: http://systematicinvestor.wordpress.com/
The only issue that I have with that is that they require an IB account. That has a min of $10k. Thats not exactly easy to come by.
If you don't have $10,000 to risk, you probably shouldn't attempt to trade in an automated fashion. You need enough capital, because any one trade is likely to have a small percentage upside and you need to turn it over frequently.
For the next week my comments on Hacker News -- except this one -- will attempt to garner as many points as possible by exploiting various HN tendencies.
Feel free to upvote this comment if you think this is cool btw :)
Feel free to upvote this comment if you think this is cool btw :)
can I add comment ?
Trading is hard. I run a proprietary trading firm with modern technology infrastructure and we have a lot of trading experience. You really have very little chance of success unless you're doing this full time, and even then it will take years before you have enough experience to build a sustainable income.