Hi, I'm the founder of Q. You're right about the turnover, though it is a multi-faceted trade-off. While increasing the trading frequency accelerates the accumulation of data, it also increases drag from transaction costs and that tends to lower a strategy's capacity.
To be fair, we invested our own money in the winner.
The composition of the algo portfolio for the fund != the winners only. Our challenge is to choose the optimal set of algorithms from a huge pool. Optimal means maximizing returns while also minimizing correlation between the strategies.
The holy grail of investing is 10 or so uncorrelated return streams. We're building a community and platform to consistently turn out algorithms with uncorrelated return streams.
Here's how I think about our contest:
If you only had backtesting, you could win by fitting.
If you only had 1 month of paper trading, you could win by luck (taking excessive risks).
But our contest is 2 years of backtesting + live trading for a month. I don't think it is likely you can both overfit and be lucky with one strategy.
Investing is planning for the future. It is a critical tool for society. Everyone from individuals to important institutions like universities and pensions need to invest. It is a need of the many, not a diversion for the few.
Because of HFT, it has become popular to blindly condemn investing.
I went to college because an endowment was able to provide me with a scholarship. Investing has a huge effect on society and on people.
Sector weighting equally is based on diversification. Market cap weighting will tend to bet more on winners (the market cap is going up), concentrating your investment there.
Another way to think about it: market cap weighting is a momentum strategy. Equal across sectors is mean reversion.
zygomega, I'm one of the zipline maintainers and we'd love to collaborate with you. There are a few people doing HFT research with zipline, and there's a lot of work to do. At quantopian (my day job), we focus on longer hold periods, so there is room in the zipline ecosystem for you to do HFT.
The main benefit we've found with python as the algo language is that it allows for stat programming with pandas, but also OO or functional programming for the algo logic. This smoothes the transition from research to production, just as you're describing with R -> haskell, but you can stay in one language.
I think one of the biggest potential wins with parallelization is if you can assume all positions are closed overnight, most often true for HFT. That way, you can simulate all the trading days in a test range in parallel. This is quite similar to the parallel processing we do to handle the large number of concurrent backtests running at quantopian. We did all of that with python, but I'd be fascinated to see it done with haskell.
It would be cool to do that with this signal, if the algo was buying/selling on another signal. Maybe use the short interest signal as a gate on momentum investing for example.
You're speaking truth. We (quantopian) deal with all of those headaches, and test the algos with fully adjusted data. Splits, symbol changes, mergers, divestitures, dead companies, dividends - they're all covered.
If you click on the code and search for commissions, you'll see how those costs are taken into account. The big missing thing is the market for borrowing the stock to do the short side of the trade.
No money has traded on my version no. But, I understand that asset management firms have licensed the more sophisticated one Jess wrote at TR, so I would think they use it with real money. From what I understand, firms look at numerous signals like this, and then make investments based on a combination of the signals.