Hey—congrats on the launch! I'm a big Valorant player (peaked Immortal 3 a few seasons back) and I've also used/talked to the people at visor.gg and pursuit.gg when they were around.
Totally understand the concerns here around being at the whim of the publisher, and previous companies have been blindsided by changes in how companies like Blizzard decide what is and isn't allowed on their platforms. Interestingly, YC has had a few of these companies go through similar experiences but I think they still think there's something to be built in this space.
For what it's worth, I think this is a good idea that's going to run into similar issues, regardless of what safeguards are put in place to avoid stepping on the publishers' toes. I'm sure there are some learnings from talking to Ivan or James from the last two YC companies that tried to do this. Also happy to share some more of my thoughts :)
Congrats Geoff! You helped us a lot when we were young, first-time founders and we learned a lot from you and Tim. Thanks again for all your advice—excited about the future for YC :)
It was called Chesscademy and we were raising $750k.
We were tackling the adaptive learning problem by creating tailored curricula. The plan was to move outside of chess to other subjects as we grew, but we found monetization to be very difficult.
Chris is an amazing person who takes the time to get to know you.
When we were fundraising, instead of having us do a regular pitch he took me to an event where Peter Thiel was giving a talk about his new book. We were a chess education company, so he thought it would be good for us to talk with Thiel as he's a chess master himself.
He didn't end up investing in us but my experience with him was much more memorable than any of the other meetings we had while fundraising.
The list that's returned still contains mainly tactical surprises where Stockfish inaccurately evaluated the position at the end of depth 5. I think what I'm trying to say is there are some moves in a position that aren't tactically surprising (a piece sacrifice, a crazy attacking move, etc.) but positionally surprising (a long maneuver to get a piece to a certain square that I didn't think of). These positionally surprising moves aren't captured by this methodology because they don't involve large fluctuations in valuation when the depth changes.
As to your second point, an issue with how computer chess affects the modern scene is how playing the "best" move in any given position isn't representative of how humans play. Humans carry out plans and evaluate positions to the best of their ability, but the heuristics and procedure they use aren't the same as a computer's. For example, Karjakin didn't prepare for his match against Carlsen last month by playing a bunch of games against Stockfish. Rather he probably analyzed Carlsen's past games and opening choices to come up with a strategy.
I do think you can come up with a way to prepare against individually known opponents by identifying weaknesses programmatically. You can model a human's approach to playing chess as a distribution of parameters (material, king safety, pawn structure, etc.) that take in the current position and return the best move. You also have Stockfish's evaluation which returns the "best" move. With this, it's possible that you could use build a neural network that learns to play very similarly to a certain player by using their past games as a training set and comparing the chosen move to Stockfish's move. The network could learn to mimic the heuristics that the human individual uses to make decisions and playing against this new AI would be great practice for preparing against specific opponents.
If I understand your code correctly in analyse_evaluations, you're defining a "surprising move" as a move that has a large change in valuation when it's considered at a higher depth. So if a "human" (really Stockfish at depth 5) evaluates a move as +1 and a "computer" (Stockfish now at depth 11) evaluates the move as +5, the move is surprising.
This is pretty interesting, but I'm not sure if it fully captures all the nuances of what a surprising move is. You might be able to classify a move as tactically surprising if it becomes clear after depth 7 that the ending position is favorable. However, in my opinion truly surprising moves are ones that carry plans that I haven't even considered. Hence, this methodology doesn't capture moves that are positionally surprising as there wouldn't be such a drastic change in evaluation at different depths. I'm not sure where you would start to figure that one out though :)
They've added a few more ML courses—the go to class for undergrads is now ORF 350 (Analysis of Big Data) with Han Liu. ELE 535 (Pattern Recognition and Machine Learning) is also a new course that was added last year.
For those interested in the "Probability and Statistics" portion of the guide, Princeton recently created a certificate program in statistics and machine learning that has some more updated information on courses: http://sml.princeton.edu/
I'm pursuing the certificate right now and the courses have been great so far. Princeton's known for having a rather theory-heavy approach in their quantitative classes but I've found a good balance with applications in some of the classes (COS 424, COS 402).
It's worth mentioning that Lee Sedol mentioned in an interview that even if he loses a single game against AlphaGo, he will have lost the match. He was expecting to win all 5 games.
Totally understand the concerns here around being at the whim of the publisher, and previous companies have been blindsided by changes in how companies like Blizzard decide what is and isn't allowed on their platforms. Interestingly, YC has had a few of these companies go through similar experiences but I think they still think there's something to be built in this space.
For what it's worth, I think this is a good idea that's going to run into similar issues, regardless of what safeguards are put in place to avoid stepping on the publishers' toes. I'm sure there are some learnings from talking to Ivan or James from the last two YC companies that tried to do this. Also happy to share some more of my thoughts :)