A simulation of God's decision tree for the Serenity Prayer: "God, grant me the serenity to accept the things I cannot change, the courage to change the things I can, and the wisdom to know the difference."
I wonder if a language model can be treated as a policy over token-level actions instead of full response actions. Then each response from the language model is a full rollout of the policy. In math and coding, the reward for the response can be evaluated. This is not how DeepSeek works now, right? It treats full responses from the language model as the action if I understand correctly.
This adds PyTorch/CUDA training support to Andrej Karpathy's minbpe. It takes 2min 28sec (148 seconds) on an RTX4090 to train the BasicTokenizer with a vocab_size of 512 on 307MB of Enron emails. The original code takes 2hrs 15min (8076 seconds) on an M2 Air with Python 3.11 to do this. That is a 55x speedup.