Depends on what sort of RL you are doing. If you are trying to train agents to play small games with vision the agent will need a small cnn to process images. This will need a gpu and what you have should be enough.
I was training on atari for a while with 1080ti. The games run on the cpu so you need a decent cpu as well.
I think it's a derivative of CFR, with a lot of optimizations and abstractions. They reduce the search space by grouping different card combinations together e.g. KK and QQ are treated same. They also limit the action space and only allow half-pot or full pot bets.
This is what I remember. I went through what they published briefly sometime back. So I could be wrong here.