I'm not an expert but for me it's seems that the proposed approach doesn't compute the value function . By optimizing directly the policy function aren't we losing some key ingredient for generalization? I mean could this algorithm be used in a fully non deterministic environment ? Like human vs machine?