Polymer used to have it's own way of declaring web components to get rid of lots of boilerplate. With v1 of webcomponents, the syntax is similar to React and other class based component frameworks and since v1 is already supported in multiple browsers, they removed the abstractions and are now using native webcomponents' syntax.
Browser vendors finally agreed on how to implement shadow DOM! shadow DOM is the biggest bottleneck for the adoption of webcomponents, the other parts of webcomponents are fairly easy to polyfill. This is great news! With this news out I would say it's a lot safer to start using Polymer right now :).
As mentioned, if you just go for the Material Design look, there's several options, Materialize, MaterialUI & the elements that are on the Polymer site (Please note, they are not part of Polymer, a distinction with which Google is having problems. They want you to see them as two seperate things). Materialize doesn't come nearly as close to the "fun" experience as Polymer/React, though it's easier to get started I guess.
Now, I cannot speak much about the differences between React and Polymer, I've never used React but as mentioned already, React has better browser support. React works great in all browsers right now, whereas with Polymer it is complicated. I'm saying complicated because the general impression is that Polymer after being 1.0 is not production ready or good yet. This is not true. It is production ready if you only use Polymer. However, it still has issues working together with other DOM manipulating frameworks such as Meteor, Angular, etc. ... This is due to the fact that Polymer heavily relies on polyfilling the shadow DOM specs that are currently only implemented in Chrome & Opera.
Their shadow DOM polyfill worked pretty good, but performance was horrible especially in Safari. Which is why with 1.0 they rewrote the polyfill from scratch, now called shady DOM. It emulates shadow DOM in a less strict way, which results in other frameworks not working properly with Polymer as of now. So, look out for a patch that's currently in the works for this issue.
Aside from that, Polymer works really well in my opinion. I think what's imporant to understand though is that React and Polymer fulfill different purposes. Polymer is a very bold undertaking of trying to bring web components right now to your browser. React has a similar approach afaik in the sense that you can easily create reusable components but it's just a different technological approach based on what's possible right now. That's also why React performs better because they don't try to polyfill stuff. If you look at the webcomponents and webcomponents-lite polyfills you might think "holy shit". That's some serious effort they put into that. So much polyfilling will hurt performance obviously.
React is eventually going to use webcomponents, just like Polymer. So what will be the difference of React and Polymer in the future? Hard to say. I'm guessing they will be very similar, in fact who knows maybe React will use Polymer, not for the polyfills (which technically isn't a part of the Polymer project ... anymore[?]) but rather for the webcomponents sugar coating on top that makes writing web components easier and more fun.
Now, your question was what should you use for Material design. If you just use Polymer go for it right now (some browsers will be a bit slower, namely Safari but whatever. Progressive enhancement FTW. It'll get better, they're working on implementations). If you need to couple it with another framework, carefully do some first tests. If you know you only ship in a few months from now then it's probably safe to go with it, since a patch will certainly be out by then.
If you want great performance and browser support right now, go for React.
If you don't care about supporting ancient browsers and want to support only evergreen browsers then choose whatever you like better.
As for me, I'm about to launch my startup's site with Polymer soon, hacking some of the issues that are still there when integrating it with meteor.