I personally completely agree with you that we should use simplest javascript to achieve a project requirement. I guess the decision is really depend on:
Do you have to build single page application?
If yes, then its easier to build via a framework like Angular, react or vue.
If no, then you will have freedom to choose whatever library you want to use in order to satisfy requirement.
In addition, recently I started to look into web components. It is now part w3c standard and supported by most modern browsers. It allows us to construct components via raw javascript.