Although it is true that popular frameworks/libraries/ like Tensorflow, Keras, Pytorch..are written in C/C++, I find it very interesting what you are proposing.
Python is nice but I think it would be a good exercise to use only a compiled language like C/C++/Rust/Nim/other language...