Mojo is trying to create a new language to solve the problem, and specialized for CPU. We are using a more pragmatic way to solve GPU AI computation problem.
We developed AITemplate majorly for Meta's focus at that time, eg Ads/Ranking need. For HippoML is startup we are building for Generative AI. HippoML is not using AITemplate.
Hippo is faster than AITemplate, and supports more generative models. We haven't compared vs TVM, but for absolute token/s on M2 Max, Hippo is able to run decoding on LLAMA with datacenter level GPUs performance (with other SW).
Summary:
Tensor program is able to be optimized by using machine learning and transfer learning. The numerical program optimization model is trained on feature from low-level AST of the program.
Experiments:
Tasks: ResNet, MobileNet, LSTM LM, DQN
Hardware: CUDA/ARM GPU/ARM CPU
Speed up compare to CUDNN, TensorFlow Lite and ARMComputeLib: ~from 1.2X to 3.8X faster in end-to-end test.
There is a huge distributed performance advantages vs TensorFlow. You can get a hint from Prof. Carlos Guestrin's keynote talk at Data Science Summit 2016. Also, CMU CS Dean Andrew Moore cited MXNet as "is the most scalable framework for deep learning I have seen"
The author is here :) Thanks for your suggestion. The detail of network modification will be released in my thesis, there is a "fast good net" but currently I don't have time to make it faster, so I simply use fast poor net.