D is awesome for what we do with it. It unlocks our small team of data scientists to have a huge impact on the company, as we don't have to rely on frameworks such as your typical hadoop/spark cluster which require engineers just to maintain it. We can also use the same code path offline and online where we have very strong latency and QPS requirements.
D makes us ship more often, with less humans.
The runtime is not perfect yet, but the language and compiler are awesome. It keeps our codebase small, simple and very efficient.
In very large-scale sparse settings, the optimization strategy is tightly coupled to the modeling, not orthogonal to it. The reason why is that "data beats algorithm": meaning that a "dumb" model that can be trained on 1TB of data using a "dumb" optimizer will easily outperform a fancier model for which we only have at our disposal more complex optimizers that can only eat 1GB of data in the same amount of time. This paper is a classic when it comes to the kind of trade-offs you have to deal with in large-scale web settings: http://papers.nips.cc/paper/3323-the-tradeoffs-of-large-scal...
The kernel trick uses an implicit mapping into a higher-dimensional feature-space. On the other hand, Factorization Machines uses an explicit mapping into the polynomial kernel space. However it learns jointly the "right" polynom, by mapping the base features into a low-dimensional dense space where the higher-order terms of the polynom are dot-products (we're not taking dot-products in the original feature-space!). FM then learns the right polynomial (a non-convex taks) jointly with learning the original supervised learning task.
The idea is that FM is a way to learn the polynomial kernel, by representing each high-order term as a low dimensional dot-product. It improves generalization error by learning a better polynomial kernel, because terms in the kernel are learnt jointly instead of separately, which helps a lot on the sparse combinations of features.
Not quite - it's different from the kernel trick, which is impossible at this scale (there's no way you can train an RBF kernel in a decent amount of time when your space has 10^8 features and your training set has 10^9 observations). The idea of factorization machine is to learn the polynomial kernel, and it provides a mechanism to do so which scales "linearly" (with some arbitrary constant that you're in control of).
We built all of our machine learning backend in D at AdRoll. Some examples:
- learning of large-scale classifiers and regressors using custom optimizers
- real-time pricing of billions of ads a day using these models on ad exchanges. <.5ms latency to parse complex bid requests and compute sparse and simd dense dot-products
- a real-time event processing system that hits DynamoDB with ~4.5K json queries per sec on a single node
We literally have D systems deployed on hundreds of ec2 instances as we speak and responsible for mission critical tasks of a >100$M run rate company.
D is ready for prime time and works at scale.
D is awesome for what we do with it. It unlocks our small team of data scientists to have a huge impact on the company, as we don't have to rely on frameworks such as your typical hadoop/spark cluster which require engineers just to maintain it. We can also use the same code path offline and online where we have very strong latency and QPS requirements.
D makes us ship more often, with less humans. The runtime is not perfect yet, but the language and compiler are awesome. It keeps our codebase small, simple and very efficient.