It's quite unfavorable on modern hardware. A Sapphire Rapids core can do 2 separate 32 half-precision FMAs (vfmadd132ph, [1]) per clock, which is 128 FLOPs/cycle. It is not possible to achieve that kind of throughput with an 8-bit LUT and accumulation, even just a shuffle with vpshufb is too slow.
I think the mathematical concept that you are looking for is that of the dual space. Essentially if you have a vector space V, you can construct a dual space V* where the elements of the dual space are functions taking elements of V to the underlying field F, and under certain conditions these spaces are isomorphic (the same) - so there is a 1:1 correspondence between elements of the vector space and the functions in the dual space.
One way to view this formula is to use the fact that the Beta distribution is a conjugate prior for the binomial distribution.
Essentially if you have a Beta(a, b) prior then your prior mean is a/(a+b) and after observing n samples from a Bernoulli distribution that are all positive, your posterior is Beta(a+n, b) with posterior mean (a+n)/(a+n+b). So in your example you effectively have a Beta(0, x) prior and x (“suspicious”/“gullible”) is directly interpreted as the strength of your prior!
TF32 is not IEEE-754 float32, it is a reduced precision format designed for machine learning usecases. The correct specsheet number for FP32 (and FP64 which is the relevant precision here) throughput on H100 is more like 60TFLOP/s, so your number is off by roughly an order of magnitude.
I think the two key texts here are Graeber’s Bullshit Jobs and Mark Fisher’s Capitalist Realism, specifically his riff on “market Stalinism”:
> The idealized market was supposed to deliver ‘friction free’ exchanges, in which the desires of consumers would be met directly, without the need for intervention or mediation by regulatory agencies. Yet the drive to assess the performance of workers and to measure forms of labor which, by their nature, are resistant to quantification, has inevitably required additional layers of management and bureaucracy. What we have is not a direct comparison of workers’ performance or output, but a comparison between the audited representation of that performance and output. Inevitably, a short-circuiting occurs, and work becomes geared towards the generation and massaging of representations rather than to the official goals of the work itself. Indeed, an anthropological study of local government in Britain argues that ‘More effort goes into ensuring that a local authority’s services are represented correctly than goes into actually improving those services’. This reversal of priorities is one of the hallmarks of a system which can be characterized without hyperbole as ‘market Stalinism’. What late capitalism repeats from Stalinism is just this valuing of symbols of achievement over actual achievement.
For folks wondering about applications of this theorem: it is a key building block in the theory of reproducing kernel Hilbert spaces (RKHS), which in turn are the building block of kernel support vector machines (kernel SVMs), which are widely used in machine learning applications.
The "kernel trick" from kernel SVMs only works because of the existence and uniqueness result from the RRT on the underlying Hilbert space.
- https://gist.github.com/nadavrot/5b35d44e8ba3dd718e595e40184...
might be of interest