More niche than many yes - but there’s something about learning an array language that changes how you look at programming in general. And for the better. Of course, the places where these things are used are generally very well paying positions. They tend to be areas with a very high amount of data that is incredibly valuable.
intentional. I consider kdb/kdb+ to be understood as the KX products - but L runs the same database style functionality (the qSQL is essentially just sugar atop the q/k code. Most of which is written in k [or C in L for a number of components])
fair point. I am not a front end designer and I did want something less spartan than proverbial q.txt [1] .
To be honest - many folks in the community have had access for some time , and a website was just put up this week as a 'larger community access'.
That said the mail group has more details and the blog posts are effectively internal posts turned into shorter (less code) versions. The binary itself is years of effort - with small improvements over many years and shaving a few kb each time ... 'built the old way' like amish furniture :)
the early versions of L (circa ... 2011/2012) couldn't really deliver any substantial performance improvement over what is out there (BQN/ngn/Kona). The memory bandwidth was the limit - I simply couldn't keep the cores active. ~2018/2019 I started from scratch with arrays (vectors) using compression by default. That was a massive unlock - and I could not keep most of the CPU doing actual compute! Then it was years of working on compression native operations - some of which were obvious[3] like sum/reductions ... others not so much!
Definitely not a vibe coded language - I really wish the ai models were more helpful - but I should do a write up of the assembly analysis (and asm2vec). ATW is working on even more impressive things at the moment! Well beyond the scope of an interpreter itself.
KlongPy and KlongPy + duckDB are wonderful. GREAT job. I think there's a whole world of backprop / ML work that can be done in that style! Given the heritage not surprising that most Klong code is ~= L in k mode! Same with ngn/k . E.g. sigmoid is almost the same (:: for assign in klongpy , : for l/q/k). The autograd work is great!
Sorry about that. Basically you can now write and run code in the K/q/or qsql languages on your laptop for free. For a very long time (20 years+) this language has not been accessible to most - it’s almost exclusively used on Wall Street and has a very high price barrier to entry. Q programmers are still some of the most highly paid engineers consistently - as the workforce is small and the use cases are (generally) extremely lucrative. If you are a programmer wondering what comes next… q might be for you. Over the decades I’ve taught a few dozen people Q- and I’m happy to say they’ve gone on to have wonderful and stable careers in high paying jobs with deep stability. Mostly hedge funds and banks (and exchanges).
It’s a wonderful and often times world changing language and now you can play with it yourself.
It’s also very very fast. As a database it’s still insanely great for many usecases.
Unfortunately the code itself is in a style of C many find difficult to read. I blame my upbringing. ATW open sourced examples and it was not really helpful. More recently others are doing a step by step in more standard C https://github.com/ardentsia-cgs/kparser/
simple example at https://lv1.sh/blog/compute-on-compressed/
But in general compression is reducing the bit width of the input through an encoder (FOR or Frame of Reference is an old and good example). So we store the base in an offset location then the large payload is a much smaller size. E.g. i64 can goto i16. Then simd gets more #'s per cycle on the smaller, and the base is added to the scratch in stack (for sum). avg is similar (since it is just sum / count)
yup website is Claude Design for prototype. For the core ... AI has been less helpful than I hoped - I believe largely because array style languages have so little source in training? But where Claude was especially helpful (other than the web design w/ Claude Design for the prototype) was analyzing ASM output of functions and optimizing those (although it was hard to go ASM>C or ASM>rust). E.g. lots of small mistakes would have been missed by not having restrict/const in places. Claude was great at compile all functions, analyze ASM, suggest optimized ASM (and ASM2vec was helpful as well for finding any "similar" code paths that could be combined (e.g. var/dev/cov are just moments)
BQN & CBQN are absolutely wonderful pieces of code.
L is mac/lin but linux is avx512 only specifically to try to deal with that problem. The compute on compressed algos helps fit more in those cache lines! https://lv1.sh/blog/compute-on-compressed/