Stages of denial in encountering K(nsl.com)
nsl.com
Stages of denial in encountering K
http://nsl.com/papers/denial.html
422 comments
I use to program financial applications in K/Q for a stint. Very elegant and orthogonal language. And very well documented. One major issue I had with it and other APL-ish languages: lack of skimmability.
I can easily skim thousands of lines of Perl and Java in seconds and have a rough idea of what the code does. Reading K, however, requires careful and deliberate attention to every single character. I found it useful to maintain documentation on how to /read/ the code (almost a literate programming approach).
I can easily skim thousands of lines of Perl and Java in seconds and have a rough idea of what the code does. Reading K, however, requires careful and deliberate attention to every single character. I found it useful to maintain documentation on how to /read/ the code (almost a literate programming approach).
the thing that's odd to me is that you could write this exact same post about other languages - a person who learns Haskell ends up writing that exact same range/reduce/plus statement. Modern lisps like Racket or Clojure take you there. Forth programmers and Joy programmers already noticed that their two languages had converged: and they'd write basically exactly this, too. Really, it's only the "mainstream" languages that are different!
This was written by 'RodgerTheGreat, whose comments and submissions on this site are a treasure trove.
Look, I don't hate K. It's probably a pretty good language for the task it seems to have been designed for, which appears to be numerical computing. Being able to fit an entire code on a screen is an interesting concept; while I'm not completely convinced it should be a goal in and of itself it's certainly something I can't rule out as a possible productivity booster.
However, I still find it problematic, and it's not really about the language itself: it's more about how it's promoted in the article, by the people in this thread, and in general. Specifically, the claims it's perfectly readable, and anyone who says anything otherwise is either stupid, lying, or too poor to understand it are not useful. (Yes, you know who you are for that last one. You're not helping your case.) The fact of the matter is that most programmers are used to ALGOL-esque syntax, and there is significant value in being able to lift concepts and interoperate between those languages at a superficial level. Every "inspired" languages hits these issues, depending on how "strange" it is: Objective-C is ugly, Lisp is alien, and on the far end APL derivatives are simply labelled as unreadable. That doesn't mean they're bad languages, but to those promoting it: drop the attitude, and understand why we have reservations. Your language isn't the "true" way to program, that only the chosen elect can understand and lecture us on. Don't ignore or gloss over languages that have similar features to yours. We want to hear what the interesting things are that you can do in the language, actual guidance for how we can adapt to the language (not: "one-letter identifiers are just what we do; deal with it"). Thanks.
However, I still find it problematic, and it's not really about the language itself: it's more about how it's promoted in the article, by the people in this thread, and in general. Specifically, the claims it's perfectly readable, and anyone who says anything otherwise is either stupid, lying, or too poor to understand it are not useful. (Yes, you know who you are for that last one. You're not helping your case.) The fact of the matter is that most programmers are used to ALGOL-esque syntax, and there is significant value in being able to lift concepts and interoperate between those languages at a superficial level. Every "inspired" languages hits these issues, depending on how "strange" it is: Objective-C is ugly, Lisp is alien, and on the far end APL derivatives are simply labelled as unreadable. That doesn't mean they're bad languages, but to those promoting it: drop the attitude, and understand why we have reservations. Your language isn't the "true" way to program, that only the chosen elect can understand and lecture us on. Don't ignore or gloss over languages that have similar features to yours. We want to hear what the interesting things are that you can do in the language, actual guidance for how we can adapt to the language (not: "one-letter identifiers are just what we do; deal with it"). Thanks.
Not mentioned in the article is that the K programming language is used for kdb+, which is a fast in-memory time-series database that's heavily used in the finance industry for securities trading systems. I've worked with it a tiny bit, but the pros are all Dark Wizards.
I love the brevity of regular expressions and use them on a daily basis. It is the same argument that keeps me returning to K: the syntax is terse and compact, the semantics are simple and composable, and your eyes get used to it.
Beyond a point however, I cannot read my own regex's after a month's absence. Which is why I use perl's /x modifier extensively to split up regex components onto multiple lines and to document them thoroughly, even if they are for throwaway scripts, because I don't always throw them away!.
For example:
This is where K fails me. It may not be a fault of the language, but everyone in the community has bought into this strange idiomatic style. I can't imagine debugging it, or checking it for correctness, or foisting it on a less experienced developer. Here's a canonical example an xml parser, on their website.
https://a.kx.com/a/k/examples/xml.k
Where's the pedagogy? Where are the comments? Why is this line noise considered acceptable?
Beyond a point however, I cannot read my own regex's after a month's absence. Which is why I use perl's /x modifier extensively to split up regex components onto multiple lines and to document them thoroughly, even if they are for throwaway scripts, because I don't always throw them away!.
For example:
$_ =~ m/^ # anchor at beginning of line
The\ quick\ (\w+)\ fox # fox adjective
\ (\w+)\ over # fox action verb
\ the\ (\w+) dog # dog adjective
(?: # whitespace-trimmed comment:
\s* \# \s* # whitespace and comment token
(.*?) # captured comment text; non-greedy!
\s* # any trailing whitespace
)? # this is all optional
$ # end of line anchor
/x; # allow whitespace
This is where K fails me. It may not be a fault of the language, but everyone in the community has bought into this strange idiomatic style. I can't imagine debugging it, or checking it for correctness, or foisting it on a less experienced developer. Here's a canonical example an xml parser, on their website.
https://a.kx.com/a/k/examples/xml.k
Where's the pedagogy? Where are the comments? Why is this line noise considered acceptable?
This article should end with something like, "...and then you find out about arcfide's Co-dfns compiler..." https://github.com/Co-dfns/Co-dfns
You don't have to like it, but the existence of K, et. al. shows that most of us are wasting a huge amount of time and energy using bad tools. These things are so powerful and not that hard to learn.
You don't have to like it, but the existence of K, et. al. shows that most of us are wasting a huge amount of time and energy using bad tools. These things are so powerful and not that hard to learn.
A million-line program isn't readable by anybody, no matter how readable the language is. If the equivalent program can be written in, say, a thousand lines in some more concise language, that's more than worth the learning curve, even if the language is strange and off-putting.
How does error handling work in K? For example, what happens when you try to read a file that doesn’t exist? How does the read function indicate failure to the caller? How does the caller check and recover (show an error to the user and continue with some default values)?
Looks like the author figured it all out at the very beginning:
> If each punctuation character in the K is a separate part of speech, and you reverse their order
https://en.wikipedia.org/wiki/Maxwell%27s_equations#Formulat...
You would not expect them to read them like a paragraph. You are expect to read them symbol by symbol and unpack from the inner out.
K seems to be a dense language. Expect to read it symbol by symbol, not word by word or line by line.
If will be come more readable if you (1) slow down (2) get used to the programming patterns.
Not saying K is a good language or anything. But the frustration of the author seems to have more to do with impaticence and mismatched expectations than with flaws in the language itself.
> If each punctuation character in the K is a separate part of speech, and you reverse their order
+ / ! 100
plus reduce range 100
https://en.wikipedia.org/wiki/Maxwell%27s_equations#Formulat...
You would not expect them to read them like a paragraph. You are expect to read them symbol by symbol and unpack from the inner out.
K seems to be a dense language. Expect to read it symbol by symbol, not word by word or line by line.
If will be come more readable if you (1) slow down (2) get used to the programming patterns.
Not saying K is a good language or anything. But the frustration of the author seems to have more to do with impaticence and mismatched expectations than with flaws in the language itself.
I've seen a few posts here and there over the years about array languages, but I can't recall any (or any discussion in their comments) that have mentioned any disadvantages beyond more human factors.
So for the k/q/APL/array language gurus out there: in what situations would I not want to use k or another array language, if the relative obscurity of the syntax and mental model were not a factor (e.g., assuming you, your team (if any), and anyone who would ever look at your code would be very familiar with the language and idioms)?
I may not want to choose Go or Java because of latency concerns, or Rust because of compilation speed/iteration time concerns, or C/C++ because of safety issues, or Python/Ruby/Javascript because of performance issues. Why would I not want to pick k?
So for the k/q/APL/array language gurus out there: in what situations would I not want to use k or another array language, if the relative obscurity of the syntax and mental model were not a factor (e.g., assuming you, your team (if any), and anyone who would ever look at your code would be very familiar with the language and idioms)?
I may not want to choose Go or Java because of latency concerns, or Rust because of compilation speed/iteration time concerns, or C/C++ because of safety issues, or Python/Ruby/Javascript because of performance issues. Why would I not want to pick k?
Only because earlier today I so definitely thought about the whole "code is a liability (debt)" reality we all live in... that is to say, I'm spit-balling here:
but for once K makes me go "oh shit!" and not because of the craziness of its syntax... rather, if that syntax is so precise/terse, and even if to the outsider it looks fucking insane, does it have a wicked lower TCO because you literally have insanely less lines of liability in your code, libraries, dependencies, etc?
Is that the reason finance uses it, they maybe empirically know what the rest of us don't seem to grasp? Or just because it's so domain specific? Or... something else entirely?
but for once K makes me go "oh shit!" and not because of the craziness of its syntax... rather, if that syntax is so precise/terse, and even if to the outsider it looks fucking insane, does it have a wicked lower TCO because you literally have insanely less lines of liability in your code, libraries, dependencies, etc?
Is that the reason finance uses it, they maybe empirically know what the rest of us don't seem to grasp? Or just because it's so domain specific? Or... something else entirely?
> +/!10
I personally find the given example harder to read than the equivalent in python/matlab/javascript with lodash/probably many other languages
sum(range(10))
sum(1:10)
With "K", you have to know a few new conventions. In the second case you just have to read, and the data flow is more obvious.
I personally find the given example harder to read than the equivalent in python/matlab/javascript with lodash/probably many other languages
sum(range(10))
sum(1:10)
With "K", you have to know a few new conventions. In the second case you just have to read, and the data flow is more obvious.
I went through a similar transformation when I finally understood LiSP. I knew Python and had programmed in it for several years but it was breaking into Scheme that changed my approach. I wonder if learning new paradigms always changes how you use other languages.
what is `<<` ("ordinal") supposed to be doing? i understand how `<` works, but how is it useful to apply it a second time?
the closest i got to finding some pattern is this:
(i guess "ordinal" really is too ambiguous...)
EDIT
alright, i see it now - `<<xs` is "for each item x of xs, where does x land when you sort xs?
the closest i got to finding some pattern is this:
>>> xs
'34210'
>>> grade(xs)
[4, 3, 2, 0, 1]
>>> grade(grade(xs))
[3, 4, 2, 1, 0] # :O
grade = lambda xs: (
[i for (i,_) in sorted(enumerate(xs), key=swap)]
)
swap = lambda p: (p[1], p[0])
(i guess "ordinal" really is too ambiguous...)
EDIT
alright, i see it now - `<<xs` is "for each item x of xs, where does x land when you sort xs?
People are so quick to reject K and APL-style languages for superficial reasons that they never get to the deep and interesting reasons! I am mostly familiar with APL, but I think the things I appreciate and dislike are about the same in K.
One interesting philosophical difference, at least among some APL programmers, is that building abstractions should be avoided. TFA has a hint of that philosophy, in its suggestion that perhaps naming (the root of abstraction) hinders clarity. I think this is definitely worth considering, and it doesn't really have anything to do with the (supposedly) cryptic syntax.
One reason I don't like K/APL-ish vector programming is performance. I once spent some time working with others on a GPU-targeting APL compiler, and I found that some common programming patterns are quite antithetical to good performance. In particular, the use of "nested arrays" (not the same as multidimensional arrays) induces pointer structures that are difficult to do anything with. Such nested arrays are typically necessary when you want to do the equivalent of a "map" operation that does not apply to each of the scalars at the bottom, but perhaps merely the rows of a matrix. Thus, control flow and data are conflated. This is fine conceptually, but makes it difficult to generate high-performance code.
Another concern is that encoding control flow as data requires a lot of memory traffic (unless you have a Sufficiently Smart Compiler; much smarter than I have ever seen). Consider computing the Mandelbrot set, which is essentially a 'while' for each of a bunch of points. An idiomatic APL implementation will often put the while loop on the outside of the loop over the point array, which means it will be written to memory for every iteration. In other languages, it would be more idiomatic to apply the 'while' loop to each point individually, which will then be able to run entirely in registers. You can also do that in APL, but it is normally not idiomatic (and sometimes awkward) to apply complex scalar functions to array elements.
Just look at this Mandelbrot implementation from Dyalog; they do it in exactly the way I described: https://www.dyalog.com/blog/2014/08/isolated-mandelbrot-set-...
Specifically, the conceptual 'while' loop has been turned into an outer 'for' loop (this is OK because the 'while' loop is always bounded anyway):
One interesting philosophical difference, at least among some APL programmers, is that building abstractions should be avoided. TFA has a hint of that philosophy, in its suggestion that perhaps naming (the root of abstraction) hinders clarity. I think this is definitely worth considering, and it doesn't really have anything to do with the (supposedly) cryptic syntax.
One reason I don't like K/APL-ish vector programming is performance. I once spent some time working with others on a GPU-targeting APL compiler, and I found that some common programming patterns are quite antithetical to good performance. In particular, the use of "nested arrays" (not the same as multidimensional arrays) induces pointer structures that are difficult to do anything with. Such nested arrays are typically necessary when you want to do the equivalent of a "map" operation that does not apply to each of the scalars at the bottom, but perhaps merely the rows of a matrix. Thus, control flow and data are conflated. This is fine conceptually, but makes it difficult to generate high-performance code.
Another concern is that encoding control flow as data requires a lot of memory traffic (unless you have a Sufficiently Smart Compiler; much smarter than I have ever seen). Consider computing the Mandelbrot set, which is essentially a 'while' for each of a bunch of points. An idiomatic APL implementation will often put the while loop on the outside of the loop over the point array, which means it will be written to memory for every iteration. In other languages, it would be more idiomatic to apply the 'while' loop to each point individually, which will then be able to run entirely in registers. You can also do that in APL, but it is normally not idiomatic (and sometimes awkward) to apply complex scalar functions to array elements.
Just look at this Mandelbrot implementation from Dyalog; they do it in exactly the way I described: https://www.dyalog.com/blog/2014/08/isolated-mandelbrot-set-...
Specifically, the conceptual 'while' loop has been turned into an outer 'for' loop (this is OK because the 'while' loop is always bounded anyway):
:For cnt :In 1↓⍳256 ⍝ loop up to 255 times (the size of our color palette)
escaped←4<coord×+coord ⍝ mark those that have escaped the Mandelbrot set
r[escaped/inds]←cnt ⍝ set their index in the color palette
(inds coord)←(~escaped)∘/¨inds coord ⍝ keep those that have not yet escaped
:If 0∊⍴inds ⋄ :Leave ⋄ :EndIf ⍝ anything left to do?
coord←set[inds]+×⍨coord ⍝ the core Mandelbrot computation... z←(z*2)+c
:EndForThis looks cool, seems like it's proprietary and expensive though. Is there anything similar that's free?
Using / for reduction reminds me of the Bird–Meertens formalism.
The problem is also that you encounter K/APL in these circumstance, hardcore math functions or ad hoc queries. I'd say that most languages look their worst in these circumstances, deeply nested for loops for e.g. 3D math in C aren't exactly instantly clear either.
Once you see e.g. input verification or other more "tedious" parts of programs, things get decidedly less cryptic.
And less optimized/concise, of course. A bit like the people who complain about GUI hello world programs taking 10 lines.
Once you see e.g. input verification or other more "tedious" parts of programs, things get decidedly less cryptic.
And less optimized/concise, of course. A bit like the people who complain about GUI hello world programs taking 10 lines.
I have the same question for K aficionados as I have for Forth ones: what real-world, human-usable, important software has been written in it? A GUI framework, a web browser, a window manager, a text editor, etc. Anything?
I'm asking because the article is poking at C-like languages, i.e. implying that K is good for general-purpose use. Yet all I hear about is that K is good at multiplying numbers and matrices, which is a pretty limited playground.
I'm asking because the article is poking at C-like languages, i.e. implying that K is good for general-purpose use. Yet all I hear about is that K is good at multiplying numbers and matrices, which is a pretty limited playground.
How do people do error handling in K? Or is the data typically well-formed in real programs? Not bashing anything, I'm just genuinely curious.
“ and you can add 5 to an entire matrix at once as easily as a single number. Why you would want this property remains elusive-...”. Why would that be elusive I wonder ? MATLAB has been doing it since the 80s as an example and that exact elusive feature would have been used countless times.
I totally buy that this is elegant for small number- or lust-based things, but since a financial database is named as the flagship product: how does I/O look like in K? Both user input and disk access. How do you make a user interface? How do you networking?
Is it still as elegant?
Is it still as elegant?
My problem with K isn't its syntax, it's its culture: People who are so self-satisfied with their ability to read their own code they refuse to give variables meaningful names, aping mathematical tropes they half-understand at best, because those mathematical equations are only tolerable in the context of a paper where the rest of the lifting is done by natural language prose. They eschew comments without realizing that those comments enable terse expressions by showing you have the basic maturity to express yourself in a way others can understand.
Knuth tried to move us to that state with Literate Programming. How many K programmers use Literate Programming? One? Two? Any?
Knuth tried to move us to that state with Literate Programming. How many K programmers use Literate Programming? One? Two? Any?
My hat is off to the people who can handle this kind of thing; I myself cannot.
[deleted]
Some context often missing form these "code golf" showcases:
Domain specific languages have syntax optimized towards specific tasks. This often means very compact symbolic expressions which looks completely inscrutable to outsiders but are highly efficient when you know the language. String processing in Perl, pointer arithmetic in C, selectors in jQuery. Nobody can guess what the code does by looking at it, you have to learn the language. But it is worth the investment when the task in question is often used.
So the question is, are you working in a domain where the investment in learning a particular DSL will pay off?
But this context is completely absent in the article. By comparing the K syntax for "reduce" with a for-loop, it assumed the competition is systems languages like C or Go where a for-loop is idiomatic. But then the question is, how often do you use "reduce"-like operations in this domain anyway? It is really worth optimizing the langauge towards?
If on the other hand the article admitted K is a domain specific language optimized for numerical processing, then a reasonable comparison would be against Numpy or Matlab, perhaps Haskell, not for-loops.
Domain specific languages have syntax optimized towards specific tasks. This often means very compact symbolic expressions which looks completely inscrutable to outsiders but are highly efficient when you know the language. String processing in Perl, pointer arithmetic in C, selectors in jQuery. Nobody can guess what the code does by looking at it, you have to learn the language. But it is worth the investment when the task in question is often used.
So the question is, are you working in a domain where the investment in learning a particular DSL will pay off?
But this context is completely absent in the article. By comparing the K syntax for "reduce" with a for-loop, it assumed the competition is systems languages like C or Go where a for-loop is idiomatic. But then the question is, how often do you use "reduce"-like operations in this domain anyway? It is really worth optimizing the langauge towards?
If on the other hand the article admitted K is a domain specific language optimized for numerical processing, then a reasonable comparison would be against Numpy or Matlab, perhaps Haskell, not for-loops.
author ended article with
> Leaning back in your chair, you think to yourself:
> |/
what does this translate into?
> Leaning back in your chair, you think to yourself:
> |/
what does this translate into?
Seeing this article and the top comment, is this that contentious an issue? I wasn't aware people care that much about Algol vs. APL-ish languages.
https://a.kx.com/a/k/readme.txt
https://web.archive.org/web/20041024065350/http://www.kx.com...
https://web.archive.org/web/20040823040634/http://www.kx.com...
For some 'licence plate' K algorithm dissections, you might like this Twitter account:
https://twitter.com/kcodetweets