top | item 22504106

Stages of denial in encountering K

439 points| kick | 6 years ago |nsl.com | reply

422 comments

order
[+] chrispsn|6 years ago|reply
From Arthur:

    design goal: speed
    write fast, run fast, modify fast.
    minimize product of vocabulary and program size.
https://a.kx.com/a/k/readme.txt

    comprehensive programming environments 
    (programming, files, comms, procs, ...)
    are depressingly complex.
    commonlisp, c/lib/syscalls, java, winapi
    are all in the 1000's of entry points.
    many with multiple complex parameters.
    with k we try to do more with less
    in order to write correct programs faster.
https://web.archive.org/web/20041024065350/http://www.kx.com...

    > Just curious, but does the state of California charge you a large tax
    > for each character in a variable name?
    
    yes.
    
    we also try to fit our algorithms onto licence plates.
https://web.archive.org/web/20040823040634/http://www.kx.com...

For some 'licence plate' K algorithm dissections, you might like this Twitter account:

https://twitter.com/kcodetweets

[+] kstenerud|6 years ago|reply
The readme examples remind me of Perl code. Example:

    print<>!~($/=$")+2*map 1..$s{$_%$'}++,<>
[+] lonelappde|6 years ago|reply
Don't use indentation for blockquotes. HN rendered indented lines as code, which is unreadable on mobile.
[+] snicker7|6 years ago|reply
I use to program financial applications in K/Q for a stint. Very elegant and orthogonal language. And very well documented. One major issue I had with it and other APL-ish languages: lack of skimmability.

I can easily skim thousands of lines of Perl and Java in seconds and have a rough idea of what the code does. Reading K, however, requires careful and deliberate attention to every single character. I found it useful to maintain documentation on how to /read/ the code (almost a literate programming approach).

[+] BiteCode_dev|6 years ago|reply
That's bad in my book.

I write a line once for a 100 times I read it.

[+] oddthink|6 years ago|reply
Yeah, I agree. I wrote some mortgage prepayment models in q for a while, but didn't dip into K itself. Q is fantastic as a database query language (assuming you have time-series databases and are mostly doing fairly simple operations.) Implementing a full model in it is fine, but can get hard to maintain.

Also it has (or had, maybe they fixed it) a bunch of weird quirks, like "sum foo" skipping all the NAs in a list while "+/foo" didn't, resulting in NaN. (Or maybe it was +/ over an int array didn't work while a double array did. I forget.) Arrays of dicts were immediately turned into tables, which didn't work the same as an array of dicts, etc.

But, yeah, it's not crazy implementing things in q/K, as long as they're mostly linear things. Trees are hard, iterative methods are hard, and it's really hard to skim code, other than to just rely on names and comments.

[+] jes5199|6 years ago|reply
the thing that's odd to me is that you could write this exact same post about other languages - a person who learns Haskell ends up writing that exact same range/reduce/plus statement. Modern lisps like Racket or Clojure take you there. Forth programmers and Joy programmers already noticed that their two languages had converged: and they'd write basically exactly this, too. Really, it's only the "mainstream" languages that are different!
[+] TurboHaskal|6 years ago|reply
A Common Lisp programmer would write it like that too. He would then disassemble the function, run away in terror and eventually come back to rewrite the whole thing with a more efficient loop expression.
[+] oblio|6 years ago|reply
That's an interesting take on things. Another one would be that ~40-50 million programmers do things one way and ~10000 (totally random guess) do it another way :-)
[+] kick|6 years ago|reply
This was written by 'RodgerTheGreat, whose comments and submissions on this site are a treasure trove.
[+] nurettin|6 years ago|reply
his eerie mind reading tone is entertaining and almost entirely accurate
[+] saagarjha|6 years ago|reply
Look, I don't hate K. It's probably a pretty good language for the task it seems to have been designed for, which appears to be numerical computing. Being able to fit an entire code on a screen is an interesting concept; while I'm not completely convinced it should be a goal in and of itself it's certainly something I can't rule out as a possible productivity booster.

However, I still find it problematic, and it's not really about the language itself: it's more about how it's promoted in the article, by the people in this thread, and in general. Specifically, the claims it's perfectly readable, and anyone who says anything otherwise is either stupid, lying, or too poor to understand it are not useful. (Yes, you know who you are for that last one. You're not helping your case.) The fact of the matter is that most programmers are used to ALGOL-esque syntax, and there is significant value in being able to lift concepts and interoperate between those languages at a superficial level. Every "inspired" languages hits these issues, depending on how "strange" it is: Objective-C is ugly, Lisp is alien, and on the far end APL derivatives are simply labelled as unreadable. That doesn't mean they're bad languages, but to those promoting it: drop the attitude, and understand why we have reservations. Your language isn't the "true" way to program, that only the chosen elect can understand and lecture us on. Don't ignore or gloss over languages that have similar features to yours. We want to hear what the interesting things are that you can do in the language, actual guidance for how we can adapt to the language (not: "one-letter identifiers are just what we do; deal with it"). Thanks.

[+] yiyus|6 years ago|reply
I love small languages (lisps, forths, apls) and think that there is much to learn from them, but you need an open mind. You need to start from zero and totally forget your ALGOL-mindset while you are learning the new paradigm.

It's like learning Chinese. Everyone learns it in China, and everyone there will tell you that it's perfectly readable and you don't need to be extraordinarily intelligent to speak it. But if you expect to figure it out based on your knowledge of English, it will be terribly frustrating. This does not mean that German or Dutch (which share roots with English) are better languages, or even easier to learn. Do not expect your Chinese teacher to give you a different answer from "these weird symbols are just what we do; deal with it".

This article is quite good. Clearly the author understand your reservations and he is saying: give it a try and it will click. The learning material might be better, but it's there (start with Iverson and work from that). It's much easier than learning Chinese!

[+] chrispsn|6 years ago|reply
I started with Python and still miss its lazy generators. K's grain favours calculations that drive down long 'straights' of arrays without many conditional turns, but it's eager-first, which can be a drag for some problems (eg making a prime number generator object that I can take an arbitrary number of elements from lazily). Still, it hasn't been as bad as I feared.

I like the choices of primitives in K and the novel ways they can be combined to do things more efficiently (and with fewer pieces!) than you may expect. Good examples are 'where' (&), the grade twins (< and >), scan (\), and eachprior (':).

I love the adverb syntax: each (') lets you map a function over a data structure with zero syntactic effort, and 'over' (/) and 'scan' (\) make reductions trivial.

With regards to an on-ramp, I started with John Earnest's oK REPL and did a bunch of Project Euler and Advent of Code problems. I then moved to trying to understand the examples on kparc.com. I then converted a spreadsheet application from JS to k7. In all of this, John's documentation was essential.

For me, having single variable names isn't paramount, but in some cases - particularly functions - I can see it helps to 'drive' your mind towards finding simpler ways to do things. It lets the characters that do the 'real' work come to the forefront.

[+] AnimalMuppet|6 years ago|reply
If you're going to use a language (call it X; we're not just talking about K), then you really have to know X's syntax and primitives. There's just no other way.

You also would be much better off learning X's idioms. These are standard ways of saying something. They're standard for a reason - because they work, and because everybody understands them. If you can't read them, then a lot of code is going to be difficult to understand.

That's just table stakes for using X. If you want to use X, but don't want to learn the syntax and the primitives, then you don't really want to use X. You may want the benefits that you think X would get you, but you don't actually want X.

Now for the other side: If you're a proponent of X, you need to remember that many languages are magic within certain domains, and anywhere from "meh" to horrible outside it. Take K, for instance. I'm pretty sure it wouldn't be a good fit for most embedded systems. I'm fairly sure it wouldn't be a good fit for web programming. I'm not sure I'd want to do text processing in K. So it's not "the one way" (except possibly within certain domains). It's not "the way of the future" (except possibly within certain domains). It's not "what all the smart and enlightened people are using" (except possibly within certain domains). And, unless we're working in those certain domains, it really isn't all that important whether the rest of us learn it or understand it.

[+] geocar|6 years ago|reply
> the claims it's perfectly readable, and anyone who says anything otherwise is either stupid, lying, or too poor to understand it are not useful.

Yes: It is perfectly readable once you know how, and anyone who says anything otherwise is just plain wrong. Including you 58 days ago.

I don't know if you are wrong because you are lying, or you believe this because you are stupid. I wrote off at the time that it was well-meaning ignorance, but it doesn't really matter: The key thing, that is, the thing holding you back from reading it, is you.

I believe you can learn to read it and read it as well as me as quickly as with a few months of study. If you think anyone else is telling you otherwise, ignore them.

I don't believe you need to not be "poor" to do it - everything you need is available to you.

It's all about whether you want to do this or not.

> Every "inspired" languages hits these issues, depending on how "strange" it is: Objective-C is ugly, Lisp is alien, and on the far end APL derivatives are simply labelled as unreadable. That doesn't mean they're bad languages, but to those promoting it: drop the attitude, and understand why we have reservations.

We understand because we were once you: Every X-evangelist you're talking to (where X is Lisp, or Objective-C, or in this case K) was once like you in at least the way of not knowing X, so we know you are wrong in a way (as we now know X) that you simply cannot comprehend yet. Think about that. Maybe re-read Paul Graham's blub article and appreciate really what it means to be the blub programmer.

Many K programmers have some familiarity with APL. Some (like myself) have used Objective-C and Common Lisp (I've got over a decade of professional history in CL), and loads of other languages. Until you learn K, all you have is our testimony; our gospel that this is worth learning. You can literally do anything in blub, being able to do it easier is the point.

[+] neo2006|6 years ago|reply
Moreover if I need to keep in my head the corresponding concepts for all of this symbols, it's like learning a new natural language. We sometimes forget that one of the most important feature programming f language is to unburden the programmer from the language it self and get him to focus on the concepts. I also don't see the point of optimizing on the number of line of code if the complexity per character is increased (did I just invented a kpi here?)
[+] agumonkey|6 years ago|reply
I think there are two worlds.

The one where shallow and social compatibility is seen as important.

The one where depth and seemingly looking sharp concepts is important.

If you care about your devs only be able to read some business logic then good, syntax familiarity is great. If you care about advanced problem solving, this is unimportant. And people prefer to iterate concise formulas presenting powerful concepts rather than instance-level readability (as in overlyVerboseNamingSchemes)

[+] Roboprog|6 years ago|reply
Putting a space between discrete symbols would have helped the first example considerably

    + / ! 1000
[+] kortex|6 years ago|reply
> Specifically, the claims it's perfectly readable, and anyone who says anything otherwise is either stupid, lying, or too poor to understand it are not useful.

+1. All of these "compact" languages (I'd even extend it to Perl to some extent, the way some write it), are seriously lacking in the ergonomics department. Sure, J Leet Hacker might have no problem parsing pages of the stuff. What about the coworker with dyslexia, ADHD, vision problem, etc?

Even if you aren't, what are the real gains? It seems resistant to tabcomplete, git diffs are probably near-inscrutable, grepping for snippets is probably an exercise in judicious application of backslashes... it's cute, but I don't see where this holier-than-thou is coming from (not that ANY language justifies that attitude)

Edit: oops, touched a nerve, I see. I'll add to the above the 7 +/-2 rule [0]. Sure, lets say even you COULD write a program in 1/100th or 1/1000th of the lines in k. I don't WANT a source code that looks like that. Way too much information density.

I'll give them this: perhaps the criticism is that ALGOLikes are too sparse, could stand to be denser, and k is the manifestation of the far other end of the spectrum.

- https://en.m.wikipedia.org/wiki/The_Magical_Number_Seven,_Pl...

[+] caymanjim|6 years ago|reply
Not mentioned in the article is that the K programming language is used for kdb+, which is a fast in-memory time-series database that's heavily used in the finance industry for securities trading systems. I've worked with it a tiny bit, but the pros are all Dark Wizards.
[+] sriram_malhar|6 years ago|reply
I love the brevity of regular expressions and use them on a daily basis. It is the same argument that keeps me returning to K: the syntax is terse and compact, the semantics are simple and composable, and your eyes get used to it.

Beyond a point however, I cannot read my own regex's after a month's absence. Which is why I use perl's /x modifier extensively to split up regex components onto multiple lines and to document them thoroughly, even if they are for throwaway scripts, because I don't always throw them away!.

For example:

    $_ =~ m/^                         # anchor at beginning of line
            The\ quick\ (\w+)\ fox    # fox adjective
            \ (\w+)\ over             # fox action verb
            \ the\ (\w+) dog          # dog adjective
            (?:                       # whitespace-trimmed  comment:
              \s* \# \s*              #   whitespace and comment token
              (.*?)                   #   captured comment text; non-greedy!
              \s*                     #   any trailing whitespace
            )?                        # this is all optional
            $                         # end of line anchor
           /x;                        # allow whitespace
(source: https://www.perl.com/pub/2004/01/16/regexps.html/)

This is where K fails me. It may not be a fault of the language, but everyone in the community has bought into this strange idiomatic style. I can't imagine debugging it, or checking it for correctness, or foisting it on a less experienced developer. Here's a canonical example an xml parser, on their website.

https://a.kx.com/a/k/examples/xml.k

Where's the pedagogy? Where are the comments? Why is this line noise considered acceptable?

[+] carapace|6 years ago|reply
This article should end with something like, "...and then you find out about arcfide's Co-dfns compiler..." https://github.com/Co-dfns/Co-dfns

You don't have to like it, but the existence of K, et. al. shows that most of us are wasting a huge amount of time and energy using bad tools. These things are so powerful and not that hard to learn.

[+] dang|6 years ago|reply
A million-line program isn't readable by anybody, no matter how readable the language is. If the equivalent program can be written in, say, a thousand lines in some more concise language, that's more than worth the learning curve, even if the language is strange and off-putting.
[+] i_don_t_know|6 years ago|reply
How does error handling work in K? For example, what happens when you try to read a file that doesn’t exist? How does the read function indicate failure to the caller? How does the caller check and recover (show an error to the user and continue with some default values)?
[+] heinrichhartman|6 years ago|reply
Looks like the author figured it all out at the very beginning:

> If each punctuation character in the K is a separate part of speech, and you reverse their order

     +      /     !    100
     plus reduce range 100

This seems to be how this works. Compare that to math equations like these ones:

https://en.wikipedia.org/wiki/Maxwell%27s_equations#Formulat...

You would not expect them to read them like a paragraph. You are expect to read them symbol by symbol and unpack from the inner out.

K seems to be a dense language. Expect to read it symbol by symbol, not word by word or line by line.

If will be come more readable if you (1) slow down (2) get used to the programming patterns.

Not saying K is a good language or anything. But the frustration of the author seems to have more to do with impaticence and mismatched expectations than with flaws in the language itself.

[+] aw1621107|6 years ago|reply
I've seen a few posts here and there over the years about array languages, but I can't recall any (or any discussion in their comments) that have mentioned any disadvantages beyond more human factors.

So for the k/q/APL/array language gurus out there: in what situations would I not want to use k or another array language, if the relative obscurity of the syntax and mental model were not a factor (e.g., assuming you, your team (if any), and anyone who would ever look at your code would be very familiar with the language and idioms)?

I may not want to choose Go or Java because of latency concerns, or Rust because of compilation speed/iteration time concerns, or C/C++ because of safety issues, or Python/Ruby/Javascript because of performance issues. Why would I not want to pick k?

[+] rubyn00bie|6 years ago|reply
Only because earlier today I so definitely thought about the whole "code is a liability (debt)" reality we all live in... that is to say, I'm spit-balling here:

but for once K makes me go "oh shit!" and not because of the craziness of its syntax... rather, if that syntax is so precise/terse, and even if to the outsider it looks fucking insane, does it have a wicked lower TCO because you literally have insanely less lines of liability in your code, libraries, dependencies, etc?

Is that the reason finance uses it, they maybe empirically know what the rest of us don't seem to grasp? Or just because it's so domain specific? Or... something else entirely?

[+] geocar|6 years ago|reply
> does it have a wicked lower TCO because you literally have insanely less lines of liability in your code, libraries, dependencies, etc?

Yes. q (a language and distribution of k4) has a built-in webserver with websockets, messaging, all types can be serialised over IPC (except pointers to C code) and most data structures have an efficient on-disk representation.

You might've recalled a few days ago someone announced a specialised database for doing "have-i-been-pwned" lookups in 49µsec. I wrote[1] one line of q that is about 10x faster:

[1]: https://news.ycombinator.com/item?id=22467866

It's just a single operation, and it's built-in, so maybe you think Python could just have mmap-efficient variables and a bin operator and you'd be there, but those things don't exist in Python. If I had the same problem as the author, it certainly wouldn't materialise as "custom database" of any kind, because I'd already be done.

In fact, one of the biggest challenges with "big data" is choosing an efficient representation. The author spent a lot of time on this, but my language just had one built-in. How much time? What's the target? I mean, if I've already paid for q it's clearly a no-brainer, but q isn't free either (even if you get it for no cost: you still need to learn it!)

> Is that the reason finance uses it, they maybe empirically know what the rest of us don't seem to grasp? Or just because it's so domain specific? Or... something else entirely?

Back in the day, k was the only thing you could put tick data into. Now there's lots of systems, and computers are a lot faster, so quants can use Python. Their code doesn't need to be "fast", but "fast enough".

My code still needs to be fast though.

[+] scottlocklin|6 years ago|reply
Finance uses it because it's great for sorting out medium-big time oriented data. The IQ test aspect of it is probably considered a bonus in some shops.

There are non-terse APLs out there; the most obvious one floating around is Nial[1]. Culturally though, once you've figure out how to read and write in the compressed ascii runes, it's kind of hard to go back to the other way. J is actually denser because of the hooks, forks[2], rank and full tacit aspect of things. Personally whenever I've composed an algorithm in J I feel like I really understand it in a way that pretty much nothing else gives me with all the extra typing.

[1] https://github.com/danlm/qnial7

[2] https://crypto.stanford.edu/~blynn/c/apl.html

[+] hmry|6 years ago|reply
The "amount" of code is the same as in any functional language though. It's just written in a smaller text file.
[+] duchenne|6 years ago|reply
> +/!10

I personally find the given example harder to read than the equivalent in python/matlab/javascript with lodash/probably many other languages

sum(range(10))

sum(1:10)

With "K", you have to know a few new conventions. In the second case you just have to read, and the data flow is more obvious.

[+] nitely|6 years ago|reply
You still need to know that range starts at 0 and the end is not inclusive. Also, the second example is not the same as the first one, unless the second is non-inclusive.
[+] dang|6 years ago|reply
Yes, but as you multiply that difference 100 or 1000, the advantages of the concise notation become obvious.
[+] keymone|6 years ago|reply
As I understand author’s point, your example with “sum” is only valid because “sum” word has very few meanings, especially in software engineering world.

Even the word “product” would introduce certain ambiguity in context of real world codebase: is it “math” product or some “domain” product?

It’s even worse with most other words.

So if in any given codebase you still have to know the specific meaning of every word, how is it different from having to know specific meaning of every K symbol?

[+] Beltiras|6 years ago|reply
I went through a similar transformation when I finally understood LiSP. I knew Python and had programmed in it for several years but it was breaking into Scheme that changed my approach. I wonder if learning new paradigms always changes how you use other languages.
[+] uryga|6 years ago|reply
what is `<<` ("ordinal") supposed to be doing? i understand how `<` works, but how is it useful to apply it a second time?

the closest i got to finding some pattern is this:

  >>> xs
  '34210'
  >>> grade(xs)
  [4, 3, 2, 0, 1]
  >>> grade(grade(xs))
  [3, 4, 2, 1, 0] # :O
where

  grade = lambda xs: (
     [i for (i,_) in sorted(enumerate(xs), key=swap)]
  )
  swap = lambda p: (p[1], p[0])
no clue what it means though.

(i guess "ordinal" really is too ambiguous...)

EDIT

alright, i see it now - `<<xs` is "for each item x of xs, where does x land when you sort xs?

[+] icen|6 years ago|reply
< grades upwards: for each element, the element of the returned array is its position if the array is sorted.

    x: 10?A:"abcdefghijklmnopqrstuvwxyz"
     x
    "ysyselacwl"
     +`x`y`z!(x;<x;<<x)
    x   y z
    --- - -
    "y" 6 8
    "s" 7 5
    "y" 4 9
    "s" 5 6
    "e" 9 2
    "l" 1 3
    "a" 3 0
    "c" 8 1
    "w" 0 7
    "l" 2 4
So <x gives a sort index, <<x tells you how far each element is from the minimum in that sort index
[+] uryga|6 years ago|reply
one cool explanation i found is that grading a permutation list effectively gives you its inverse:

  xs = "cab"

  <xs = [1,2,0] =
  {
    0 → 1
    1 → 2
    2 → 0
  }
  
  <<xs = [2,0,1] =
  {
    0 → 2
    1 → 0
    2 → 1
  }
which is exactly what "ordinal" is supposed to do! it kinda makes sense - grading the indexes given by `<xs` gives a permutation list that maps each index back to its actual position (i.e. [1,2,0]==>[0,1,2]). or at least it seems that way if you stare long enough, would be good if someone could confirm!
[+] Athas|6 years ago|reply
People are so quick to reject K and APL-style languages for superficial reasons that they never get to the deep and interesting reasons! I am mostly familiar with APL, but I think the things I appreciate and dislike are about the same in K.

One interesting philosophical difference, at least among some APL programmers, is that building abstractions should be avoided. TFA has a hint of that philosophy, in its suggestion that perhaps naming (the root of abstraction) hinders clarity. I think this is definitely worth considering, and it doesn't really have anything to do with the (supposedly) cryptic syntax.

One reason I don't like K/APL-ish vector programming is performance. I once spent some time working with others on a GPU-targeting APL compiler, and I found that some common programming patterns are quite antithetical to good performance. In particular, the use of "nested arrays" (not the same as multidimensional arrays) induces pointer structures that are difficult to do anything with. Such nested arrays are typically necessary when you want to do the equivalent of a "map" operation that does not apply to each of the scalars at the bottom, but perhaps merely the rows of a matrix. Thus, control flow and data are conflated. This is fine conceptually, but makes it difficult to generate high-performance code.

Another concern is that encoding control flow as data requires a lot of memory traffic (unless you have a Sufficiently Smart Compiler; much smarter than I have ever seen). Consider computing the Mandelbrot set, which is essentially a 'while' for each of a bunch of points. An idiomatic APL implementation will often put the while loop on the outside of the loop over the point array, which means it will be written to memory for every iteration. In other languages, it would be more idiomatic to apply the 'while' loop to each point individually, which will then be able to run entirely in registers. You can also do that in APL, but it is normally not idiomatic (and sometimes awkward) to apply complex scalar functions to array elements.

Just look at this Mandelbrot implementation from Dyalog; they do it in exactly the way I described: https://www.dyalog.com/blog/2014/08/isolated-mandelbrot-set-...

Specifically, the conceptual 'while' loop has been turned into an outer 'for' loop (this is OK because the 'while' loop is always bounded anyway):

       :For cnt :In 1↓⍳256                      ⍝ loop up to 255 times (the size of our color palette)
           escaped←4<coord×+coord               ⍝ mark those that have escaped the Mandelbrot set
           r[escaped/inds]←cnt                  ⍝ set their index in the color palette
           (inds coord)←(~escaped)∘/¨inds coord ⍝ keep those that have not yet escaped
           :If 0∊⍴inds ⋄ :Leave ⋄ :EndIf        ⍝ anything left to do?
           coord←set[inds]+×⍨coord              ⍝ the core Mandelbrot computation... z←(z*2)+c
       :EndFor
[+] jstimpfle|6 years ago|reply
> One interesting philosophical difference, at least among some APL programmers, building abstractions should be avoided

Humans need abstractions to think. What programmers need are abstractions that are not black boxes. I.e. can be opened.

[+] earenndil|6 years ago|reply
What is TFA?

> the use of "nested arrays" (not the same as multidimensional arrays) induces pointer structures that are difficult to do anything with

J doesn't allow nested arrays by default; if you want to create such a structure, you have to use boxes (and then unbox the values inside of them to get at them), which makes it explicit and reduces its usage. AFAIK, k doesn't have nested arrays at all.

[+] smadge|6 years ago|reply
Using / for reduction reminds me of the Bird–Meertens formalism.
[+] mhd|6 years ago|reply
The problem is also that you encounter K/APL in these circumstance, hardcore math functions or ad hoc queries. I'd say that most languages look their worst in these circumstances, deeply nested for loops for e.g. 3D math in C aren't exactly instantly clear either.

Once you see e.g. input verification or other more "tedious" parts of programs, things get decidedly less cryptic.

And less optimized/concise, of course. A bit like the people who complain about GUI hello world programs taking 10 lines.