top | item 16397482

(no title)

ktamura | 8 years ago

Programmers continue to discover and re-discover APL and array-oriented programming.

If someone is serious about array programming/APL and wants to make a lucrative career in it, study KDB+/Q/K. There's a small but active community around them.

Just to relate an anecdote: I used to be a fairly active KDB+ programmer in finance but left both finance and programming ~8 years ago. Just yesterday, I got an email from my friend who's building a statarb fund around cryptocurrencies. They were going to use KDB+ and offered me a job for "300-500k base with incentives"...

My friends clearly don't know that I now work in sales & marketing :D

Aside: Roger Hui, the creator of J, is a programming sibling of Arthur Whitney, the creator of K/Q and the founder of www.kx.com

discuss

order

nemo1618|8 years ago

I recall there was an article about Arthur Whitney on HN a while back. It contained a small C program that he wrote for a college assignment, which I preserved here: https://gist.github.com/lukechampine/e759098e717e42bc11c5

Looking at this "C" code, it's pretty clear that he was destined to be an APL-style programmer :)

lhorie|8 years ago

> finance

Did you think finance programming industry (specifically around the KDB+/Q/K niche) fostered a "bro" culture (think wolf of wall street)? It always seemed like an interesting field, but every once in a while I hear discouraging stories. Curious about your experience with it.

branchless|8 years ago

I think kdb / q developers are possibly as far from wolf of wall street as it's possible to be.

One guy in London who worked in kdb was a huge arse, the tens of others I've encountered have all been lovely. It's an insane little language, though I'm only a basic user.

qznc|8 years ago

I understand the advantages of array-oriented programming, but why the J/APL syntax? There are plenty of column-oriented databases available. https://en.wikipedia.org/wiki/List_of_column-oriented_DBMSes

Also related is range-based programming. https://wiki.dlang.org/Component_programming_with_ranges

    return
        // Start by generating all dates for the given year
        datesInYear(year)
        // Group them by month
        .byMonth()
        // Group the months into horizontal rows
        .chunks(monthsPerRow)
        // Format each row
        .map!(r =>
                // By formatting each month
                r.formatMonths()
                 // Storing each month's formatting in a row buffer
                 .array()
                 // Horizontally pasting each respective month's lines together
                 .pasteBlocks(colSpacing)
                 .join("\n"))
        // Insert a blank line between each row
        .join("\n\n");
Much more readable than something like "(~R∊R∘.×R)/R←1↓ιR".

ken|8 years ago

Is it more readable? I'm even less familiar with D than APL, and that D code only looks more readable to me right now because it's mostly comments.

Wouldn't adding 8 lines of comments help newcomers to APL, too? This seems like a rather unfair contest.

Familiarity counts for a lot. 25 years ago when my primary language was 6502 assembly, I would have said 20 pages of assembly code look more readable than either of these.

In the long run, though, in every field I've worked in, being able to turn 20 lines of code into 20 characters has turned out to be a huge benefit. Once you familiarize yourself with the vocabulary, you can operate at a higher level. Nobody fears concise names and symbols when it's in a context they understand. I think even the most APL-phobic would admit that "a←b+c" is more readable than "assign(a,sum(b,c))".

Avshalom|8 years ago

okay but in J thats:

  require 'dates general/misc/format'
  4 3$ calendar 2018
4 3$ just re$hapes it from a big as line to a 4x3 calendar. If you wanted to print it to a file you'd want to convert it to a string of course so:

  ": 4 3$calendar 2018

michaelcampbell|8 years ago

Everything you know how to read, and peppered with comments you know how to read is more readable than everything you don't, without such helper devices.

ldarcyftw|8 years ago

What’s the catch though? If learning KDB+/Q/K nets you a nice comp why not everyone and their dog are learning it yet?

beagle3|8 years ago

You don't just have to learn it (that's the easy part); you actually have to grok it, and that's the hard part.

The piece of code |/0(0|+)\ (all 9 characters of it) efficiently computes the maximum subarray sum[0] of an array using Kadane's algorithm. You'll either have to learn K or trust me on that.

While it is possible to break it down to multiple parts and document each, it is idiomatic to just use it and document the whole line. Or not document it at all, because experienced K people know that already. Why call a function "average" (7 chars) when an implementation (+/x)%#x is only 6? Furthermore, you know from reading it what the average of a zero length list is (NaN). Do you know what a function called "average" would return in this case?

I would say the answer to your question is that K is very different than other programming languages, and requires a different mindset. Somehow, attempts to give it a more mainstream face (e.g. article author's "Kerf" project) do not seem to take off.

[0] https://en.wikipedia.org/wiki/Maximum_subarray_problem

lucozade|8 years ago

I don't know either party but I'd hazard a guess that they're not really offering 500k for a K developer.

They're more likely offering 500k for someone who has a lot of experience managing financial tick data in a KDB environment.

The catch will be that not everyone who has read the KDB for Dummies book (OK, I made that up) will be able to walk into a stat arb hedge fund and add 500k of value.

vram22|8 years ago

Yes, I was wondering the same thing. Some guesses: some people may not be comfortable with the somewhat un-conventional syntax / semantics (e.g. no operator precedence, just right to left or as they call it, left of right :) (I'm fine with it myself), or might think you can only get jobs using kdb+/q/k in big cities like New York / London, and may not want to move there. Possibly low market demand (compared to mainstream languages), although high comp, may be another reason. Also people may think you need finance knowledge too, and may not have it. Those are some guesses, that's all. Interested to hear if anyone else has some ideas on this.

Edit: Just saw beagle3's comment after posting mine. Some interesting points there. Still wonder about the non-tech factors though, as listed in above part of my comment.