[ktamura]

I used to use KX/kdb/Q/K daily for several years. I wrote a full implementation of reinforcement learning (15 lines), a lightweight MVC framework (to show reports and tables in an internal webapp) and even a Q syntax checker (abusing table as a data structure to hold parse trees). Good or bad, for the longest time, Q was my "go-to" programming language.

Based on that experience...

1) Yes, but that's not huge by modern standard.

2) Q is a DSL version of K. As others have commented, K is a pretty clean implementation of APL, and Q makes K more approachable.

3) I have to agree here, but Q for Mortals makes up for it.

4) It is really fast. As we all know, a vast majority of us actually don't have terabytes and terabytes of data, especially after a reasonably cleanup / ETL / applying common sense. I suppose it helped that I worked in finance, which meant my desktop had 16GB of memory in 2009 and 128GB of memory on a server shared by 4-5 traders.

Finally, Q was never intended for general-purpose computing nor a widespread adoption. At least when I was an active user, the mailing list had the same 20-30 people asking questions and 3-4 people answering them, including [e-mail] (= Arthur Whitney, the creator). Back then, I'd say there were at most 2-3k active users of Q/K in the world. Now that Kx Systems is part of First Derivative and has been working on expanding their customer base, perhaps they have more...?

[hpcjoe]

It is worth pointing out that really fast is ... well ... really fast. See [1] for some benchmarks they did for small, medium, large data sets.

The machines that $dayjob-1 used to build dominated the STAC-M3 for a few years (2013-2015) because we paid careful attention to how kdb liked to work, and how users liked to structure their shards. Our IO engine was built to handle that exceptionally well, so, not only did in-memory operations roar, the out of memory streaming from disk ops positively screamed on our units (and whimpered on others).

I miss those days to some degree. Was kind of fun to have a set of insanely fast boxen to work with.

[1] http://kparc.com/q4/readme.txt

[nkurz]

1) Yes, but that's not huge by modern standard.

OP could have phrased it better, but I presume his point was that 500KB is extremely small by modern standards. The whole executable fits comfortably in L3, so you'll probably never have a full cache miss for instructions. On the other hand, while it's cool that it's small, I'm not sure that binary size is a good proxy for performance. Instruction cache misses are rarely going to be a limiting factor.

[beagle3]

> Instruction cache misses are rarely going to be a limiting factor.

k's performance is a combination of a lot of small things, each one independently doesn't seem to be that meaningful. And yet, the combination screams.

The main interpreter core, for example, used to be <16K code and fit entirely within the I-cache; that means bytecode dispatch was essentially never re-fetched or re-decoded to micro instructions, and all the speculative execution predictors have a super high hit rate.

When Python switched the interpreter loop from a switch to a threaded one, for example, they got ~20% speedup[0]; I wouldn't be surprised if the fitting entirely within the I-cache (which K did and Python didn't at the time) gives another 20% speedup.

[0] https://bugs.python.org/issue4753

[oskarth]

Would be really interesting to read a write up on your experience. What do you program in now? How do you look at other PLs now? What do you miss and what are you happy "just works"? What do you think other PLs (especially languages like Lisp, which are very high in terseness) can learn from Q?

[avmich]

I would compare Q (and other APL-related languages) to Vim editor. There you have some carefully chosen operations which are easy to perform. They don't take much efforts. They are also easy to compose in useful ways - because the corresponding properties support that. Since the basis of editing operations is fairly large, you have many operations; but when you know many of them, you can work powerful edits.

Lisp on the other hand is more like Emacs - naturally. Here we have a small, carefully chosen orthogonal basis of abstract operations - not domain-specific, but "theoretically-foundational" small basis. Then you have a library of macros on top of that, and ability, of course, to extend.

In other words, basis for APL is "classical" math, made executable and expanded with mechanisms required to put in one line programming constructs (logic, control flow, ordering...). It's harder to expand, but you don't often need that. Lisp is a specific branch of math, lambda calculus, which is provably enough to solve arbitrary programming problem. The "inner core" of Lisp is also hard to expand, but what you expand for your task is "the usage" of the language, which is made to be straightforwardly expandable.

To me it's hard to say what is better.

[ghosthamlet]

Have you open source the reinforcement learning and MVC framework code? Can't wait to study it.