WingNews logo WingNews
top | item 41145069

(no title)

Jorge1o1 | 1 year ago

Their pykx integration is going a long way to fix some of the gaps in:

- charting

- machine learning/statsmodels

- html processing/webscrapes

Because for example you can just open a Jupyter Notebook and do:

  import pykx as kx
  df = kx.q(“select from foo where bar”)
  plt.plot(df[“x”], df[“y”])
It’s truly an incredibly seamless and powerful integration. You get the best of both worlds and it may be the saving feature of the product in the next 10 years

discuss

order

nivertech|1 year ago

I think this will only work with regular qSQL on a specific database node, i.e. RDB, IDB, HDB[1]. It will be much harder for a mortal Python developer to use Functional qSQL[2] which will join/merge/aggregate data from all these nodes. The join/merge/aggregation is usually application-specific and done on some kind of gateway node(s). Querying each of them is slightly different, with different keys and secondary indices, and requires using a parse tree (AST) of a query.

---

[1] RDB - RAM DB (recent in-memory data), IDB (Intraday DB - recent data which doesn't fit into RAM), HDB - Historical DB (usually partitioned by date or other time-based or integral column).

[2] https://code.kx.com/q/basics/funsql/

Jorge1o1|1 year ago

That’s accurate enough. I think the workflow was more built for a q dev occasionally dipping into python rather than the other way around.

I think you touch on something really interesting which is the kink in the kdb+ learning curve when you go from really simple functions,tables, etc. to actually building a performant kdb architecture.