Show HN: Data Painter – A Different Way to Interact with Your Data

[loa_observer]

Data Painter is RATH's new feature to interact with your data. It helps you to specify complex visual patterns and find explanations and potential causals.

It allows you to do on-flight data labeling, cleaning or even create new features does not exited in original dataset. Everything can be done with a brush tool(painter), You can even play with your data with your fingers on mobile.

RATH is an open-source alternative to Tableau, but with more automations. Feedbacks and suggestions are appreciated.

RATH's Github: https://github.com/Kanaries/Rath

RATH's HomePage: https://kanaries.net/

[untangle]

Interesting product. I first encountered this "data painter" paradigm in the 80's in a product called DataDesk. That software was created by Cornell professor Paul Velleman. Paul was a student of John Tukey and credits his influence on the product.

https://datadescription.com/

[gbasin]

Wow I would have killed for this in my algo trading days :) Great looking tool

[Yoric]

Children, this is why you to have be very careful when you hire someone for algo trading. Some of these people are killers.

[mighty_donkey]

Yessss this is really great! I have a bunch of time-series for climate and remote sensing data sources I could use for this

[albert_e]

I am hearing about this tool first time and the git repo examples look impressive.

Is this free and open-source in the sense that we can host it on our own infra with no license fees? Is there any guide or example for deploying this in a AWS environment with typical data sources like S3 data lake (CSV/Parquet/Delta) and Redshift?

[loa_observer]

RATH is now under AGPL license, we are considering move to a less limited license like apache if there are more open-source users and developer in its community.

The related docs will be published soon, we are working on connecting RATH to more infra now, you can join the discord to the latest discussions. https://discord.com/invite/Z4ngFWXz2U

[Hakkin]

I was really impressed that it seems to be doing everything locally in the browser, but I see once you start trying to do more complex visualizations it POSTs the dataset to the cloud. Still neat.

[loa_observer]

I tested 1 million rows * 12 columns. It works fine with pure client's web-workers and indexedDB. For larger datasets(like CSV file > 150MB), especially for the "mega automation" feature, It turns to be slow and sometimes lead crash in browsers.

For now, I have some temporally solutions, setting a sample size is the easiest.

Another way is put some heavy computation tasks to scaleable OLAP services, like a clickhouse clusters.

I will continue to research more ways to make the client handle as much data as possible and fast enough so more people can deploy it by their own with few costs.

[boredemployee]

Dope tool, will try it exhaustively!

One thing tho, when I try the causal analysis feature, it shows in japanese to me (github show all in english tho) is there a knob to turn it to english?

[loa_observer]

If you are on Kanaries homepage (kanaries.net), the language setting is on the top right corner near login button. The language switch button icon looks like this one: https://www.flaticon.com/free-icon/language_4509733

If you are on RATH product page (rath.kanaries.net), the language setting is on the left bottom of the page.

[dr_kiszonka]

Nicely done!

Tangent: I am looking for a different "data painter," i.e., a simple GUI where I can draw data points and have it generate a table or CSV file with them.

[e12e]

I found this... Something like what you have in mind? (not Foss)

https://plotdigitizer.com/

Ed: or maybe this (gpl) - didn't realize this existed... https://markummitchell.github.io/engauge-digitizer/

Ed2: another along similar lines (pun not intended). Looks like most assume you have a bitmap of a graph, rather than let you just draw datapoints.

https://automeris.io/WebPlotDigitizer/

[loa_observer]

I thought about "draw to create data" before, but it brings problems when your data contains high dimensions (lots of columns). Painted data only defines limited number of dimensions, for other columns, Some prediction model can be used to infer the values. but with higher dimension in data, the prediction gets less convincing.

[samstave]

I need this!

I look forward to testing it out.

[thedudeabides5]

Terrific, have been looking for something like this since Hadley's work on the grand tourr

https://github.com/ggobi/tourr https://www.jstatsoft.org/article/view/v040i02 https://www.researchgate.net/publication/346903245_High-dime...

[paulgb]

ggobi is very cool. I was always surprised that the ideas it exemplified were so slow to catch on. Glad OP is working on rectifying that.

[aliqot]

I think the 'k6s' title is an unintentional misdirection

[pimlottc]

I was confused why the title bar says Kanaries but the product name is RATH. As someone coming in via a deep link with no knowledge of this product or company, it was confusing to figure out what was the actual product was called. I also couldn’t easily find an explanation of what RATH stands for.