Right now I'm heavily into Grafana but when it comes to BI it kinda falls flat, I regularly have to fall back to using the Plotly plugin to create the charts (but it's getting better, at least you can do a normal scatter or bar chart out of the box since version 8. Labeling the axes is still a problem though). Navigation is also problematic, like jumping to a source table takes a lot of effort to make (basically you have to create a new dashboard and do some hyperlinking instead of there being a ready to go "view source data" button). I feel like there's a lot of friction to get Grafana to do BI, but I've also invested so much time in it I'm afraid to jump ship...
Superset looked good, but operating superset quickly runs into the same Python issues all Python software suffers from.
Sometimes it would just break for no apparent reason. Configuring it was a nightmare of magic Python code and unclear settings. Trying to use plugins was equally painful: due to the poor boundary separating the applications dependencies from the plugins dependencies, adding a db connector could just bork the whole application.
What do people use for an "analytics.js" for reporting events with common items like campaign data, user device and user profile measurements, and related from browsers and devices?
Is there any open source BI tool that can be embedded in some other product so that users (not product developers but final users) can create their own dashboards?
Looks pretty neat. Having not fully investigated it yet, I will say the one thing I usually run into with this and other BI tools is a lack of flexibility in the UI for forming queries. It's sometimes limited to one table or view at a time. I wonder why more of them don't use more flexible querying techniques, perhaps just due to the risk of a bad query being formed?
>Where can I find persistent application data storage?
>We use the .us-data folder to store PostgreSQL data permanently. You can delete this folder if you want, it will be recreated with the demo data after restarting the datalens-us container
I really like the concept of Domo. They have ETL, modeling, a warehouse and BI in one app ("data-stack-in-a-box"). I've interviewed 20 of their customers and the general sentiment was pretty bad. There's a long sales process, a longer process to get it set it up, and they've built all the modeling and connectors themselves (vendor lock in, none are best-in-class).
Definite (https://www.definite.app/) is a data-stack-in-a-box. We have a built-in modeling layer for core metrics and an AI assistant to answer any one-off questions.
A few ways we're different:
Built on open source - We run the data stack for you and give you a single app to manage and analyze your data, but it's all built on open source standards. So if you decide at any point you want to run it all yourself, the code is yours to lift and shift to your own infrastructure.
Battle tested connectors - We're using Meltano / Singer (open source library from Stitch) for our connectors, so they've been used heavily in production for years.
Self-serve that actually works - A lot of tools promise self-serve, but AI is making this real. We've invested heavily in making it possible to ask questions and get accurate answers. The AI queries a modeled view of your data that can answer questions that depend on well defined metrics (e.g. ARR, DAU, etc.).
We have Domo at work and it just seems overly complicated and insane. I'm wanting to learn it instead of just polling data sources and adding them to a local Opensearch instance, but... too verbose for me.
Cool. Rolling your own BI seems fraught with peril at most orgs where I imagine the buy vs build decision is always buy. How many PowerBI or Tableau seats do you need before rolling your own internal BI platform starts to make sense?
I don't think it ever makes sense because the large players will always be able to make new innovations (mobile apps, natural language querying, SSO integrations, etc) that, short of large corps hiring BI teams to invest in the open source ecosystem like superset, your open source solution won't have.
I must like to live dangerously. In all seriousness though there are low cost alternatives to those mega BI tools that suit many use cases. If I wasn't rolling my own I'd probably start with Metabase or Superset. What I use: https://github.com/totalhack/zillion
Since the discussion started in the comments already, I have a similar question: any recommendations for a solution (don’t care if OSS or not) that has the best UX for nontechnical people to assemble some data and reports anyhow? I have Salesforce, some mariadb/postgres and (optionally) hubspot as data sources.
I can buy or manually provision anything, no technical hurdles or policies from that side. My absolute focus is the raw UX for business people.
Honestly Metabase has given best balance between allowing non-technical users to self-service and technical users to dig in and use raw sql if that's what they want. Also it OSS core so you can self host. It is super feature rich and has most everything in the OSS version as long as you don't need enterprise features like SAML auth, audit log, ...,etc
I'm a co-founder of Dataland.io where we're building a powerful dataset viewer + search engine that can work on top of your Postgres or data warehouse.
We designed it specifically to provide an excellent UX to business users while reducing BI burden on the data team. We find that most business users often just need to search, filter, and sort instead of looking at charts to make operational decisions.
UX-wise, what sets us apart are:
- <1s full-text search (even on billions of rows of data), feels like Cmd+F in Google sheets, but faster
- Performance: we stream billions of rows into the web browser, seamless scrolling (no paging of 50 records at a tieme)
- Rich cells make tables easier to scan/read (enum strings => colored tags, numbers => color-coded based on value => checkboxes, timestamptz => clear date time pills)
If that fits what you need, happy to give you a demo.
arthur(at)dataland.io
Otherwise, I think the simplest BI (if charts are impt) could be something like evidence.dev or Metabase.
But I also think it's going to require some curation on your part. Can you reasonably expect business users to navigate the entire schema/table tree across these three sources? That's where I think the bottleneck often lies -- if your BI tool allows engineering to just expose a subset of curated core tables.
The problem is that developing the perfect UI for nontechnical people to assemble reports probably requires a bespoke frontend for your business, and one that likely lags behind the reality of its changes. Most businesses instead opt to just hire semitechnical people that can do a bit of data work and give answers to the report-writers, as they can accommodate business changes over time and understand how to construct new queries out of the overall business' data sources.
Maybe that'll change one day with AI, and when it does that will be bought by every big company in the world (-:
[+] [-] xnx|2 years ago|reply
[+] [-] amcaskill|2 years ago|reply
I work on an open source code-based BI tool called Evidence, which might be of interest to you.
It's effectively a static site generator aimed at building automated reports and analysis.
https://github.com/evidence-dev/evidence
Previous discussions on HN:
https://news.ycombinator.com/item?id=28304781 - 91 comments
https://news.ycombinator.com/item?id=35645464 - 97 comments
[+] [-] RedShift1|2 years ago|reply
[+] [-] FridgeSeal|2 years ago|reply
Superset looked good, but operating superset quickly runs into the same Python issues all Python software suffers from.
Sometimes it would just break for no apparent reason. Configuring it was a nightmare of magic Python code and unclear settings. Trying to use plugins was equally painful: due to the poor boundary separating the applications dependencies from the plugins dependencies, adding a db connector could just bork the whole application.
[+] [-] noughtme|2 years ago|reply
[+] [-] doctorpangloss|2 years ago|reply
[+] [-] zxspectrum1982|2 years ago|reply
[+] [-] moltar|2 years ago|reply
Also CubeJS if you want to a bit more flexibility.
[+] [-] throw03172019|2 years ago|reply
[+] [-] cvalka|2 years ago|reply
[+] [-] mdekkers|2 years ago|reply
[+] [-] felixhummel|2 years ago|reply
[+] [-] totalhack|2 years ago|reply
My preferred approach is implemented in Zillion, which I use for BI at my company: https://github.com/totalhack/zillion
[+] [-] dwheeler|2 years ago|reply
it's => its
[+] [-] pklausler|2 years ago|reply
[+] [-] unixhero|2 years ago|reply
>We use the .us-data folder to store PostgreSQL data permanently. You can delete this folder if you want, it will be recreated with the demo data after restarting the datalens-us container
Why "us"?
[+] [-] nijave|2 years ago|reply
[+] [-] RedShift1|2 years ago|reply
[+] [-] capableweb|2 years ago|reply
[+] [-] kasperni|2 years ago|reply
[+] [-] lasermike026|2 years ago|reply
[+] [-] mritchie712|2 years ago|reply
I really like the concept of Domo. They have ETL, modeling, a warehouse and BI in one app ("data-stack-in-a-box"). I've interviewed 20 of their customers and the general sentiment was pretty bad. There's a long sales process, a longer process to get it set it up, and they've built all the modeling and connectors themselves (vendor lock in, none are best-in-class).
Definite (https://www.definite.app/) is a data-stack-in-a-box. We have a built-in modeling layer for core metrics and an AI assistant to answer any one-off questions.
A few ways we're different:
Built on open source - We run the data stack for you and give you a single app to manage and analyze your data, but it's all built on open source standards. So if you decide at any point you want to run it all yourself, the code is yours to lift and shift to your own infrastructure.
Battle tested connectors - We're using Meltano / Singer (open source library from Stitch) for our connectors, so they've been used heavily in production for years.
Self-serve that actually works - A lot of tools promise self-serve, but AI is making this real. We've invested heavily in making it possible to ask questions and get accurate answers. The AI queries a modeled view of your data that can answer questions that depend on well defined metrics (e.g. ARR, DAU, etc.).
[+] [-] _boffin_|2 years ago|reply
[+] [-] RedShift1|2 years ago|reply
[+] [-] wutwutwat|2 years ago|reply
[+] [-] 6c696e7578|2 years ago|reply
https://www.intellinews.com/russian-tech-titan-yandex-ceo-vo...
[+] [-] mobileexpert|2 years ago|reply
[+] [-] htrp|2 years ago|reply
[+] [-] totalhack|2 years ago|reply
[+] [-] londons_explore|2 years ago|reply
It's a shame that geopolitics means most of it will have to be reinvented by someone else before it'll see any use.
[+] [-] efxhoy|2 years ago|reply
[+] [-] unknown|2 years ago|reply
[deleted]
[+] [-] anonyfox|2 years ago|reply
I can buy or manually provision anything, no technical hurdles or policies from that side. My absolute focus is the raw UX for business people.
Suggestions?
[+] [-] davidarenas|2 years ago|reply
[+] [-] arthurwu|2 years ago|reply
We designed it specifically to provide an excellent UX to business users while reducing BI burden on the data team. We find that most business users often just need to search, filter, and sort instead of looking at charts to make operational decisions.
UX-wise, what sets us apart are:
- <1s full-text search (even on billions of rows of data), feels like Cmd+F in Google sheets, but faster
- Performance: we stream billions of rows into the web browser, seamless scrolling (no paging of 50 records at a tieme)
- Rich cells make tables easier to scan/read (enum strings => colored tags, numbers => color-coded based on value => checkboxes, timestamptz => clear date time pills)
If that fits what you need, happy to give you a demo.
arthur(at)dataland.io
Otherwise, I think the simplest BI (if charts are impt) could be something like evidence.dev or Metabase.
But I also think it's going to require some curation on your part. Can you reasonably expect business users to navigate the entire schema/table tree across these three sources? That's where I think the bottleneck often lies -- if your BI tool allows engineering to just expose a subset of curated core tables.
[+] [-] mritchie712|2 years ago|reply
> has the best UX for nontechnical people to assemble some data
If they can use Excel / pivot tables, they can use Definite. They can also just ask in natural language and we generate the report for them.
> Salesforce, some mariadb/postgres and (optionally) hubspot as data sources
We have pipelines for all of these and can spin up a managed data warehouse to store all the data if you don't already have one.
Drop me a note at [email protected] if you're interested
[+] [-] robertlagrant|2 years ago|reply
Maybe that'll change one day with AI, and when it does that will be bought by every big company in the world (-:
[+] [-] tillvz|2 years ago|reply
Follows a conversational "ChatGPT-like" approach since already 2016.
Info: I'm one of the founders.
[+] [-] Aaronstotle|2 years ago|reply
[+] [-] unknown|2 years ago|reply
[deleted]
[+] [-] riffic|2 years ago|reply
[+] [-] Logans_Run|2 years ago|reply
ps - I spotted the pedant that is technically correct, and I claim my 5 McFun bucks!
[+] [-] culebron21|2 years ago|reply
[+] [-] unknown|2 years ago|reply
[deleted]
[+] [-] esotericimpl|2 years ago|reply
[deleted]
[+] [-] vgt|2 years ago|reply
[+] [-] alexlur|2 years ago|reply
[+] [-] macinjosh|2 years ago|reply
[deleted]