Co-founder of Quickwit here. Seeing our acquisition by Datadog on the HN front page feels like a truly full-circle moment.
HN has been interwoven with Quickwit's journey from the very beginning. Looking back, it's striking to see how our progress is literally chronicled in our HN front-page posts:
- Searching the web for under $1000/month [0]
- A Rust optimization story [1]
- Decentralized cluster membership in Rust [2]
- Filtering a vector with SIMD instructions (AVX-2 and AVX-512) [3]
- Efficient indexing with Quickwit Rust actor framework [4]
- A compressed indexable bitset [5]
- Show HN: Quickwit – OSS Alternative to Elasticsearch, Splunk, Datadog [6]
- Quickwit 0.8: Indexing and Search at Petabyte Scale [7]
- Binance built a 100PB log service with Quickwit [9]
- Datadog acquires Quickwit [10]
Each of these front-page appearances was a milestone for us. We put our hearts into writing those engineering articles, hoping to contribute something valuable to our community.
I'm convinced HN played a key role in Quickwit's success by providing visibility, positive feedback, critical comments, and leads that contacted us directly after a front-page post. This community's authenticity and passion for technology are unparalleled. And we're incredibly grateful for this.
Anyway tantivy is great! I love pg_search https://www.paradedb.com/blog/introducing_search (which appears to be built by another company, but on top of tantivy, which is a great feature of open source)
Now, I am worried about development being stalled after this acquisition. How does further developing tantivy in the open helps Datadog's bottom line?
Congratulations! The fact you and your team managed to built Tantivy is a huge contribution to open source.
As someone who never managed to built a fond relationship with Apache Lucene based products (Solf, Elastic). I was extremely happy to see Tantivy in open source.
BM25 scoring, proper asian language support, speed, memory foot prints, etc - amazing job! Thank you so much!
If Tantivy itself just stays permanently under Apache2 licence and find a sustainable path to co exist with the rest of open source community - it's all good guys. You are more than deserve a commercial success.
Well, it looks like Quickwit was going to add an Enterprise license as of earlier this year (PR #5529), which I had been keeping eyes on, but this announcement says they're instead going to relicense as Apache 2.0 so the "community can continue on":
> We will be focused on building a new product with Datadog, and to ensure our open-source community can continue, we will soon release a major update of both Quickwit with a relicense to Apache License 2.0 and tantivy.
So, it looks like we'll get a more liberally licensed Quickwit, but reading between the lines suggests development of it is might otherwise be winding down? It has been pretty nice and stable in my experience, so I can't really complain much. But I was really looking forward to what else it could bring.
"So, it looks like we'll get a more liberally licensed Quickwit, but reading between the lines suggests development of it is might otherwise be winding down?"
They will stop fulltime day-to-day effort in it themselves, probably because they have been relocated to writing a similar service but closed and integrated in DD, but it seems they want to opensource the current product with a OSI compliant license in the hopes that the community picks up the tab.
I think that's a nice trade. Could have been much worse.
By the way, also note that DD is not a total stranger in the OSS space. They actually opensourced their observability pipeline tooling for general use as Vector, which is a rock solid product. - https://vector.dev/
OrioleDB continues to be a fully open source and liberally licensed. We're working with the OrioleDB team to provide an initial distribution channel so they can focus on the storage engine vs hosting + providing lots of user feedback/bug reports. Our shared goal is to advance OrioleDB until it becomes the go-to storage engine for Postgres, both on Supabase and everywhere else.
Acquisitions don't necessarily mean the end of innovation. Sometimes, it allows them to take innovations they've worked hard on for years and expand the reach to a significantly larger audience :)
I have met the founders of all 3 of these companies and can assure you they all care tremendously about bringing their work to the world.
ParadeDB is independent and without plans to sell anytime soon, though :)
They already do. For example, for containers you have 5 per host included in the Pro plan (I have more in my hobby projects...) and then you pay extra for each. Their pricing model is quite complex and with time you start to wonder if all these nice features are really worth that high prices.
> Mezmo recently put in production Quickwit to serve thousands of customers and petabytes of logs, drastically reducing infrastructure cost and complexity while delivering the same user experience.
I can't imagine they feel great about Quickwit getting bought by a competitor after that.
Either they did it on their own, in which case that's open source for you, or they have some kind of contract with quickwit, in which case I'd be really surprised if they don't have some clauses which handle this.
> This summer, the wind started to turn. We witnessed stronger open-source traction, our revenue increased dramatically, and VCs became more insistent. It was time for us to open a new chapter for the company and raise a series A round.
Rhetorically, why was it time for this?
Practically, the answer is right there: the VCs wouldn't accept a mere rapidly-growing company with great tech. It's either an up round so they can mark up the value on their portfolio, or if the market isn't hot enough for a high-priced Series A, force an exit.
> why was it time for this?
Practically, the answer is right there: the VCs wouldn't accept a mere rapidly-growing company with great tech.
Remember that this is a decision you make when you accept the seed money, not when it comes to looking for further money. If you want to build a profitable growing company, bc funding is not the path to that.
Practically, what happens if the founders don’t want to play ball? VCs can muscle up and force a sale on worse terms? Just not clear how much of what goes on is asking nicely vs coercion, and where the actual leverage lies.
Our seed round was 100% made of SAFE, so VCs did not have the power to force us to do anything.
The sentence in the blog post is a tad misleading. I suspect François is not really talking about VCs that had already invested in quickwit, but about the usual flow of other VCs who contacted us, to know about the company and be part of our eventual series A.
It just generally felt like we were "at a crossing".
I hate Datadog. We use their name as an epithet at our company for how not to sell/market. Their selling tactics circa 2015-2018 completely burned us out. Endless calls and emails. The icing on the cake was an AWS reInvent presentation on Lambda right when lambda was first announced. We were pumped to get in on lambda early. Got the whole crew to attend the talk. Turned out to be a rudimentary copy of a Barr "lambda up and running" blog wrapped in a stand up comedy routine hawked by a Datadog employee who made sure to tell us he was a Datadog employee. Get us all drunk and happy and think Datadog is cool.
Genuine question: has the company changed enough in the interim to deserve a second look?
There are multiple possible outcomes from the merge with Datadog.
As my ex manager once told - there is no such thing as nice people in P&L statement. Someone has to pay
It's very easy to be anxious and see the path to the dark side here.
However one of possible outcomes - there will be a valid open source competitor to Grafana ecosystems, and this along secure the rest of scene from relicensing. There is a chance it will be all win-win with clear sustainable path and no money and power struggles for founders.
I want to stay on the positive here. Time will show.
Loki by Grafana Labs is nice (https://grafana.com/oss/loki/). There was a time (3+ years ago) where the product was changing pretty rapidly and much of the documentation was on git, so we had a few headaches doing minor version bumps, but I believe its much more mature now.
Take a look at VictoriaLogs - https://docs.victoriametrics.com/victorialogs/ . It is a single small executable, which runs out of the box with the default configs (e.g. it is zero-config) and stores all the data into a single directory (support for object storage coming soon).
> Organizations in financial services, insurance, healthcare, and other regulated industries must meet stringent data residency, privacy, and regulatory requirements while maintaining full visibility into their systems. This becomes challenging when logs need to remain at rest in customers’ environments or specific regions, hindering teams’ ability to attain seamless observability and insight. To help our customers meet these requirements without sacrificing visibility or introducing multiple logging tools, we are pleased to announce that Quickwit—a popular open source distributed search engine—is joining Datadog.
they are planning to allow you to run your logs in your own datacenter/cloud and put something like a proxy there or being built into quickwit that your logs show up in the datadog UI
My guess is you will be billed per gig or something but not nearly the cost of shipping your logs to DD
Sqreen was a great product for startups. Unfortunately acquired by the Datadog. Product was never fully integrated into their offerings and there is no such alternative.
Hey I'm an engineer who worked at Sqreen and now Datadog. What kind of feature is still missing in Datadog ASM? To my knowledge everything is there except managing CSP and security headers now.
I integrated Quickwit into our o11y platform. It was great tech with lots of promise, and now it's dead. Yet another reason to never do business with Datahog.
Do any engineers actually like DD? Execs/managers seem to love DD and get upsold all the time on it and ask engineers to implement some of their half-baked features. It seems like its good for alerts and dashboards for infrastructure teams. But as an engineer its a pain and using it for log analysis is much more annoying than just tracking down the actual relevant logs and grepping.
Easily the best product in its category. The value is in APM/tracing - if you're just using it for logs you're missing a lot.
If you're used to traditional Enterprise pricing it's fairly priced for the value you get but anybody coming from self-funded or VC it's very expensive. If you're already using Splunk you can afford it.
One of its best features is it's consumption priced not by seats so easy to open up for all of eng including product/QA teams and not just devs to use which is great for breaking down barriers.
It's a bit like aws in that it's a platform - use of one product tends to encourage using more from their suite.
It’s good, I like it. I don’t use the logs product (we splunk) but I’m big fan of the automatic profiling and trace/span stuff. It’s like 50% better UX compared to other tools I’ve tried. But it’s expensive enough we’re always thinking about moving off it. That will be a very sad day for me.
francoismassot|1 year ago
HN has been interwoven with Quickwit's journey from the very beginning. Looking back, it's striking to see how our progress is literally chronicled in our HN front-page posts:
- Searching the web for under $1000/month [0]
- A Rust optimization story [1]
- Decentralized cluster membership in Rust [2]
- Filtering a vector with SIMD instructions (AVX-2 and AVX-512) [3]
- Efficient indexing with Quickwit Rust actor framework [4]
- A compressed indexable bitset [5]
- Show HN: Quickwit – OSS Alternative to Elasticsearch, Splunk, Datadog [6]
- Quickwit 0.8: Indexing and Search at Petabyte Scale [7]
- Tantivy – full-text search engine library inspired by Apache Lucene [8]
- Binance built a 100PB log service with Quickwit [9]
- Datadog acquires Quickwit [10]
Each of these front-page appearances was a milestone for us. We put our hearts into writing those engineering articles, hoping to contribute something valuable to our community.
I'm convinced HN played a key role in Quickwit's success by providing visibility, positive feedback, critical comments, and leads that contacted us directly after a front-page post. This community's authenticity and passion for technology are unparalleled. And we're incredibly grateful for this.
Thank you all :)
[0] https://news.ycombinator.com/item?id=27074481
[1] https://news.ycombinator.com/item?id=28955461
[2] https://news.ycombinator.com/item?id=31190586
[3] https://news.ycombinator.com/item?id=32674040
[4] https://news.ycombinator.com/item?id=35785421
[5] https://news.ycombinator.com/item?id=36519467
[6] https://news.ycombinator.com/item?id=38902042
[7] https://news.ycombinator.com/item?id=39756367
[8] https://news.ycombinator.com/item?id=40492834
[9] https://news.ycombinator.com/item?id=40935701
[10] https://news.ycombinator.com/item?id=42648043
nextaccountic|1 year ago
Anyway tantivy is great! I love pg_search https://www.paradedb.com/blog/introducing_search (which appears to be built by another company, but on top of tantivy, which is a great feature of open source)
Now, I am worried about development being stalled after this acquisition. How does further developing tantivy in the open helps Datadog's bottom line?
adeptima|1 year ago
As someone who never managed to built a fond relationship with Apache Lucene based products (Solf, Elastic). I was extremely happy to see Tantivy in open source.
BM25 scoring, proper asian language support, speed, memory foot prints, etc - amazing job! Thank you so much!
https://github.com/quickwit-oss/tantivy
IMHO Datadog made a smart move!
If Tantivy itself just stays permanently under Apache2 licence and find a sustainable path to co exist with the rest of open source community - it's all good guys. You are more than deserve a commercial success.
blinded|1 year ago
aseipp|1 year ago
> We will be focused on building a new product with Datadog, and to ensure our open-source community can continue, we will soon release a major update of both Quickwit with a relicense to Apache License 2.0 and tantivy.
So, it looks like we'll get a more liberally licensed Quickwit, but reading between the lines suggests development of it is might otherwise be winding down? It has been pretty nice and stable in my experience, so I can't really complain much. But I was really looking forward to what else it could bring.
Congrats to the team, in any case!
mindcrash|1 year ago
They will stop fulltime day-to-day effort in it themselves, probably because they have been relocated to writing a similar service but closed and integrated in DD, but it seems they want to opensource the current product with a OSI compliant license in the hopes that the community picks up the tab.
I think that's a nice trade. Could have been much worse.
By the way, also note that DD is not a total stranger in the OSS space. They actually opensourced their observability pipeline tooling for general use as Vector, which is a rock solid product. - https://vector.dev/
Hixon10|1 year ago
1. https://www.warpstream.com/
2. https://www.orioledb.com/
3. https://quickwit.io/
oliverrice|1 year ago
OrioleDB continues to be a fully open source and liberally licensed. We're working with the OrioleDB team to provide an initial distribution channel so they can focus on the storage engine vs hosting + providing lots of user feedback/bug reports. Our shared goal is to advance OrioleDB until it becomes the go-to storage engine for Postgres, both on Supabase and everywhere else.
Happy to hear any concerns you have
philippemnoel|1 year ago
I have met the founders of all 3 of these companies and can assure you they all care tremendously about bringing their work to the world.
ParadeDB is independent and without plans to sell anytime soon, though :)
acaloiar|1 year ago
hdjjhhvvhga|1 year ago
Too|1 year ago
maxwellg|1 year ago
I can't imagine they feel great about Quickwit getting bought by a competitor after that.
PittleyDunkin|1 year ago
The good part for the rest of us is it's a signal that there's likely some appetite for a fork if Datadog screws the pooch.
sealeck|1 year ago
gnabgib|1 year ago
Binance built a 100PB log service with Quickwit (228 points, 6 months ago, 195 comments) https://news.ycombinator.com/item?id=40935701
Show HN: Quickwit – OSS Alternative to Elasticsearch, Splunk, Datadog (145 points, 1 year ago, 51 comments) https://news.ycombinator.com/item?id=38902042
slt2021|1 year ago
propter_hoc|1 year ago
Rhetorically, why was it time for this?
Practically, the answer is right there: the VCs wouldn't accept a mere rapidly-growing company with great tech. It's either an up round so they can mark up the value on their portfolio, or if the market isn't hot enough for a high-priced Series A, force an exit.
maccard|1 year ago
Remember that this is a decision you make when you accept the seed money, not when it comes to looking for further money. If you want to build a profitable growing company, bc funding is not the path to that.
bomewish|1 year ago
fulmicoton|1 year ago
The sentence in the blog post is a tad misleading. I suspect François is not really talking about VCs that had already invested in quickwit, but about the usual flow of other VCs who contacted us, to know about the company and be part of our eventual series A.
It just generally felt like we were "at a crossing".
No one twisted our arm.
Rastonbury|1 year ago
scop|1 year ago
Genuine question: has the company changed enough in the interim to deserve a second look?
orf|1 year ago
adeptima|1 year ago
As my ex manager once told - there is no such thing as nice people in P&L statement. Someone has to pay
It's very easy to be anxious and see the path to the dark side here.
However one of possible outcomes - there will be a valid open source competitor to Grafana ecosystems, and this along secure the rest of scene from relicensing. There is a chance it will be all win-win with clear sustainable path and no money and power struggles for founders.
I want to stay on the positive here. Time will show.
Justin_K|1 year ago
thebruce87m|1 year ago
rr808|1 year ago
Damn I never get sales pitches like that.
datavirtue|1 year ago
liminal|1 year ago
tdubey|1 year ago
makeavish|1 year ago
https://github.com/SigNoz/signoz
PS: I am maintainer at SigNoz
valyala|1 year ago
unknown|1 year ago
[deleted]
bk146|1 year ago
okbro|1 year ago
> Organizations in financial services, insurance, healthcare, and other regulated industries must meet stringent data residency, privacy, and regulatory requirements while maintaining full visibility into their systems. This becomes challenging when logs need to remain at rest in customers’ environments or specific regions, hindering teams’ ability to attain seamless observability and insight. To help our customers meet these requirements without sacrificing visibility or introducing multiple logging tools, we are pleased to announce that Quickwit—a popular open source distributed search engine—is joining Datadog.
politelemon|1 year ago
slt2021|1 year ago
everfrustrated|1 year ago
What happened to Vector the last opensource they bought? Are they still hired?
winrid|1 year ago
hijinks|1 year ago
My guess is you will be billed per gig or something but not nearly the cost of shipping your logs to DD
psanford|1 year ago
I loved that I was able to setup Quickwit on AWS lambda and have a good cloud based search engine for $0.01 / month.
o0-0o|1 year ago
jhgg|1 year ago
bdcravens|1 year ago
iandanforth|1 year ago
seaghost|1 year ago
Moutiix|1 year ago
hackernoops|1 year ago
unknown|1 year ago
[deleted]
dirtshell|1 year ago
everfrustrated|1 year ago
If you're used to traditional Enterprise pricing it's fairly priced for the value you get but anybody coming from self-funded or VC it's very expensive. If you're already using Splunk you can afford it.
One of its best features is it's consumption priced not by seats so easy to open up for all of eng including product/QA teams and not just devs to use which is great for breaking down barriers.
It's a bit like aws in that it's a platform - use of one product tends to encourage using more from their suite.
jitl|1 year ago
SteveNuts|1 year ago
geraldwhen|1 year ago
Showing ALL the logs isn’t cool. Showing SOME logs at random is cool.
hackernoops|1 year ago
datavirtue|1 year ago
morelish|1 year ago
gigatexal|1 year ago