Huh. When this was initially posted it had a weird and commercially restrictive license, but it looks like that's been reverted, possibly after (polite) discussion on /r/rust. It's MPL 2 now.
Performance figures are awesome. Also language support is great.
> Sonic only keeps the N most recently pushed results for a given word
This index discards old entries. This is fine for messages, in which aging items lose relevance. Yet the developer uses it for a help desk, which I think should give equal importance to all items.
In this area I would say the main comperitor is Groonga. It can be integrated into PostgreSQL with the PGroonga extension, and it indexes all of the data. However it consumes way more ram.
Wow. I’ve spent the last three weeks building a custom search solution in Kotlin - in my case I’m using tokenizers from Lucene and using a radix trie as an index. I actually looked at using Bleve (another rust search lib) initially but it didn’t have the right language support
Glancing over this it looks like a nearly perfect fit for my use case I just wish I had seen this a couple of weeks earlier!
I was looking for something just like this for a project in my team. We had been using this setup where a huge chunk of the data was being stored in triplicate: some of it in ES, some more of it in another database and finally the whole dataset in our data warehouse.
Hopefully I can use this to only provide the index + full text capability and just use the warehouse itself as the main db because the query performance is similar enough and the warehouse is criminally underused for what we pay for it.
That’s just false. The default config might set the JVM HEAP to 2GB (though I’m fairly certain it’s 1GB) but ES will start up with half a GB of heap with no issue.
Personally, I would advise finding an itch to scratch - something you'd like to see improved. Then try to understand the code from this perspective - where would you put the functionality, which pieces would it be connected to? Try to follow the lead, make notes as you read the code. You'll eventually get a feel of the infrastructure, going from the deal to the big picture - this is my default way of navigating projects.
If at any moment you feel lost, look through related issues and merge requests, as well as Git history. Perhaps you'll see how things get changed in the project, patterns intrinsic to it.
Also, keep in mind that you can always try to contact the community/author or invite someone to try figuring out your goal with you - once you collaborate, you'll get a solution tailored to the way you think. It does engage other people, but it also makes coding social and (at least to me) more satisfying.
manigandham|7 years ago
There's also Toshi: https://github.com/toshi-search/Toshi which is built on top of Tantivy: https://github.com/tantivy-search/tantivy
And for C++, there's Xapiland: https://github.com/Kronuz/Xapiand
And for Go, there's Blast: https://github.com/mosuka/blast built on Bleve: https://github.com/blevesearch/bleve
mrec|7 years ago
https://github.com/valeriansaliou/sonic/issues/52#issuecomme...
javitury|7 years ago
> Sonic only keeps the N most recently pushed results for a given word
This index discards old entries. This is fine for messages, in which aging items lose relevance. Yet the developer uses it for a help desk, which I think should give equal importance to all items.
In this area I would say the main comperitor is Groonga. It can be integrated into PostgreSQL with the PGroonga extension, and it indexes all of the data. However it consumes way more ram.
mellab|7 years ago
Glancing over this it looks like a nearly perfect fit for my use case I just wish I had seen this a couple of weeks earlier!
networkimprov|7 years ago
https://github.com/blevesearch/bleve
O_H_E|7 years ago
FridgeSeal|7 years ago
I was looking for something just like this for a project in my team. We had been using this setup where a huge chunk of the data was being stored in triplicate: some of it in ES, some more of it in another database and finally the whole dataset in our data warehouse.
Hopefully I can use this to only provide the index + full text capability and just use the warehouse itself as the main db because the query performance is similar enough and the warehouse is criminally underused for what we pay for it.
winrid|7 years ago
dannycastonguay|7 years ago
marmaduke|7 years ago
Xylakant|7 years ago
sidcool|7 years ago
d33|7 years ago
If at any moment you feel lost, look through related issues and merge requests, as well as Git history. Perhaps you'll see how things get changed in the project, patterns intrinsic to it.
Also, keep in mind that you can always try to contact the community/author or invite someone to try figuring out your goal with you - once you collaborate, you'll get a solution tailored to the way you think. It does engage other people, but it also makes coding social and (at least to me) more satisfying.
Let me know what you think of this approach!
StavrosK|7 years ago
MosheZada|7 years ago
marmaduke|7 years ago
https://github.com/valeriansaliou/sonic/blob/master/PROTOCOL...
deadwisdom|7 years ago
21stio|7 years ago