[hijinks]

Take a look at quickwit. Its basically a clone of elasticsearch but in rust.

I have around 380 TBs of logs currently in s3 and have sub 1s searches for needle in the haystack searches. It handles all that with just 5 search nodes running on kubernetes with 6gig of RAM each.

I'm ingesting around 20TBs of logs a day on it.