top | item 37600731

(no title)

Okkef | 2 years ago

"log files of several several gigabytes ... Process them in minutes"

Thanks but no thanks.

discuss

order

eigenvalue|2 years ago

I misspoke there-- meant to say:

"The application has been tested with log files several gigabytes in size from dozens of machines and can process all of it in minutes."

That's the time it takes to connect to 20+ machines, download multiple gigs of log files from all of them, and parse/ingest all the data into a sqlite. If you have a big machine with a lot of cores and a lot of RAM, it's incredibly performant for what it does.