top | item 31693941

(no title)

schmeckleberg | 3 years ago

i think you and the above poster are in vehement agreement. ingesting 2 MBs of logs per minute is impressive in its pluperfect unimpressiveness.

maybe the presentation was called "Timmy's first named pipe" or "Sally explores /etc/logrotate.d"

discuss

order

FredPret|3 years ago

That’s a LOT of text to describe me sitting on my couch. 2MB per minute is far more than the most detailed biography in existence.

exdsq|3 years ago

220m users. Let’s imagine 50m are streaming concurrently. That’s 100TB an hour in logs lol. They could be storing an entire petabyte of logs a day. My friend did some data center stuff for the large hadron collider and wasn’t hitting these data ingestion states, and these are just to record me binging the office.

ummonk|3 years ago

The comment said "log processing system". Sounds more like it's a stream and not stored logs.

eightysixfour|3 years ago

2 MB/per minute/per stream at Netflix scale is crazy.