top | item 45573428

(no title)

jlundberg | 4 months ago

I really like just encoding each object as JSON and then concatinating them with a new line between.

Allows parsing and streaming without any special libraries and allow for an unlimited amount of data (with objects being reasonably sized).

Usually gives these files the .jsonlines suffix when stored on disk.

Allows for batch process without requiring huge amounts of memory.

discuss

order

codesnik|4 months ago

jq allows to covert normal json document to jsonlines and back, though it does it much faster if it can slurp an original doc into memory (no --stream option)

0x6c6f6c|4 months ago

Based on this thread that's called NDJSON

Newline Delimited JSON

TIL

keitmo|4 months ago

It's also known as JSONL (JSON Lines).