top | item 38187510

(no title)

mavam | 2 years ago

The extra data type of unstructured/bytes is optional in that you don’t have to use it if you don’t need it. Just start with a table if that’s your use case.

In security, binary artifacts are common, e.g., to scan YARA rules on malware samples and produce a structured report (“table”). Turning packet traces into structured logs is another example. Typically you have to switch between a lot of tools for that, which makes the process complex.

(The “void” type is only for symmetry in that every operator has an input and output type. The presence of void makes an operator a source or sink. A “closed” pipeline invariant is one with source and sink, and only closed pipelines can execute in our mental model.)

discuss

order

code_biologist|2 years ago

I appreciate the explanation and your initial comment makes more sense with the context.