top | item 36059567

(no title)

mccanne | 2 years ago

Apologies if our examples in the article weren't rich enough or clear. The beauty of the error type is that you can wrap any value in an error and make the error as rich as you'd like, even stacking errors from different stages of an ingest pipeline so you can see the lineage of an error alongside data that wasn't subject to errors. e.g., imagine an error like this:

  error({
      stage: "transform",
      err: "input error",
      value: {
          stage: "normalize",
          err: "input error",
          value: {
              stage: "metrics",
              err: "divide by zero",
              value: {
                  sum: 123.5,
                  n: 0
              }
          }
      }
  })
... and you can quickly deduce that your "metrics" stage is dividing by "n" even if n is 0 and you can fix up your logic as well as fix the errors in place after fixing the bug in the ingest pipeline.

discuss

order

yyyk|2 years ago

My point is that you know the real source of the error, so there's no point for these terse messages. The example in the article, you know at point of conversion that you get a float, but instead you need to do a dance to get the original. Here it's a divide by zero - you can tell me the name of the variable too.

There's no need to repeat the infamous MSSQL "String or binary data would be truncated" message saga here - there's no reason you can't give much more verbose errors by default.

yyyk|2 years ago

For example, the error could have been '1064 is not a string value' or at least 'float64 is not a string value' etc.