top | item 40609758

(no title)

harisamin | 1 year ago

On the noisy NodeJS auto-instrumentation, it is indeed very noisy out of the box. Myself along with a bunch of other ppl finally got the project to allow you to select the instrumentations via configuration. Saves having to create your own tracer.ts/js file.

Here's the PR that got merged earlier in the year: https://github.com/open-telemetry/opentelemetry-js-contrib/p...

The env var config is `OTEL_NODE_ENABLED_INSTRUMENTATIONS`

Anyways, love Opentelemetry success stories. Been working hard on it at my current company and yielding fruits already :)

discuss

order

tnolet|1 year ago

That is awesome. Had no idea this was available as an env var. After diving into OTel for our backend, we also found some of this stuff is just too noisy. We switch it of using this code snippet, for anyone bumping into this thread:

   instrumentations: [getNodeAutoInstrumentations({
      '@opentelemetry/instrumentation-fs': {
        enabled: false,
      },
      '@opentelemetry/instrumentation-net': {
        enabled: false,
      },
      '@opentelemetry/instrumentation-dns': {
        enabled: false,
      },

harisamin|1 year ago

yeah I totally turned all of those off...way too noisy :)

Veserv|1 year ago

Why would you disable instrumentation instead of just filtering the recorded log?

That only makes sense if the instrumentation overhead itself is significant. But, for a efficient recording implementation that should only really start being a problem when your average span is ~1 us.

tnolet|1 year ago

Oh, simple answer. The tools you use to inspect those traces just blow up with noise. Like a trace that shows 600+ of file reads that all take less than half a millisecond.

This is all noise when you are trying to debug more common issues than your FS being too slow.

+ also storage cost. Most vendors charge by Mb stored or span recorded.

roboben|1 year ago

This is great. Last time I tried this I couldn’t even find a way in code to disable some.

serverlessmom|1 year ago

This is so cool! I’ve had this exact problem before.