Ask HN: When it's too much observability?

[diegojromero]

I have been adding observability (via OpenTelemetry) to some web applications for almost a year. I tend to add a span for each function with its input arguments (if possible), besides other data like information about the flow or the return value. Being able to see the function call cascade has been useful for debugging applications.

Having said that, I have two questions for the community:

1. Should I only add telemetry only for critical parts?

Thanks to adding a lot of boilerplate in each function, I'm able to debug issues in production easily. Would you say it is a bad practice to add a lot of telemetry information?

2. Where do you put the "limit" of your observability efforts in your applications when developing them?

[curious_curios]

If you’re getting value from it, keep adding it!

Two cases where this advice breaks down:

- If the telemetry boilerplate code starts hampering your ability to read and reason about the underlying business logic code.

- When your observability bill gets too big.

[diegojromero]

Well, I was so worried about boilerplate that I created a Python decorator for avoiding polluting the code with so much of that (https://github.com/diegojromerolopez/otelize).

About the cost, AFAIK it is not prohibitive.

I will continue working as I was, thanks for your opinion!