I agree. I tried to get it to work recently with datadog, but there was so many hiccups. I ended up having to use datadogs solution mostly. The documentation across everything is also kind of confusing
It's a nice dream. At Google Cloud Next last year, the vendors kinda of came in two buckets. Datadog, and everyone trying to replace Datadog's outrageous bills.
Yeah their agent will accept traces from the standard Otel SDK but there is no way to change their SDK to send the traces to anyone other than Datadog when I last checked a couple(?) of years ago.
I mean I understand why they did that but it really removes one of the most compelling parts about Otel. We ended doing the hard work of using the standard Otel libraries. I had to contribute a PR or two to get it all to work with our services but am glad that's the route we went because now we can switch vendors if needed (which is likely in the not too distant future in our case.
part of the reason for that experience is also because DataDog is not open telemetry native and all their docs and instructions encourage use of their own agents. Using DataDog with Otel is like trying to hold your nose round over your head
You should try Otel native observability platforms like SigNoz, Honeycomb, etc. your life will be much simpler
Disclaimer : i am one of the maintainers at SigNoz
The biggest barrier to setting up oTel for me is the development experience. Having a single open specification is fantastic, especially for portability, but the SDKs are almost overwhelmingly abstract and therefore difficult to intuit.
I used to really like Datadog for being a one-stop observability shop and even though the experience of integrating with it is still quite simple, I think product and pricing wise they've jumped the shark.
I'm much happier these days using a collection of small time services and self-hosting other things, and the only part of that which isn't joyful is the boilerplate and not really understanding when and why you should, say, use gRPC over HTTP, and stuff like that.
You are generally correct but I've used https://github.com/openobserve/openobserve for several projects for dev-only complete OTel stack (dashboards included) and I liked it. There are better dashboards out there for sure, but for what I needed locally it did the job fantastically well. Zero complaints.
It's extremely easy to self-host, either on a dev machine, a VPS, or in any Docker-based PaaS.
SomaticPirate|1 year ago
OTel is a bear though. I think the biggest advantage it gives you is the ability to move across tracing providers
rikthevik|1 year ago
It's a nice dream. At Google Cloud Next last year, the vendors kinda of came in two buckets. Datadog, and everyone trying to replace Datadog's outrageous bills.
jensensbutton|1 year ago
bebop|1 year ago
hangonhn|1 year ago
I mean I understand why they did that but it really removes one of the most compelling parts about Otel. We ended doing the hard work of using the standard Otel libraries. I had to contribute a PR or two to get it all to work with our services but am glad that's the route we went because now we can switch vendors if needed (which is likely in the not too distant future in our case.
pranay01|1 year ago
You should try Otel native observability platforms like SigNoz, Honeycomb, etc. your life will be much simpler
Disclaimer : i am one of the maintainers at SigNoz
ljm|1 year ago
I used to really like Datadog for being a one-stop observability shop and even though the experience of integrating with it is still quite simple, I think product and pricing wise they've jumped the shark.
I'm much happier these days using a collection of small time services and self-hosting other things, and the only part of that which isn't joyful is the boilerplate and not really understanding when and why you should, say, use gRPC over HTTP, and stuff like that.
pdimitar|1 year ago
It's extremely easy to self-host, either on a dev machine, a VPS, or in any Docker-based PaaS.
mdaniel|1 year ago
Heaven help you if it's a contrib collector bugfix