Great question! Perforator indeed looks similar to Pyroscope. However, we think that the closest existing solutions are https://parca.dev, closed-source Google Wide Profiling, and, speaking of the agent, the beautiful OpenTelemetry eBPF profiler. The main technical differences with Pyroscope we see are:
- Pyroscope's Java support is superior as of now because Pyroscope offloads it to the amazing async-profiler.
- Pyroscope expects native binaries to be compiled with frame pointers: https://grafana.com/docs/pyroscope/latest/configure-client/g.... This is often not the case, and that's the problem we've tried to solve with Perforator. Perforator uses .eh_frame, which is nearly universal and does not impose additional requirements on compiled binaries.
- Pyroscope symbolizes profiles on an agent, while Perforator symbolizes profiles offline, greatly reducing symbolization costs and agent's overhead. It seems Pyroscope is heading toward the same architecture we use: https://github.com/grafana/pyroscope/pull/3799.
- Perforator can be (and should be!) run as a standalone replacement for perf record.
- Perforator supports sPGO profiles.
In summary, we try to implement native profiling almost perfectly. It's worth noting that Pyroscope is a mature, well-established product that integrates excellently with the Grafana ecosystem. We have just focused on different things: our focus has been on optimizing native code profiling and making it as accurate and low-overhead as possible.
BigRedEye|1 year ago
- Pyroscope's Java support is superior as of now because Pyroscope offloads it to the amazing async-profiler.
- Pyroscope expects native binaries to be compiled with frame pointers: https://grafana.com/docs/pyroscope/latest/configure-client/g.... This is often not the case, and that's the problem we've tried to solve with Perforator. Perforator uses .eh_frame, which is nearly universal and does not impose additional requirements on compiled binaries.
- Pyroscope symbolizes using symtab: https://grafana.com/docs/pyroscope/latest/configure-client/g.... We use DWARF/GSYM to get as correct and verbose stacks as possible (we benchmark our stacks against stacks from gdb).
- Pyroscope symbolizes profiles on an agent, while Perforator symbolizes profiles offline, greatly reducing symbolization costs and agent's overhead. It seems Pyroscope is heading toward the same architecture we use: https://github.com/grafana/pyroscope/pull/3799.
- Perforator can be (and should be!) run as a standalone replacement for perf record.
- Perforator supports sPGO profiles.
In summary, we try to implement native profiling almost perfectly. It's worth noting that Pyroscope is a mature, well-established product that integrates excellently with the Grafana ecosystem. We have just focused on different things: our focus has been on optimizing native code profiling and making it as accurate and low-overhead as possible.