Go has added Valgrind support

[bracewel]

Author of the linked CL here: we added this mostly so that we could abuse the memory initialization tracking to test the constant-time-ness of crypto code (similar to what BoringSSL does, proposed by agl around fifteen years ago: https://www.imperialviolet.org/2010/04/01/ctgrind.html), which is an annoyingly hard property to test.

We're hoping that there are also a bunch of other interesting side-effects of enabling the usage of Valgrind for Go, in particular seeing how we can use it to track how the runtime handles memory (hopefully correctly!)

edit: also strong disclaimer that this support is still somewhat experimental. I am not 100% confident we are properly instrumenting everything, and it's likely there are still some errant warnings that don't fully make sense.

[chrsig]

w.r.t. your edit: Is there anything the community at large can do to aid your efforts?

[on_the_beach]

This is super cool. Hopefully it will flush out other issues in Go too.

But I wonder why its not trivial to throw a bunch of different inputs at your cyphering functions and measure that the execution times are all within an epsilon tolerance?

I mean, you want to show constant time of your crypto functions, why not just directly measure the time under lots of inputs? (and maybe background Garbage Collection and OS noise) and see how constant they are directly?

Also some CPUs have a counter for conditional branches (that the rr debuger leverages), and you could sample that before and after and make sure the number of conditional branches does not change between decrypts -- as that AGL post mentions branching being the same is important for constant time.

Finally, it would also seem trivial to track the first 10 decrypts, take their maximum time add a small extra few nanoseconds tolerance, and pad every following decrypt with a few nanoseconds (executing noops) to force constant time when it is varying.

And you could add an assert that anything over that established upper bound crashes the program since it is violating the constant time property. I suppose the real difficulty is if the OS deschedules your execution and throws off your timing check...

[pjmlp]

> Instead of adding the Valgrind headers to the tree, and using cgo to call the various Valgrind client request macros, we just add an assembly function which emits the necessary instructions to trigger client requests.

Love that they have taken this route, this is the way bootstraped toolchains should be, minimal building blocks and everything else on the language itself.

[giancarlostoro]

I am still curious, had they not gone this route, and avoided the other two routes mentioned, what could they have done to make this process as simple as the rest of Go tends to be, and nearly as performant? I guess this is an ongoing question to be solved at a future date.

[chrsig]

I'm glad to see rsc still actively involved. And commenting on commit messages.

The older I get the more I value commit messages. It's too easy to just leave a message like "adding valgrind support", which isn't very useful to future readers doing archaeology.

[pstuart]

rsc is a rock star! I believe his focus now is on using AI to manage issues and PRs and such -- I'm sure it will bear copious fruit.

[amelius]

It only works if every package tests with it.

Otherwise the relevant warnings get swamped by a huge amount by irrelevant warnings.

This is why running Valgrind on Python code does not work.

[jzwinck]

If that were true it would also apply to C and C++. I have used Valgrind with Python + Boost C++ hybrid programs and it worked fine after spending an hour making a suppressions file.

[esbranson]

Simplification of overwhelming information sounds like a good use case for local LLMs. So I agree with other comments that toolchains are better positioned to include batteries like Valgrind.

[Thaxll]

What do you mean if every package tests with it in the context of Go?

[rwmj]

Valgrind is a hidden super-power. In much of the software I write, there's 'make check' which runs the test cases, and 'make check-valgrind' that runs the same test cases under valgrind. The latter is only used on developer machines. It often reveals memory leaks or other subtle memory bugs.

[stingraycharles]

Somewhat yes, but as soon as you enter the world of multi-threading (which Go does a lot), the abstraction doesn’t work anymore: as I understand it (or rather, understood: last time I really spent a lot of time digging into it with C++ code was a while ago) it uses its own scheduler, and as such, a lot of subtle real world issues that would arise due to concurrency / race conditions / etc do not pop up in valgrind. And the performance penalty in general is very heavy.

Having said that, it saved my ass a lot of times, and I’m very grateful that it exists.

[DishyDev]

Very cool. Should flush out a few bugs.

I'd be interested to know why Valgrind vs the Clang AddressSanitizer and MemorySaniziter. These normally find more types of errors (like use-after-return) and I find it significantly faster than Valgrind.

[tasn]

Go doesn't use clang/llvm, so they can't use these tools.

[acidx]

Go has had its own version of msan and asan for years at this point.

[cirelli94]

I'm interested too. I'm using a Go program that call a cpp library with SWING and I was interested in find out if that library had a memory leak, or maybe the SWING wrap I wrote. But this kind of problem can't be detected via pprof, so I tought, what if Go support Valgrind?? and find out this changes.

I'm not sure if this will work though, will it @bracewel?

[yxhuvud]

Valgrind also does stuff like memory tracking and memory-profiling, so this is great also from a performance tracking point of view.

[1718627440]

Valgrind is way faster and can be attached to a running program.

[defraudbah]

looks very promising, one of the biggest issue in golang for me is profiling and constant memory leaks/pressure. Not sure if there is an alternative of what people use now

[felixge]

I'd love to hear more! What kind of profiling issues are you running into? I'm assuming the inuse memory profiles are sometimes not good enough to track down leaks since they only show the allocation stack traces? Have you tried goref [1]?. What kind of memory pressure issues are you dealing with?

[1] https://github.com/cloudwego/goref

Disclaimer: I work on continuous profiling for Datadog and contribute to the profiling features in the runtime.

[0x696C6961]

How are you getting "constant memory leaks" in a GC'd language?

[pjmlp]

Ideally, they would have learnt from other languages, and offered explicit control over what goes into the stack instead of relying into escape analysis alone.

As it is, the only way to currently handle that is with " -gcflags -m=3" or using something like VSCode Go plugin, via "ui.codelenses" and "ui.diagnostic.annotations" configurations.

[sethammons]

> constant memory leaks/pressure

In Go, never launch a goroutine that you don't know exactly how it will be cleaned up.

[Thaxll]

pprof is pretty good, what do you need?

[tasn]

This feels more like a failure than a win.

Don't get me wrong, I love Valgrind, and have been using it extensively in my past life as a C developer. Though the fact that Go needs Valgrind feels like a failure of the language or the ecosystem. I've been doing Rust for ~6 years now, and haven't had to reach for Valgrind even once (I think a team member may have use it once).

I realize that's probably because of cgo, and maybe it's in-fact a step forward, but I can help but feel like it is a step backwards.

[JyB]

I never understand why there's always one of the top comment on every Go post being derogatory and mentioning Rust. It never fails. It starts to feel like a weird mix of defensiveness and superiority complex.

[bracewel]

This is mainly actually for testing constant-time code, rather than doing proper memory tracking (see https://www.imperialviolet.org/2010/04/01/ctgrind.html for a slightly out-of-date description of this technique).

[stusmall]

I've had to use valgrind a bit in Rust. Not much but the need is there. It really depends on the product you are working with. Rust is an extremely flexible language and can be used in many spaces. From high level abstract functional code to low level C-like code on a microcontroller. Some use cases would never image using unsafe, some can't live without it. For most FFI to C is just a fact of life, so it comes up somewhere.

When I used it before I was working on a Rust modules that was loaded in by nginx. This was before there were official or even community bindings to nginx.... so there was a lot of opportunity for mistakes.

[pjmlp]

Depends on how much unsafe you actually happen to write, use unsafe crates, or link into C and C++ libraries.

I also seldom need something like this in Java, .NET or node, until a dependency makes it otherwise.

[paulf38]

That's quite nice. There is a small risk that the client request mechanism might change . The headers don't change much - mostly when a new platform gets added. Go is only targeting amd64 and arm64.

This isn't so much about leaks. The most important thing that this will enable is correct analysis of uninitialised memory. Without annotation memory that gets recycled will not be correctly poisoned. I imagine that it will also be useful for the other tools (except cachegrind and callgrind).

[suobset]

Not even a Go user, and yet this is one of the best things I have read today morning. Valgrind is possibly one of the most powerful tools I have in my belt!!

[DarkNova6]

Would you mind to elaborate? I don't program in C but it sounds interesting.

[olivia-banks]

I love Valgrind, but since my main development machine is an M3, I don’t get to use it nearly as much as I would like.

[jasonjmcghee]

https://github.com/LouisBrunner/valgrind-macos

[0x696C6961]

This is only useful for cgo correct?

[kevincox]

I presume it is also useful if you are using the unsafe APIs as well to mess with pointers and do raw memory reads.

[9rx]

It's really for crypto. https://news.ycombinator.com/item?id=45348445

But maybe others will find a way to use it. Who knows?

[holyknight]

damn, i remember using valgrind when writing C in university a long time ago.

[willy_k]

And I’m going to be using Valgrind in a few weeks, writing C in university now.

[pjmlp]

I remember when it came to be, and I was already working. Yep feeling gray.

[starboyy]

oh man. you came at the right time.

[alias_neo]

C is also half a century old. Perhaps Valgrind wasn't a high priority because the newer language also has newer tools?

Nothing wrong with adding tried and tested tools later if people want them.

Did you have a need for Valgrind in Go that wasn't served by any other tools until now?

[worksonmine]

> Next up: maybe Go will discover gdb integration and we can debug like it's 1999

That's already possible and documented[1]. I don't understand if you're sarcastic though, what's wrong with GDB? I use it in vim :termdebug and I wish all languages had native support for it.

[1]: https://go.dev/doc/gdb

[preisschild]

And yet it's mature enough to be used for highly critical software such as Kubernetes...