[johnnyjeans]

I'm not surprised to see that Jank's solution to this is to embed LLVM into their runtime. I really wish there was a better way to do this.

There are a lot of things I don't like about C++, and close to the top of the list is the lack of standardization for name-mangling, or even a way mangle or de-mangle names at compile-time. Sepples is a royal pain in the ass to target for a dynamic FFI because of that. It would be really nice to have some way to get symbol names and calling semantics as constexpr const char* and not have to deal with generating (or writing) a ton of boilerplate and extern "C" blocks.

It's absolutely possible, but it's not low-hanging fruit so the standards committee will never put it in. Just like they'll never add a standardized equivalent for alloca/VLAs. We're not allowed to have basic, useful things. Only more ways to abuse type deduction. Will C++26 finally give us constexpr dynamic allocations? Will compilers ever actually implement one of the three (3) compile-time reflection standards? Stay tuned to find out!

[benreesman]

Carmack did very much almost exactly the same with the Trinity / Quake3 Engine: IIRC it was LCC, maybe tcc, one of the C compilers you can actually understand totally as an individual.

He compiled C with some builtins for syscalls, and then translated that to his own stack machine. But, he also had a target for native DLLs, so same safe syscall interface, but they can segv so you have to trust them.

Crazy to think that in one computer program (that still reads better than high-concept FAANG C++ from elite lehends, truly unique) this wasn't even the most dramatic innovation. It was the third* most dramatic revolution in one program.

If you're into this stuff, call in sick and read the plan files all day. Gives me googebumps.

[no_wizard]

Carmack actually deserves the moniker of 10x engineer. Truly his work in his domain has reached far outside it because id the quality of his ideas and methodologies

[MangoToupe]

Linking directly to C++ is truly hell just considering symbol mangling. The syntax <-> semantics relationship is ghastly. I haven't seen a single project tackle the C++ interface in its entirety (outside of clang). It nearly seems impossible.

There's a reason Carmack tackled the C abi and not whatever the C++ equivalent is.

[johnnyjeans]

Any particular year?

[Jeaye]

I hear you when it comes to C++ portability, ABI, and standards. I'm not sure what you would imagine jank using if not for LLVM, though.

Clojure uses the JVM, jank uses LLVM. I imagine we'd need _something_ to handle the JIT runtime, as well as jank's compiler back-end (for IR optimization and target codegen). If it's not LLVM, jank would embed something else.

Having to build both of these things myself would make an already gargantuan project insurmountable.

[almostgotcaught]

> LLVM into their runtime

they're not embedding LLVM - they're embedding clang. if you look at my comment below, you'll see LLVM is not currently sufficient.

> [C++] is a royal pain in the ass to target for a dynamic FFI because of that

name mangling is by the easiest part of cpp FFI - the hard part is the rest of the ABI. anyone curious can start here

https://github.com/rust-lang/rust-bindgen/issues/778

[Jeaye]

To be fair, jank embeds both Clang and LLVM. We use Clang for C++ interop and JIT C++ compilation. We use LLVM for IR generation and jank's compiler back-end.

[johnnyjeans]

> they're not embedding LLVM - they're embedding clang

They're embedding both, according to the article. But it's also just sloppy semantics on my part; when I say LLVM, I don't make a distinction of the frontend or any other part of it. I'm fully relying on context to include all relevant bits of software being used. In the same way I might use "Windows" to refer to any part of the Windows operating system like dwm.exe, explorer.exe, command.com, ps.exe, etc. LLVM a generic catch-all for me, I don't say "LLI" I say "the LLVM VM", for example. I can't really consider clang to be distinct from that ecosystem, though I know it's a discrete piece of software.

> name mangling is by the easiest part of cpp FFI

And it still requires a lot of work, and increases in effort when you have multiple compilers, and if you're on a tiny code team that's already understaffed, it's not really something you can worry about.

https://en.m.wikiversity.org/wiki/Visual_C%2B%2B_name_mangli...

You're right, writing platform specific code to handle this is more than possible. But it takes manhours that might just be better spent elsewhere. And that's before we get to the part where embedding a C++ compiler is extremely inappropriate when you just want a symbol name and an ABI.

But this is besides the point: The fact that it's not a problem solved by the gargantuan standard is awful. I also consider the ABI to be the exact same issue, that being absolutely awful support of runtime code loading, linking and interoperation. There's also no real reason for it, other than the standards committee being incompetent.

[o11c]

> the lack of standardization for name-mangling, or even a way mangle or de-mangle names at compile-time.

Like many things, this isn't a C++ problem. There is a standard and almost every target uses it ... and then there's what Microsoft does. Only if you have to deal with the latter is there a problem.

Now, standards do evolve, and this does give room for different system libraries/tools to have a different view of what is acceptable/correct (I still have nightmares of trying to work through `I...E` vs `J...E` errors) ... but all the functionality does exist and work well if you aren't on the bleeding edge (fortunately, C++11 provided the bits that are truly essential; everything since has been merely nice-to-have).

[mort96]

Like many things people claim "isn't a C++ problem but an implementation problem"... This is a C++ problem. Anything that's not nailed down by the standard should be expected to vary between implementations.

The fact that the standard doesn't specify a name mangling scheme leads to the completely predictable result that different implementations use different name mangling schemes.

The fact that the standard doesn't specify a mechanism to mangle and demangle names (be it at runtime or at compile time) leads to the completely predictable result that different implementations provide different mechanisms to mangle and demangle names, and that some implementations don't provide such a mechanism.

These issues could, and should, have been fixed in the only place they can be fixed -- the standard. ISO is the mechanism through which different implementation vendors collaborate and find common solutions to problems.

[rs186]

> There is a standard and almost every target uses it ... and then there's what Microsoft does. Only if you have to deal with the latter is there a problem.

Sounds like there isn't a standard, then.

[plq]

> the lack of standardization for name-mangling

I don't see the point of standardizing name mangling. Imagine there is a standard, now you need to standardize the memory layout of every single class found in the standard library. Without that, instead of failing at link-time, your hypothetical program would break in ugly ways while running because eg two functions that invoke one other have differing opinions about where exactly the length of a std::string can be found in the memory.

[johnnyjeans]

The naive way wouldn't be any different than what it's like to dynamically load sepples binaries right now.

The real way, and the way befitting the role of the standards committee is actually putting effort into standardizing a way to talk to and understand the interfaces and structure of a C++ binary at load-time. That's exactly what linking is for. It should be the responsibility of the software using the FFI to move it's own code around and adjust it to conform with information provided by the main program as part of the dynamic linking/loading process... which is already what it's doing. You can mitigate a lot of the edge cases by making interaction outside of this standard interface as undefined behavior.

The canonical way to do your example is to get the address of std::string::length() and ask how to appropriately call it (to pass "this, for example.)

[duped]

This standard already exists, it's called the ABI and the reason the STL can't evolve past 90s standards in data structures is because breaking it would cause immeasurable (read: quite measurable) harm

Like, for fuck's sake, we're using red/black trees for hash maps, in std - just because thou shalt not break thy ABI

[Someone]

I would think name mangling is out of scope for a programming language definition, more so for C and C++, which target running on anything under the sun, including systems that do not have libraries, do not have the concept of shared libraries or do not have access to function names at runtime.

> It would be really nice to have some way to get symbol names and calling semantics

Again, I think that’s out of scope for a programming language. Also, is it even possible to have a way to describe low level calling semantics for any CPU in a way such that a program can use that info? The target CPU may not have registers or may not have a stack, may have multiple types of memory, may have segmented memory, etc.

[kazinator]

> embed LLVM into their runtime

That comically reads like "embed a blue whale into your hammock".

[kccqzy]

> de-mangle names at compile-time

Far from being standardized but it's possible today on GCC and Clang. You just abuse __PRETTY_FUNCTION__.

[dataflow]

That's not demangling a mangled name, it's retrieving the unmangled name of a symbol.