The Lost Art of C Structure Packing

[justinplouffe]

Awareness of the layout and size of data is still very common in game development. Mike Acton did a great talk related to this at CppCon a few years ago : https://www.youtube.com/watch?v=rX0ItVEVjHc

[jzelinskie]

This is true for high performance server software, too, especially servers that resemble databases.

I personally find myself thinking about alignment more than I probably should. For Go, I use http://golang-sizeof.tips for experimenting. I've been able to make schema changes to my structs to enable more functionality without increasing memory usage.

[valarauca1]

Lost but not forgotten.

IDK, most these rules are very trivial and fairly simple to automate. Pretty much boils down to declare the smallest fields first, with a handful of exceptions based on the modulus of size.

Ultimately I like Rust’s approach. Without including ‘[repr(C)]/[repr(packed)]’ the compiler is free to layout the structure in the most efficient way for the target platform. And accessing fields with different compiler’s code is UB. You get the speed up, while keeping backwards compatibility.

Sadly C/C++ are shackled to this archaic system. Also I believe Go, Nim, and Swift are as well (this is speculation).

[loeg]

It's not even lost. Any C code that cares about performance will eventually look into how structures are laid out in cache lines.

[millstone]

If switching compilers results in UB, how is that keeping backwards compatibility?

[smeenai]

clang-tidy has a struct padding check which diagnoses non-optimal struct orders, and a coworker contributed a clang tool to reorder fields: https://github.com/llvm-mirror/clang-tools-extra/tree/master...

[nly]

This makes sense if you're only concerned with size. Locality or eliminating false sharing may be more important in large ones

[shortlived]

Not lost if you are programming on a mainframe... just not blogged about.

[boznz]

.. And not applicable to 8-bit embedded programmers.

[kevin_clever]

I tried the example below, saved as `main.c`. It seems she automatically reorders the static mem. Can that behavior be disabled? Can someone please explain?

    #include<stdio.h>
    int main(int argc, char** argv)
    {
    	char *p;
	char c;
	int x;

        printf("char p  %p\n", &p);
        printf("char    %p, offset %i\n", &c, (int)((void*)&c-(void*)&p));
        printf("integer %p, offset %i\n", &x, (int)((void*)&x-(void*)&p));
        return 0;
    }

Output on `gcc main.c; ./a.out`

    char p  0x7ffc5107dff0
    char    0x7ffc5107dfeb, offset -5
    integer 0x7ffc5107dfec, offset -4

[kps]

Those are not structure members. C provides no guarantees about the relative locations of separate variables.

[eloff]

Variables like that don't even have to have unique addresses. They may live only in registers, or if their usage doesn't overlap they can be given the same stack address!

[sigjuice]

Subtraction of arbitrary pointers is not legal.

[ksherlock]

Why would you want to disable that behavior? If you need them in a specific order, you can disable that behavior by using a struct. Ideally, your automatic variables won't have any memory location at all because they're optimized out or kept in registers.

[unwind]

First: you have UB; %p requires void *.

Second: separate variables don't have to appear in memory in source code order. This applies only to struct members, which do.

[bla2]

Those are on the stack, and the stack grows down.

[drake01]

I've found file: unaligned-memory-access.txt useful. Ref:- Linux Kernel Source: UNALIGNED MEMORY ACCESSES Link: https://github.com/torvalds/linux/blob/master/Documentation/...

[ryanmccullagh]

Hint, compile with -Wpacked

[ramzyo]

Have to disagree with you here, totally depends on the platform and if you know what you’re doing. Compiling with this option can lead to unaligned memory accesses. On some architectures, most ARM ones for example, unaligned memory accesses lead to hard faults at runtime.

[banned1]

This is going to sound rude but it is not my intention. The reality is that packing DOES NOT matter for most applications these days.

I read an article about a person who was the last in the world to speak some obscure language. Some people wanted to start a school to teach others to speak that language, wanted the government to pay for it, etc.

The two articles gave me the same combined feel of “I feel a little sorry but not a lot” and “who cares.”

The compiler will take care of most.

Unfortunately for that person on the other article, he is basically screwed.

Don’t let that be you.

[FRex]

The compiler cannot reorder the fields. This stuff comes up occasionally in optimizing layout of language VM structures, compilers, in the Linux kernel and other low level situations.

I.e. JVM compressed object pointers rely on the alignment and the native objects representation is definitely well laid out to not waste space due to alignment:

https://stackoverflow.com/a/25120926

https://blog.codecentric.de/en/2014/02/35gb-heap-less-32gb-j...

Lua implementation similarly cares about alignment in some cases:

http://lua-users.org/lists/lua-l/2009-02/msg00305.html

This is not exactly 'struct packing' but alignment is closely connected to padding so its part of a wider interconnected topic.

[mar77i]

Urm, the mere fact that somebody is capable to micro-manage every different aspect and detail about a program is purely theoretical, as nobody talks about actually doing so, or if, how often that is done.

It just makes them a competent developer/computer scientist. Just like emergency protocols aren't used often by airplane pilots, doesn't mean they're completely useless.

You may be right, as you'll never be trying to make a binary fit an artificially imposed size limit or things like this, as is sometimes necessary in the demo scene. In my experience, having aspiration always outweighs the ignorance of not having it. YMMV.

[CyberDildonics]

I deal with this every day when writing lock free data structures. If you want to really make use of all those cores you can buy now, someone has to pay attention.

[xelxebar]

Only for certain values of "most". As the author mentions, kernels and embedded applications very often need to have tight control over memory layout.

[vardump]

You probably should not be using C.

[abiox]

depending on what you mean by "most applications", that's been the case for 20+ years.

that said, compilers can't "take care" of a lot of things; and either way, people are still needed to develop and maintain those compilers.

there will also always be software that has significant performance requirements, and opportunities to write higher performing software than competitors.