Allocation is cheap in .NET until it is not

[apardoe-MSFT]

"Managed memory is free. Not as in free beer, as in free puppy."

Dev manager of Exchange used that line in a talk. Never were more insightful words spoken. Devs will move from C++ where they obsess about every allocation to .NET and they'll totally forget that allocation is expensive no matter what the platform or runtime.

[rjbwork]

>they'll totally forget that allocation is expensive no matter what the platform or runtime.

Well, it's easier to do in a managed language. When you literally don't have to agonize or obsess over every allocation because you aren't responsible for cleaning it up (unmanaged held resources withstanding), you tend not to do so.

P.S.: You're always free to drop down into C or C++ if you want to get some speed, but of course you need to clean up after yourself there. A friend of mine wrote a good guide on doing so, if anyone cares https://github.com/TheBlackCentipede/PlatformInvocationIndep...

[pcwalton]

This article demonstrates why generational GC with a bump-allocating nursery is so important. Without a semispace copying collector (which is usually impractical without a generational GC) you can't have bump allocation at all. Not having that fast path is a huge performance loss, as this article demonstrates.

[densh]

Immix [1] does bump allocation without the space overhead of the copying young generation. It's been working really well for us in Scala Native.

[1] https://dl.acm.org/citation.cfm?id=1375586

[ridiculous_fish]

Mark-sweep-compact is another GC algorithm that supports bump allocation, and doesn't have the 2x overhead of semispace.

[simooooo]

I didn't understand a bit of that

[narag]

This brought memories of that pattern (flyweight?) where the data was stored outside the objects, possibly in an array. An object was instantiated only to hold an index to the array position and allow access. That's dirty cheap!

[manigandham]

It's still commonly used. The NFX library has Pile which does this well for holding large data:

http://nfxlib.com/book/caching/pile.html

https://www.infoq.com/articles/Big-Memory-Part-1

[pjmlp]

That pattern can be done in .NET via native memory allocation (MarshalInterop and SafeHandles).

With the latest C# 7.x features will become easier to use it.

[aidenn0]

QPX also stored their database in non-GC space in order to prevent the GC from having to walk it.

[pdpi]

It's a common pattern — e.g. a lot of C++ libraries have some sort of string piece type.

[jimjimjim]

good old fashion separating data from logic.

[ridiculous_fish]

How does .NET support pinned pointers with a bump pointer allocator? Does it just eagerly move pinned objects out of the contiguous heap?

[swgillespie]

The .NET GC hands out "allocation contexts" to every thread. An allocation context is little more than two pointers: the bump pointer and the bump pointer limit. If the runtime allocates too much and exceeds the bump pointer limit, it asks the .NET GC for a "quantum" of memory (usually a few KB). Each quantum that the GC gives out is guaranteed to be free of pinned objects - it'll find a contiguous block of memory to hand out.

Pins on the ephemeral segment are generally bad in that the quantum allocator has to be aware of them and squeeze objects between them.

The GC is not permitted to eagerly move pinned objects out of the heap. This is because there are two ways an object can be pinned: a pinning GC handle or a stack scan reports a local as pinned (e.g. the "fixed" keyword in C#). The GC does not know until a GC is already in progress that an object has been pinned and, at that point, it's not legal to move the object so it must stay where it is at the current point in time.

[kevingadd]

Pinning typically just means it is left in place and exempted from compaction. This does mean that you can end up with a performance penalty and nasty holes in your heap layout. Sometimes marshaling code will opt to make a copy of the data instead (and then perhaps pin that), it depends on the type. There's not a lot of explicit documentation on this (probably because some of it is an optimization). Pinned objects can't be moved without breaking semantics - once you get a pinned-type GCHandle to an object, you can just directly get the address and it won't ever change. (I believe once the GCHandle is freed/finalized by the GC, it will automatically unpin the object.)

Typically this isn't a big problem - pinned data structures in .NET code are either pinned for short periods of time (to pass to native code), or are reusable large big buffers that stay pinned forever. Large buffers are always allocated in the large object heap right away. You can always allocate native memory directly in which case the GC doesn't care about it.

This may be changing since recent updates to C# and the runtime have introduced the concept of interior pointers to objects, where you can have a raw pointer to a field within a GCable object. Right now those are constrained to living on the stack only, so the period of time in which the object can't be moved/compacted as a result is relatively short.

[alkonaut]

Is there work ongoing or planned to try to add/improve escape analysis, as the article suggests?

[mwkaufma]

.net already has in-place allocated types (e.g. 'structs') which are dropped on the stack or inside the allocation of their owner (in the case of members fields), which explicitly covers most cases that escape analysis tries to handle implicitly. However, they still need to be heap-alloc'd when boxed (e.g. when used as a IEnumerator or something), which would definitely benefit from hotspot-style allocation-inlining (though the C# compiler already tries to do this prejit for common cases like iterators, too, for practicality's sake).

[kkokosa]

In general - yes: https://github.com/dotnet/coreclr/issues/1784. However, nothing more specific I can say...