top | item 45772064

Immutable releases are now generally available on GitHub

153 points| fastest963 | 4 months ago |github.blog

62 comments

order

hoistbypetard|4 months ago

My instant reaction was: "Wait?! They weren't immutable before?"

I'm glad they're doing this, and it's an unpleasant surprise that they didn't already work this way. I don't understand why they allow mutable releases.

danudey|4 months ago

We've had a few issues in the past where a file wasn't updated during the release process, requiring us to re-publish one small change. It's not optimal, but given the nature of our release process being long and complex it's a lot easier to fix the asset in three minutes rather than spend an entire day re-building and re-publishing an entire release, updating our docs, etc. just because of one line in one file that wasn't updated correctly.

hk1337|4 months ago

Git tags aren’t even really immutable, they’re treated as such but they’re not.

johnisgood|4 months ago

Yeah, how did it work before that it was not immutable?!

> With immutable releases, assets and tags are protected from tampering after publication

I really, really wonder how it worked before. Can anyone explain?

edflsafoiewq|4 months ago

Mutable releases are used for continuous/nightly builds.

GuestFAUniverse|4 months ago

+1

Nobody thought about mutable releases being utterly bad _before_? Baffles me...

As bad as hardware vendors selling products with different chips inside as the same model (hello Cisco -- at least in former times; hello HP, formerly selling at least three different, _incompatible_ laptop power supplies with the same label).

Mutability: surprise, surprise, I'm not what you expected! -- maybe one of IT's worst ideas.

dare944|4 months ago

> When you enable immutable releases, the following protections are enforced: • Git tags cannot be moved or deleted; • Release assets cannot be modified or deleted

On the face of it, this seems like a non-starter. If a particular immutable release represents a danger to the consumer (extreme example: the software contains a bug that could result in physical injury) one must have the ability to retract that release so that no further consumers of the software could be affected by it. It makes sense that a retraction of an immutable release should not be reversible in such a way that the release could be recreated with different contents. But retractions must be possible, for both ethical and legal reasons.

I would also argue that its not sufficient to simply apply a blanket "deny all" access control to dangerous releases (assuming such a mechanism exists), as this does not adequately convey the deprecating nature of the change (and as a result, could mistakenly be reversed in the future). Ideally the retraction itself would be immutable such that once retracted the release is inaccessible forever.

Now, it may be that all this is supported by the new feature; I haven't had the chance to test it yet. But nothing in the documentation makes this clear one way or another.

weinzierl|4 months ago

I think there are compelling reasons to support:

1. Unremovable

2. Uninstallable while keeping the data available

3. Removing the release completely (while keeping an audit log that this happened)

1 is for use-cases where availability trumps security. I'd argue this should never be the case but at the same time it is how our world ticks by and large. Hard to take this away from people.

2 is for security, forensics and heritage but at the cost of availability. Uninstallable could mean to only offer the artifacts in an archive.

3 must always be possible as a last resort for illegal content that slipped through all previous safeguard layers.

rhodey|4 months ago

I am glad for this feature

If I have anyone's attention there is something related I would like to see

Please add a small thing which users can look for on the public: repo/actions page

This small thing should let users know the action was run by github like is default and not run on a custom / private action runner

The private action runner feature makes sense but many projects tell users to look to the github action history to trust that tests A, B, C passed. If the github action ran on a private action runner then you really cannot trust that what is in e.g. run.yml actually ran

The attestation feature can be used to prove that an action was run by github and not by private / custom but users need to install the github cli to validate attestations and this is a heavy ask when I think an addition icon on repo/actions page or a diff icon color will do better

rhodey|4 months ago

I am seeing some docs now that suggest

> runs-on: [self-hosted, ...]

Must be added to run.yml to use custom / private action runners

I did not find these docs last time I looked and so my feature request may be already fulfilled

If anyone wants to chime in to say that `runs-on` can be relied on or not I would be grateful

eviks|4 months ago

Why is deletion not allowed, which supply chain attacks work by deleting a release, not changing it to a malicious one?

danudey|4 months ago

1. A release turns out to contain an exploitable bug

2. A release is published to fix the bug

3. Someone malicious with access deletes the release

4. Everyone downloading the "latest" version gets the exploitable version until the developers notice and re-publish again

I think about tools used in CI systems that are often re-downloaded in each run, like `helm` or `kubectl` or `crane` for example; if they're pinning a previous version they stay exploitable, and if they're downloading the 'latest' from Github then this switcheroo keeps them exploitable. Given that a lot of emergency security releases come with disclosure ("this is being released to resolve CVE-2025-12345") another 12 hours of exploitability can be critical.

kbolino|4 months ago

Deletion creates a hole. The hole can be filled by something else. This is a form of mutation.

What you probably want instead is one-way revocation. You place a permanent marker that says "do not use this release because it is {broken, malicious, ...}".

hiccuphippo|4 months ago

I'd guess one MO is to delete a malicious package/url shortly after releasing it to prevent researchers from getting to it.

yjftsjthsd-h|4 months ago

I assume they're doing the trivial workaround to prevent renaming by way of delete and recreate?

josephcsible|4 months ago

This seems too strong to me, especially the prohibition on deletion. Why couldn't they have instead just showed a record of modifications to releases, and/or embedded a hash of the contents in the URL to them?

lrvick|4 months ago

Just remember the only truly immutable release is one signed by a key the host does not have access to, or one where you pin a hash locally at the point of consumption.

Microsoft does not have strict third party code review policies internally, has been hit with supply chain attacks before, and will be hit again. Consider this a nice to have feature, but give it zero trust.

raphinou|4 months ago

As I'm working on a signing scheme for release authentication, this is a welcome news.

To alleviate the issue of mutable releases I had set up a mirror of releases checksums to be able to detect releases alterations. This is not needed anymore for immutable releases.

And automatically publishing checking of releases artifacts is also a good recent change by GH: in that project mentioned above I have developed a cli downloader checking the checksums of the downloaded file [1], but to be useful, it required the project to publish checksums, and the project to be mirrored. Now both of these requirements are dropped and the tool is readily useful for all GitHub immutable releases.

1: https://github.com/asfaload/asfald/

shpx|4 months ago

Would've made more sense to add a grey "Edited" to edited releases. Releases are not actually immutable, GitHub could change them. I don't know why you need to use sciency words to say "editing disabled".

NoahZuniga|4 months ago

They are immutable! The releases are signed with an attestation from a trusted third party that Github can't forge! Also these attestations are public and anyone can verify that the signing third party isn't misbehaving.

> Release attestations let you verify that an artifact is authentic and unchanged, even outside GitHub. Attestations use the Sigstore bundle format, so you can easily verify releases and assets using the GitHub CLI or integrate with any Sigstore-compatible tooling to automate policy enforcement in your CI/CD pipelines. For instructions on how to verify the integrity of a release, see our docs on verifying the integrity of a release.

They are using Sigstore, which is pretty standard in this space.

mlhpdx|4 months ago

I love the idea but I fear the implementation. I don’t use GitHub actions for builds and upload ancillary artifacts to releases. I guess I know what will be my distraction today.

IshKebab|4 months ago

Well I guess they're doing something at least. I kind of thought Github had moved to maintenance mode. Maybe we'll get stacked PRs one day?

OptionOfT|4 months ago

And yet I received an update from

        uses: github/codeql-action/init@8a06050a8c0348fb4738f28e0cfbb6727cf054ce # v4.31.2
to

        uses: github/codeql-action/init@0499de31b99561a6d14a36a5f662c2a54f91beee # v4.31.2
So someone must've moved the tag, even though that release is supposed to be immutable. https://github.com/github/codeql-action/releases

fastest963|4 months ago

That release isn't immutable

skrrtww|4 months ago

Uh, does this apply to the autogenerated source code artifacts? Those are famously not stable because they are generated on-demand with `git archive`. The value of this feature is really undermined if they don't also provide a source code download with a stable hash.

bob1029|4 months ago

I struggle to get excited about this sort of thing when the most essential functions of GitHub are falling apart. Reviewing PRs has somehow gotten even worse since the original react update.

I think the only thing that would fix this issue is for them to lose 20%+ of their customers to a competitor. Something very simple that can vacuum up the GHES migration archive and proceed as if it were 2018 again.

I'd be willing to completely sacrifice actions, project boards, copilot, et. al. if it meant I could have ultra fast views into code, issues and pulls. I really see no reason the PR view cannot be pre-rendered on the server when the branch is pushed each time. This should be an instantaneous response at review time. I don't care if it's 5 megabytes of diff - If my browser can handle the react slop, it can certainly handle a big chunk of static DOM.

lubujackson|4 months ago

If it happens, it would likely be fully integrated versioning in some LLM product.