JEP draft: Prepare to make final mean final

[GauntletWizard]

When I had the brief displeasure of working on HDFS at Facebook, we took a series of customer meetings to figure out how to get our oldest customers to upgrade their clusters. I was in a meeting with the photos team about what their requirements were and what was blocking them from upgrading, and they were very frank - they asked if the upgrade preserved the internal struct types associated with blocks on the disc servers. They didn't actually use hdfs as a file system, they allocated 1 GB files with zero replication, then used deep reflection to find the extent that comprised them on the discful storage servers, then built their own archival backup file system on top of that. I was horrified. The some of the older hats on the team were less surprised, having had some inkling of what was going on, even though they clearly didn't understand the details. Others considered it tantamount to sacrilege.

I think about this a lot. What they had built was probably actually the best distributed file system within Facebook. It was similarly structured to unraid, and had good availability, durability, and space saving properties, but the approach to engineering was just so wrong headed in my opinion that I couldn't stomach it. Talking about it with other Java programmers within facebook, nobody seemed to mind. Final was just a hint after all.

[adrianmonk]

That reminds me of a quote from some Perl documentation[1]:

> Perl does not enforce private and public parts of its modules as you may have been used to in other languages like C++, Ada, or Modula-17. Perl doesn't have an infatuation with enforced privacy. It would prefer that you stayed out of its living room because you weren't invited, not because it has a shotgun.

It's not exactly the same situation, but the point is, at the end of the day, you need to be able to rely on the people involved being willing to act reasonable. If you can't, then you're going to have problems.

---

[1] https://perldoc.perl.org/perlmodlib

[dapperdrake]

This approach is surprisingly (or unsurprisingly) one of the most robust.

[cogman10]

Hmm, not a bad approach.

I think the one thing that'd be nice is if I could somehow tell the JVM from a class that this class is open for final mutation rather than needing special flags passed into the JVM or special manifests in the Jar. It's often pretty clear to me, as a dev, what I when I need something to have final mutation (generally only with serialization objects).

For example,

    @FinalMutatableByReflection
    class Foo {
      final String bar;
    }

That'd allow me to transition code by just adding an annotation where it needs to be while also getting the benefit that final is really final everywhere else in code that isn't working with serialization.

[nightpool]

They actually have a very similar proposal in the draft already:

    The sun.reflect.ReflectionFactory class only supports deserialization of objects whose classes implement java.io.Serializable

That is, you'll still be able to mutate final fields using the ReflectionFactory class, as long as that class inherits from Serializable

[owlstuffing]

The issue is that many essential libraries and tools rely on setting internal final fields. I assume that's why the options around this have remained open-ended.

The problem with these various "integrity by default" options is that, in most cases, granting access to one effectively grants access to all. For instance, JNI, agent libraries, and JPMS options can each be used to bypass restrictions, making the separation between them largely illusory. Integrity, as framed here, is ultimately binary.

The unfortunate reality of the "integrity by default" crusade is that applications relying on libraries and tools that modify internals will continue to do so. The JDK hasn’t filled any gaps—it has only made an already delicate situation worse.

[sgammon]

One could easily generate the configurations via annotations, and I imagine Micronaut will do just that

[elric]

I've written my fair share of nasty reflexive code for testing or for abusing libraries, but I don't think I've ever overwritten final fields in this way. Private fields, sure. But not final.

Sounds like a good evolution to me.

[TOGoS]

> Application developers can avoid both current warnings and future restrictions by selectively enabling the ability to mutate final fields where essential.

/me raises hand

Maybe if you want to mutate a field, don't mark it `final`?

I know, I know, people like to pretend things are one way and then hand their objects over to some horrid framework that breaks all the rules, because apparently giant web of mutable spaghetti is just fine, not an anti-pattern at all if you let some third-party bull$#!7 ORM/dependency-injection-framework-for-people-who-don't-like-constructors do it.

[exabrial]

I'm 100% onboard with this. My thought was how are they going to make Serialization work, but looks like they thought of that.

I was trying to think of an edge case with JsonB or JAXB that would be affected by this... but generally those frameworks have told you for quite awhile not to do stupid stuff like:

``` @Getter public class HelloMessage { @JsonbProperty private final String helloMessage; } ```

I can't think of any frameworks offhand that do this.

[hyperpape]

Brian Goetz, chief architect of Java, once posted a "what they think I do" vs. "what I actually" do tweet. If I remember correctly, 25% - 50% of the "what I actually do" category was something like "get angry at serialization."

So I think it's safe to say "what about serialization?" is always going to be asked.

[magicalhippo]

So this is about warning about using deep reflection or such to modify fields marked as final.

Not a Java dev, so I thought it might be related to classes marked final somehow. But this seems like a reasonable proposal, at least in spirit.

[bironran]

A cursory glance at "setAccessible" usage reveals popular libraries such as serializers like gson and jaxb, class manipulation and generation like cglib, aspectj and even jdk.internal.reflect, testing frameworks and libraries including junit, mockito and other mocking libraries, lombok, groovy, spring, and the list goes on and on.

My bet is that this will be yet another "checked exception" or "module system", where many applications now need to add "--add-opens". If you'll use ANY of many of the more popular frameworks or libraries you'll end up giving this assurance away, which will make library developers not able to rely on it and we're back to square one.

[pron]

We've addressed that in the JEP. Serialization libraries have a special way to circumvent this (until we get Serialization 2.0), and mocking libraries may, indeed, need to set the flag, but they're rarely used in production, so if they don't enjoy some new optimisation -- no big deal.

BTW, this JEP does not apply to setAccessible generally, as that's been restricted since JDK 16, but only to the particular (and more rare) use of setAccessible to mutate instance final fields. As the JEP says, static final fields, records' internal instance fields, and final instance fields of hidden classes cannot be mutated with that approach currently, so it's never been something that's expected to work in all cases.

[PathOfEclipse]

setAccessible is also used to be able to access private fields, and not just to be able to write to final fields. Most libraries shouldn't need to set final fields, and I say this as someone who was very against when they deprecated java.lang.misc.Unsafe. I've only had to set a final field once in my career and it was related to some obscure MySql/JDBC driver bug/workaround. This particular deprecation seems very sensible to me.

[merb]

Yeah like the module system. Looks good on paper, is probably hard to deal with. There are still tons of popular libraries that have no module-info. Java does evolve, but the direction it does is so weird. And than the tooling is strange and it’s worse that there are basically two build tools, both with their upsides and downsides but they still feel more complicated than tools for other languages like cargo, go (if you consider that), msbuild (the modern csproj stuff/slnx)

[PaulHoule]

My impression is that this will be painful for the code I work on because the libraries you mention depend on being able to modify private and/or final fields.

[theanonymousone]

If there was a r/nottheonion for tech :D

Jokes aside, I thought the ability to mutate final fields was already removed/restricted after Java 17 :/

[nightpool]

    [Speculative optimizations] may not suffice in this case as future planned optimizations may wish to rely not only on immutability within the lifetime of the process, but also on the immutability of fields from one run of the application to the next.

Can someone elaborate a little more on what this means? I'm very surprised to hear that this was considered a blocker important enough to add all of this complicated machinery (and breaking several deserialization libraries...), when I've never even heard of such an optimization and can't imagine what sort of form it would take

[pron]

There's ongoing work, as part of Project Leyden, to cache certain computations -- either performed by the user code or the JVM -- from one run of the program to the next, including the caching of JIT-compiled machine code. The early parts of this work were discussed here: https://www.morling.dev/blog/jep-483-aot-class-loading-linki...

[xxs]

This is rather old[0] but still relevant.

Surprise #2! A popular open-source framework writes to final fields after object construction via generated bytecodes!... This optimization of the final field is *the* main optimization performed on final fields.

[0] https://web.archive.org/web/20121016082428/http://www.azulsy...

[Misdicorl]

I suppose serializing the JVM state itself to avoid the cold start problem might take advantage of this?

[mberning]

If they implement this in a way similar to the package visibility changes your list of JVM args is about to explode in order to support legacy apps.

[jjmarr]

From my perspective as a C++ developer, every attempt to use `const` for compiler optimization appears to be stymied by the existence of `const_cast`, because modifying a `const` value is only undefined behaviour if the underlying object is `const`. Glad to see that Java is willing to break the language to improve it.

[PhilipRoman]

The implementation of "const" in C/C++ is really annoying, in the end all you get is a bit of semantic documentation and a bunch of compiler errors or casts whenever some library doesn't use it. You can often trick the compiler into stronger optimizations by making a local variable marked as "const" (in which case it is a "true const") and copying the value to/from it.

[mb7733]

> modifying a `const` value is only undefined behaviour if the underlying object is `const`.

I found this sentence confusing. You mean that modifying a value that has been const_cast is undefined behaviour only if the original variable was const right? Or something else?

[immibis]

I don't see the point in putting another guard rail behind the override that's supposed to remove all guard rails. Java doesn't even have a security model any more, so what are you protecting?

Programmers from their own mistakes? That's fine, but this is about cases where they set the "I really mean it and this isn't a mistake" flag.

The JVM's optimized code from programmers? Plausible, but aren't there already many cases where things get deoptimized based on run-time state changes?

It feels like someone just said "final means final!" without really thinking about the purpose of a JVM. Will there be a proposal to enforce checked exceptions, too?

[Traubenfuchs]

I would like to see some realistic example scenarios where this could actually lead to a speedup and how much it actually speeds up.

[kelnos]

The article mentions one: const folding. I don't think we need a benchmark to suggest that could mean a performance improvement in some cases.

Regardless, to me this isn't about performance, this is about "integrity" (to use the same term as in the JEP): consumers of my library should not be mucking about in private implementation details, and then inevitably complaining about problems to me when something breaks. If you need a feature that I don't expose, ask for it (or better yet, submit a patch).

Sure, I've used reflection to modify library internals before, but I recognize that whenever I do that I'm inviting a maintenance headache into my world. But some people just think things should always work, even when they are breaking them.

[aardvark179]

So happy to see this, thank you Ron and Alan.

[ars]

Does this mean I should start marking my variables (and function parameters) with Final?

Up till now I always assumed the compiler would figure out on its own which variables were final, and optimize as needed. But this JEP makes it seem like there are optimizations that only happen if you manually mark the variable.

[pron]

No, this JEP only talks about fields; it has no impact whatsoever on locals. There's no positive or negative impact on performance when making locals final or not.

[satyanash]

> Prepare to make final mean final

missed opportunity to call it "final final"

[keybored]

I don’t see a way to opt-in to a hard error if this happens somewhere in the guts of the code (third-party probably).

[steveklabnik]

> --illegal-final-final-mutation=deny will result in Field::set throwing an IllegalAccessException for every illegal final field mutation.

Seems like this way?

[stickfigure]

Great! Now can we make `final` the default for all fields, variables, and parameters?

(yes yes, I know, that would break syntax... but please come up with something to discourage mutability)

[magicalhippo]

Const-ness in C++ is something I miss in other languages. Being immediate able to see that this function or method couldn't mutate the object made it so much easier to reason about the code.

Yeah I know there's ways around it, but then the author known what they told the other party to expect.

[nirvdrum]

I'd be happy with a `val` keyword. I thought that was a seriously missed opportunity when they introduced `var`.

[xxs]

>but please come up with something to discourage mutability)

Records?

[duskwuff]

  What man that sees the ever-whirling wheel
  Of Change, the which all mortal things doth sway,
  But that thereby doth find, and plainly feel,
  How Mutability in them doth play
  Her cruel sports to many men's decay?

(Edmund Spenser, 1596)

[throwaway92422]

Records in Java have final fields.

[xxs]

Around 19 year late, still better than never.

[Almondsetat]

final ly