Like what is the problem? Thanks to this stack trace your business logic fits into two lines. Do you want to have lines and lines of your own transaction manager? Lines and lines of your own DI framework? Do you suggest to write a HTTP web server yourself? Why don't we count stack frames in kernel space? And what do we have in JVM underneath us? And libc/Win32 layer?
I'm really annoyed by this kind of things like they have any meaning except "I don't know shit about how abstractions work in computers"
Why don't we count stack frames in kernel space? And what do we have in JVM underneath us? And libc/Win32 layer?
You even mentioned it yourself: this is the additional overhead added by the Java ecosystem on top of what may already be a pretty large stack of stuff. (From what I've seen, libc is pretty shallow. The majority are either leaf functions like strlen(), or <5 levels from a leaf/before execution disappears into a system call.)
Even if the overhead is not much to a machine, it is certainly going to have an effect on the human who has to figure out what's going on. When you write such code you may not think this way, but every piece of code is a possible place for a bug to be.
From my short experience with Enterprise Java years ago, such deep callstacks are usually symptomatic of code that does far more indirection than actual work --- hundreds of tiny methods that have maybe a statement or two and then call another one. I understand that it might feel good and even better to write code like this, but that simplicity is deceptive: you aren't looking at the whole picture and just focusing on micro-simplicity, when it's macro-simplicity that really counts.
This means that when you're trying to track down a bug, the "interesting parts" are scattered in tiny pieces across dozens of files, and it increases the cognitive overhead significantly in having to piece everything back together. You might not realise that something is wrong until you're deep inside, and then you have to jump back several levels above in the callstack to figure out where things went wrong, seeking in and between files to trace out the execution flow.
I think you're reading into this too much. It's posted because it's interesting, and something Java developers might not think about very often. The author doesn't imply there's any problem.
"Java EE is a lot about abstractions which I have grown to appreciate over the years. However, I find this very difficult to explain to my colleague who sits just across the room – he is a Mainframe veteran with tons of experience :)"
Debugging these stack traces is easy, just look for your packages and classes, and “caused by”. You should never be debugging framework code. Well almost never.
> Thanks to this stack trace your business logic fits into two lines.
Two function calls for business logic, a hundred dependency libraries, thousands of lines of stateful "initialization" code outside of the stack and a fuckzillion XML files for configuration. But yeah, everything's great!
Expressiveness is not always a pure ideal; reducing your business logic to two lines of code comes with tradeoffs. The ways your code work within the frameworks you choose is not always obvious, and when something goes wrong, it's harder to point at one place within your two lines of code and say: this must be it.
The opposite is not, and should not, necessarily that you write your own versions of things; after all, if you did so, you'd still have as deep of a stack trace as you do with the third-party code!
I guess, if I encountered this stack trace, I might say there's something of a code smell there. But, hey--this is also fairly old code, and I suspect things are Better Now, so let's not be so quick to cast stones!
The problem is it can be extremely difficult to map those stack trace calls to concrete Java code, because so much of a Spring application is generated from an annotation, or code getting called simply because a dependency exists on the classpath. Sometimes classes appear out of thin air, with no Java code existing for them at all, summoned into existence by some annotation somewhere.
Object Oriented Programming has been mostly replaced by Annotation Oriented Programming in modern Java web development.
I'm not sure I really see the problem. They picked a particularly pathological example, with a lot of heavyweight frameworks. It's a choice.
I just counted the frames from my typical stack, a Guice/Hibernate project. It was about 100 lines from a constraint violation handler in the postgres JDBC driver up to Thread.run(). This seems pretty reasonable, especially with the various layers I've added for authentication, declarative transaction management, AOP logging, etc.
Furthermore, IntelliJ is pretty good about only showing me what I care about. Overall I find the debugging experience no worse than Python or Ruby, and much better than exceptions from Python or Ruby native libraries. It's orders of magnitude better than debugging Go, which destroys stack context with every "if err != nil return err".
This looks like the perfect example to illustrate the point that Rich Hickey tries to make in "Simple made easy" [1].
This huge call stack has been designed to make your life as a developer easy but the price you pay is an enormous amount of complexity.
I've been working a lot with a similar Java web stack and I feel how painful this complexity is. What is worse, is that I think that a lot of this complexity is incidental. There are libraries and frameworks designed to make some things easier, but in the process end up creating a lot of problems that then requires another library or framework to overcome that problem which also has other problems and so on... The result is a huge stack like this.
One concrete example of this is Hibernate. A tool designed to make it easier (apparently) to work with databases, but in the end create so many problems that the medicine ends up being much worse than the disease.
Resolving an HTTP request that returns a the result of a database call should not be this complicated! HTTP is simple! Why do we need so many calls to so many things. I'm not advocating for a flat stack of course, but certainly a stack this deep is a clear sign that something is wrong.
I very much agree with Rich Hickey, we need to stop thinking about how to make things easier and start thinking how to make them simpler.
Resolving an HTTP request that returns a the result of a database call should not be this complicated! HTTP is simple! Why do we need so many calls to so many things. I'm not advocating for a flat stack of course, but certainly a stack this deep is a clear sign that something is wrong.
Http is pretty simple, executing sql queries against a database is simple-ish (close those connections!). Authentication, authorization, marshalling, unmarshalling, transaction boundaries, ..., are not so simple, especially not when all taken together.
People bemoan java as you are doing here, but the reality is other languages and frameworks, any that attempt to address the same problems and concerns have the same level of complexity. Java has the advantage of kick ass tooling, debugging, and monitoring infrasture, a lot in the jvm itself (visualvm).
>One concrete example of this is Hibernate. A tool designed to make it easier (apparently) to work with databases, but in the end create so many problems that the medicine ends up being much worse than the disease.
Sure.
At our startup we had the choice to let 20 programmers write custom individual SQL statements for 100s of CRUD operations, or create entities and let Hibernate generate them for us.
We used hibernate and it has worked out well.
I can't imagine how it would have been to debug 100s of bespoke SQL queries and associated object mapping code, each written in the developers unique style after a few years.
What I can't fully get my head around is how defensive people get about things like Hibernate. I've tried it out, and it doesn't do much for me, but it doesn't really get in my way, either; I can work just as fast with Hibernate as I can with JDBC. I think part of the reason for that, though, is that I can work at either level; I can work out in my head what Hibernate is doing and work with it rather than against it. Somebody higher up retorted with, "why not just write your own web server?" Indeed, why not? I've done it for relatively simple REST-API type cases; as long as you don't need a lot of the more complex HTTP cases like continuation messages, caching, digest authentication and redirects, why not? It's nice to have everything under your control and it's almost definitely faster than any third-party solution that's going to have been written to deal with dozens of corner cases that aren't relevant to what you're doing.
I’ve seen this all over the place and it looks like a totally fine reasonable stack trace to me.
Anyway while the stack (hibernate, etc) shown in this stacktrace is still heavily in use, newer async java frameworks/tools usually result in very short stacks for me. Sometimes I get a trace and it’s 5-6 frames. Of course then you may not know how you got there. So it’s really not all that great.
While that's entirely true, the glueing of those frameworks together is very common in the Enterprise Java development space especially around 2006 when this was taken.
I'm a big fan of the JVM and the Java ecosystem but in many ways the JVM ecosystem is split into two worlds: frameworks or libraries. This would be an example of a framework heavy development model where god knows what is going on between the outside world, your biz logic and the database calls.
Not every java web app needs to be this bloated. It's possible (even practical) to build robust, database driven web apps/web services without spring/hibernate and with far shallower stacktraces.
Not sure it is any smaller today. Some additional layers not seen are the DTO transformations (not seen because they don't become part of the stack when it's hitting JDBC) but two more major subsystems. Many IDEs now automatically filter out these well known stacks from view.
Personally moving from a mostly Java background to entirely typescript on the backend and front-end.
Although I like TypeScript as a language, and use Node.js, also really appreciate sharing code such as (data types/interfaces etc.) btw frontend and backend, I think call stack is much worse on the side, especially when you start using frameworks. At least here you get a huge but relevant stack, you can find your business logic, but Node.js it's not rare to get irrelevant unhelpful stack traces which you can't figure out even if you dig in.
This looks much worse than it actually is. You could implement it in a much flatter way.
function HandleRequest(request : Request) : Response
{
LogRequest(request)
CheckAuthentication(request)
CheckAuthorization(request)
var response = DispatchAndHandleRequest(request)
LogResponse(response)
return response
}
This way you would have no deep stack traces and all would be fine. But only until you need to do some additional work somewhere right in the middle, say patch malformed requests between authorization and dispatch. With the above implementation you would be somewhat screwed, the best you could do would be reimplementing HandleRequest() with your additional step in the middle.
But with implementations like those show in the article you just have a list of steps with a common interface, each step does its work and then the next step gets called. If you need to do something new somewhere in the middle, you just add a new step to the list of steps in the right place and you are done, possibly simply by putting the class name of the new step in a configuration file.
var steps =
{
new LogRequestStep(),
new CheckAuthenticationStep(),
new CheckAuthorizationStep(),
// This stupid XYZ client always sends broken requests.
new PatchMalformedRequestFromXyzStep(),
new DispatchAndHandleRequestStep(),
new LogResponseStep()
}
function HandleRequest(request : Request) : Response
{
var context = new Context(request)
foreach (var step in steps)
{
step.Execute(context)
}
return context.Response
}
This would still avoid deep stack traces because we are iterating over all the steps but note that with this implementation we would not really be able to abort processing the request somewhere in the middle, say if the authorization check failed, but we could fix this by adding a flag to the context and check it inside the loop after a step executed. But a more serious limitation is that every step only gets one chance to act, note for example that we have two separate steps for logging the request and the response.
Imagine we wanted to log the request duration, then we would need a step getting the current time at the beginning and another one getting the current time after the request was handled at the end. And the first step would have to somehow communicate the processing start time to the second one, possibly by storing it in the context. A much more elegant solution is to organize the list of steps as a linked list with all steps looking like this.
function Execute(request : Request) : Response
{
PreprocessRequest(request)
var response = GetNextStep().Execute(request)
PostprocessResponse(response)
return response
}
This creates those deep stack traces but it also creates a huge amount of flexibility and extensibility. It surly looks crazy if you do not need it, but when you need it, it is really easy with this model.
The fact that Spring even exists should be a big red flag for the value of static typing. People routinely make mistakes using type systems. And when done correctly I see static type systems work very well solving problems that they themselves created.
p2t2p|8 years ago
I'm really annoyed by this kind of things like they have any meaning except "I don't know shit about how abstractions work in computers"
%s/you/picture author
userbinator|8 years ago
You even mentioned it yourself: this is the additional overhead added by the Java ecosystem on top of what may already be a pretty large stack of stuff. (From what I've seen, libc is pretty shallow. The majority are either leaf functions like strlen(), or <5 levels from a leaf/before execution disappears into a system call.)
Even if the overhead is not much to a machine, it is certainly going to have an effect on the human who has to figure out what's going on. When you write such code you may not think this way, but every piece of code is a possible place for a bug to be.
From my short experience with Enterprise Java years ago, such deep callstacks are usually symptomatic of code that does far more indirection than actual work --- hundreds of tiny methods that have maybe a statement or two and then call another one. I understand that it might feel good and even better to write code like this, but that simplicity is deceptive: you aren't looking at the whole picture and just focusing on micro-simplicity, when it's macro-simplicity that really counts.
This means that when you're trying to track down a bug, the "interesting parts" are scattered in tiny pieces across dozens of files, and it increases the cognitive overhead significantly in having to piece everything back together. You might not realise that something is wrong until you're deep inside, and then you have to jump back several levels above in the callstack to figure out where things went wrong, seeking in and between files to trace out the execution flow.
I'm glad I don't work on code like this anymore.
burkaman|8 years ago
"Java EE is a lot about abstractions which I have grown to appreciate over the years. However, I find this very difficult to explain to my colleague who sits just across the room – he is a Mainframe veteran with tons of experience :)"
marktangotango|8 years ago
gambler|8 years ago
Two function calls for business logic, a hundred dependency libraries, thousands of lines of stateful "initialization" code outside of the stack and a fuckzillion XML files for configuration. But yeah, everything's great!
peterevans|8 years ago
The opposite is not, and should not, necessarily that you write your own versions of things; after all, if you did so, you'd still have as deep of a stack trace as you do with the third-party code!
I guess, if I encountered this stack trace, I might say there's something of a code smell there. But, hey--this is also fairly old code, and I suspect things are Better Now, so let's not be so quick to cast stones!
jimbokun|8 years ago
Object Oriented Programming has been mostly replaced by Annotation Oriented Programming in modern Java web development.
stickfigure|8 years ago
I just counted the frames from my typical stack, a Guice/Hibernate project. It was about 100 lines from a constraint violation handler in the postgres JDBC driver up to Thread.run(). This seems pretty reasonable, especially with the various layers I've added for authentication, declarative transaction management, AOP logging, etc.
Furthermore, IntelliJ is pretty good about only showing me what I care about. Overall I find the debugging experience no worse than Python or Ruby, and much better than exceptions from Python or Ruby native libraries. It's orders of magnitude better than debugging Go, which destroys stack context with every "if err != nil return err".
arrty88|8 years ago
linkmotif|8 years ago
ceronman|8 years ago
This huge call stack has been designed to make your life as a developer easy but the price you pay is an enormous amount of complexity.
I've been working a lot with a similar Java web stack and I feel how painful this complexity is. What is worse, is that I think that a lot of this complexity is incidental. There are libraries and frameworks designed to make some things easier, but in the process end up creating a lot of problems that then requires another library or framework to overcome that problem which also has other problems and so on... The result is a huge stack like this.
One concrete example of this is Hibernate. A tool designed to make it easier (apparently) to work with databases, but in the end create so many problems that the medicine ends up being much worse than the disease.
Resolving an HTTP request that returns a the result of a database call should not be this complicated! HTTP is simple! Why do we need so many calls to so many things. I'm not advocating for a flat stack of course, but certainly a stack this deep is a clear sign that something is wrong.
I very much agree with Rich Hickey, we need to stop thinking about how to make things easier and start thinking how to make them simpler.
[1] https://www.infoq.com/presentations/Simple-Made-Easy
le-mark|8 years ago
Http is pretty simple, executing sql queries against a database is simple-ish (close those connections!). Authentication, authorization, marshalling, unmarshalling, transaction boundaries, ..., are not so simple, especially not when all taken together.
People bemoan java as you are doing here, but the reality is other languages and frameworks, any that attempt to address the same problems and concerns have the same level of complexity. Java has the advantage of kick ass tooling, debugging, and monitoring infrasture, a lot in the jvm itself (visualvm).
gerbilly|8 years ago
Sure.
At our startup we had the choice to let 20 programmers write custom individual SQL statements for 100s of CRUD operations, or create entities and let Hibernate generate them for us.
We used hibernate and it has worked out well.
I can't imagine how it would have been to debug 100s of bespoke SQL queries and associated object mapping code, each written in the developers unique style after a few years.
That would have been fun.
commandlinefan|8 years ago
realharo|8 years ago
This particular problem could be solved by just having a good filtering UI.
You don't have to analyze the stack in its raw text form.
That being said, I agree that complexity in the Java world is often much higher than it needs to be, and sometimes the tradeoffs are not worth it.
linkmotif|8 years ago
Anyway while the stack (hibernate, etc) shown in this stacktrace is still heavily in use, newer async java frameworks/tools usually result in very short stacks for me. Sometimes I get a trace and it’s 5-6 frames. Of course then you may not know how you got there. So it’s really not all that great.
rhapsodic|8 years ago
meddlepal|8 years ago
I'm a big fan of the JVM and the Java ecosystem but in many ways the JVM ecosystem is split into two worlds: frameworks or libraries. This would be an example of a framework heavy development model where god knows what is going on between the outside world, your biz logic and the database calls.
wrmsr|8 years ago
asaph|8 years ago
commandlinefan|8 years ago
augbog|8 years ago
rhacker|8 years ago
Personally moving from a mostly Java background to entirely typescript on the backend and front-end.
harunurhan|8 years ago
unknown|8 years ago
[deleted]
danbruc|8 years ago
But with implementations like those show in the article you just have a list of steps with a common interface, each step does its work and then the next step gets called. If you need to do something new somewhere in the middle, you just add a new step to the list of steps in the right place and you are done, possibly simply by putting the class name of the new step in a configuration file.
This would still avoid deep stack traces because we are iterating over all the steps but note that with this implementation we would not really be able to abort processing the request somewhere in the middle, say if the authorization check failed, but we could fix this by adding a flag to the context and check it inside the loop after a step executed. But a more serious limitation is that every step only gets one chance to act, note for example that we have two separate steps for logging the request and the response.Imagine we wanted to log the request duration, then we would need a step getting the current time at the beginning and another one getting the current time after the request was handled at the end. And the first step would have to somehow communicate the processing start time to the second one, possibly by storing it in the context. A much more elegant solution is to organize the list of steps as a linked list with all steps looking like this.
This creates those deep stack traces but it also creates a huge amount of flexibility and extensibility. It surly looks crazy if you do not need it, but when you need it, it is really easy with this model.mozboz|8 years ago
bullen|8 years ago
And an integrated distributed async-to-async database that is just as shallow: http://root.rupy.se
Feel free to use it: http://github.com/tinspin/rupy
nickbauman|8 years ago
jlg23|8 years ago
mrits|8 years ago
jdmichal|8 years ago