While easy to understand, I’m constantly surprised to see this type anti-intellectualism. The fact that you don’t have the required prerequisites to understand a codebase doesn’t mean it’s bad. Educate yourself on Category Theory and functional programming techniques and learn to leverage these tools to your advantage.
Alternatively you can work in Go where braindead simplicity is the mandated norm.
Thank you for this reply. It demonstrates exactly the mindset of those few FP die-hards that I have seen wrecking havoc in a couple of companies where I worked for.
I'm not saying FP is bad, it is actually really powerful when it is in the hands of those who understand when and how to use it.
The problem with the attitude that you demonstrate in your reply is also not limited to FP: in the era of OOO, there were those that spread the evangelism of design patterns to all places regardless if it made any sense; before that in the 90's there was a group of programmers that liked to generate code until no colleague understood anymore what was happening.
And when you tried to tell them that abstractions don't come for free, but with a cost because they make code harder to understand, the answer for the last 25 years has always be the same, although rarely said straight: that it was only hard for -you- to understand, and not for the enlightened master himself.
I have seen more projects fail because of too much unneeded abstraction than by all other causes together. I even have seen companies go almost bankrupt because of projects engineered by lone wolfs where nobody understood the abstractions anymore except for the designer himself, and at some point he himself not really anymore either.
And the problem is that this mindset continues to be cultivated by CS books and conferences. Few people want to read a book that tells them that the secret of being productive as a team depends on mostly the culture in the team and the simplicity of the code, and not on the latest hyped framework, language or paradigm.
> I'm not saying FP is bad, it is actually really powerful when it is in the hands of those who understand when and how to use it. The problem with the attitude that you demonstrate in your reply is also not limited to FP: in the era of OOO, there were those that spread the evangelism of design patterns to all places regardless if it made any sense; before that in the 90's there was a group of programmers that liked to generate code until no colleague understood anymore what was happening.
Agreed, and what did we get? dogmatic decrying of how OOP is completely useless and objectively bad, not too dissimilar from some of the comments on FP and Scala on here.
> before that in the 90's there was a group of programmers that liked to generate code until no colleague understood anymore what was happening.
Ah! Just write a quine and be done with it!
> I have seen more projects fail because of too much unneeded abstraction than by all other causes together. I even have seen companies go almost bankrupt because of projects engineered by lone wolfs where nobody understood the abstractions anymore except for the designer himself, and at some point he himself not really anymore either.
"cake pattern" on one side and scalatest on the other.
As for
> I’m constantly surprised to see this type anti-intellectualism.
You hardly need any Category Theory to theory to understand type theory or mostly important Scala's type system. This the exact knowledge and allows you use it effectively. Yeah, the way he puts it makes those who like to understand the theory behind everything look bad.
For most tasks programming it's more an engineering discipline than an intellectual pursuit. In that context complexity must always be justified.
I know, and have been told, to have written hard to read code. In my case it's usually vectorized code in numpy or a C or cython extension for really hot code. But I always have a good reason, usually performance when it matters.
It's usually said that premature optimization is the root of all evil, but it's nothing compared against premature abstraction. Optimization at least is local in its nature while abstraction tends to expand all over a code base, and when the assumptions made for the abstraction no longer hold people is still forced to keep dancing for a music that no longer plays.
"Educate yourself" assumes that your interests are everyone's interests but this is a vast field and not everyone has interest nor time to learn about category theory. Most probably your coworkers are quite intelligent. If they are not interested in your idea it may be that it's not appropriate or maybe not correctly framed, or not mature.
I'm stealing that phrase:
"Premature abstraction is the root of all evil".
It's hard for me sometimes to justify in a code review why an abstraction is not required (yet) when someone has put some effort into it.
The thing is I DO understand category theory and FP after going off the deep end in Scala land. It did not make me a better programmer or make writing code any easier or faster. In fact it makes me spend 50% of my brain power on juggling those concepts and how they are encoded into a language that does not and never will support them as well as say Haskell.
I learnt the basics of FP a long time ago and understood the data-centric view of computation, the benefits of immutability etc - those things actually do help me write better code. The advanced FP, not so much. The nightmarish encoding of advanced FP shoe-horned into a language not built to support it that compiles into Java definitely not built to support it actively hurts. There are so many dirty macro shenanigans and hacks in the guts of the FP-purist libraries to force the language to implement things it just isn't particularly well suited to implement, which makes reading or debugging library code a nightmare. Like if you want Haskell, just use Haskell FFS and let us code reasonable Scala in peace.
There are so many parts of math that are much more useful in various fields - basic optimization in solving sudoku puzzles[1], linear programming in Z3, gradient descent in ML, calculus & vectors in graphics programming etc.
Everytime the “go learn category theory if you want to become a better programmer” trope arises on HN (less often than it used to, but still occasionally) I’m left scratching my head. I actually learned category theory as a math grad student, before changing fields and going in a more applied direction. I’ve spent decades writing software since then, and I can easily think of about 10 areas of math and CS that are way more useful day to day. Once you’ve mastered calculus, linear algebra, probability theory, statistics, numerical analysis, optimization, algorithms, complexity theory, combinatorics, asymptotic analysis—sure, go pick up category theory.
But Category Theory is different to all the other examples you gave, as it describes program composition. i.e. building larger programs from smaller ones. Computer scientists have been searching for the "mathematics of program construction" for a long time, desperately trying to make the process more formal. Category Theory is what many seem to have arrived at. There are even books about it, e.g. "The Algebra of Programming".
Most computer science graduates are well schooled in algorithms, complexity, linear algebra etc. However, the vast majority of their time commercially will not be spent implementing some core algorithm, but instead gluing systems together and consuming/writing APIs. Any mathematics that helps them better accomplish this would be good to add to their toolbox.
The author of "The Science of Functional Programming" book (https://github.com/winitzki/sofp) makes an argument that CT is at best tangential to practicing FP.
I found category theory to be helpful in giving a vocabulary for some very simple and essential things that otherwise get lost. My goto example are Monoids and Semi-Groups, which are almost mindbogglingly simple but pop up everywhere.
Category theory gives you the precise language to talk about such stuff and helps to communicate and talk about more complex concepts and that's always a plus.
I sympathize with your thinking. There are two "readabilities".
1. How easy it is for an average programmer to understand the code.
2. How understandable the code actually is if the reader is fluent with that prior knowledge.
If one finds the latter is much better than code written in conventional techniques, but people are scared off by the former, it can get pretty frustrating, and "people are stupid" may be a conclusion drawn.
---
But my experience is that those techniques do not yield better code.
So, I’ll probably get shit on for this, but I really cannot wait to leave this industry.
I hate how “best tool for the job” really just means “what we can have a large hiring pool of people who’ve only ever learned one programming style and possibly only one language of” or “for legacy reasons this is the only choice”.
The code base had multiple problems, none of which I would blame on category theory or Cats.
- Engineers had written higher abstractions seemingly just because they could. When I audited how internal libraries were used across our services, calling applications weren't making use of the advanced abstractions. I'm talking about things like using Cats to abstract across AWS S3 error handling. Cool, except that it wasn't needed because we never actually encountered the exotic compositions of failures anticipated by the libraries.
- The abstractions written for our own business logic were worse than abstracting over S3. They were premature. Our business logic had to change frequently because the end user experience was still evolving rapidly. Changes that violated previous assumptions and their corresponding abstractions took longer than they should have and/or led to very awkward code.
- At least at the time, tooling had more problems with the "advanced" code. The IntelliJ IDEA Scala plugin could not yet show how implicits were used. It couldn't find senders sending to an Actor the way it can easily find plain callers of a function. You would need to manually force a "clean" in certain modules before code changes would compile as expected. IDEs would also fail to flag code that couldn't compile, and incorrectly flag code that would compile, at a higher rate compared to plainer Scala.
I'm still glad that I have access to Cats, Akka, and other advanced parts of the Scala ecosystem. They're still used in a few places where their value is greater than their cost. Even in the plain code, I'm still very glad I have pattern matching, immutability-by-default, rich collections, map, flatMap, filter, fold, scan, find, etc. I have no plans to transition our company off Scala internally. If I were starting a greenfield project with myself as the sole developer, I'd probably be using Scala for that too. But I prefer to write a bunch of simple repetitive code first, then develop abstractions after it's clear what the commonalities are.
There are others options besides Go and category theory. I'd say that for now, languages that are like ML are enough. They're coming in the mainstream with pattern matching, immutable records, etc. It'll take some time for people to learn this, learn to use it properly, teach it to others. And maybe some years later a monad will be a pattern as common and well-known as an iterator. But today isn't that day yet, and until now you have to collaborate with your peers. You also have to assume that you may be wrong about some of this stuff, and keep an open mind for FP alternatives. For example, I think algebraic effects aren't part of category theory, yet they're an exciting new feature that might have an impact in "industrial" languages one day.
There are plenty of companies where everyone has a very mature understanding of monads and where new joiners are coached to reach that level of understanding.
And there are plenty of places that aren't using source control, CI and unit tests.
If all you do is add numbers together, is it anti-intellectual to question why one needs to grok Principia Mathematica first rather than just doing some arithmetic?
In other words, do you solve problems that actually require category theory or are you just navel gazing?
This is development nowadays, hire people fresh out if am eigh week javascript boot camp and everyone has to write code they can understand.
Instead of bringing the juniors up, we drag the more experienced guys down to their level.
The result is really shit basic code everywhere. or you end up using a language like go where there's basically only for, if and arrays and writing code in it is miserable and tedious.
This ain't anti-intellectualism. Scala fanatics (more like FP fanatics who use scala) write overly completed code just for purity's sake. Its ok, but it creates a maintenance nightmare and a huge hiring problem.
misja111|4 years ago
I'm not saying FP is bad, it is actually really powerful when it is in the hands of those who understand when and how to use it. The problem with the attitude that you demonstrate in your reply is also not limited to FP: in the era of OOO, there were those that spread the evangelism of design patterns to all places regardless if it made any sense; before that in the 90's there was a group of programmers that liked to generate code until no colleague understood anymore what was happening. And when you tried to tell them that abstractions don't come for free, but with a cost because they make code harder to understand, the answer for the last 25 years has always be the same, although rarely said straight: that it was only hard for -you- to understand, and not for the enlightened master himself.
I have seen more projects fail because of too much unneeded abstraction than by all other causes together. I even have seen companies go almost bankrupt because of projects engineered by lone wolfs where nobody understood the abstractions anymore except for the designer himself, and at some point he himself not really anymore either.
And the problem is that this mindset continues to be cultivated by CS books and conferences. Few people want to read a book that tells them that the secret of being productive as a team depends on mostly the culture in the team and the simplicity of the code, and not on the latest hyped framework, language or paradigm.
the_only_law|4 years ago
Agreed, and what did we get? dogmatic decrying of how OOP is completely useless and objectively bad, not too dissimilar from some of the comments on FP and Scala on here.
scalaIsKindaMeh|4 years ago
Ah! Just write a quine and be done with it!
> I have seen more projects fail because of too much unneeded abstraction than by all other causes together. I even have seen companies go almost bankrupt because of projects engineered by lone wolfs where nobody understood the abstractions anymore except for the designer himself, and at some point he himself not really anymore either.
"cake pattern" on one side and scalatest on the other.
As for
> I’m constantly surprised to see this type anti-intellectualism.
You hardly need any Category Theory to theory to understand type theory or mostly important Scala's type system. This the exact knowledge and allows you use it effectively. Yeah, the way he puts it makes those who like to understand the theory behind everything look bad.
linspace|4 years ago
I know, and have been told, to have written hard to read code. In my case it's usually vectorized code in numpy or a C or cython extension for really hot code. But I always have a good reason, usually performance when it matters.
It's usually said that premature optimization is the root of all evil, but it's nothing compared against premature abstraction. Optimization at least is local in its nature while abstraction tends to expand all over a code base, and when the assumptions made for the abstraction no longer hold people is still forced to keep dancing for a music that no longer plays.
"Educate yourself" assumes that your interests are everyone's interests but this is a vast field and not everyone has interest nor time to learn about category theory. Most probably your coworkers are quite intelligent. If they are not interested in your idea it may be that it's not appropriate or maybe not correctly framed, or not mature.
flowcont|4 years ago
zaptheimpaler|4 years ago
I learnt the basics of FP a long time ago and understood the data-centric view of computation, the benefits of immutability etc - those things actually do help me write better code. The advanced FP, not so much. The nightmarish encoding of advanced FP shoe-horned into a language not built to support it that compiles into Java definitely not built to support it actively hurts. There are so many dirty macro shenanigans and hacks in the guts of the FP-purist libraries to force the language to implement things it just isn't particularly well suited to implement, which makes reading or debugging library code a nightmare. Like if you want Haskell, just use Haskell FFS and let us code reasonable Scala in peace.
There are so many parts of math that are much more useful in various fields - basic optimization in solving sudoku puzzles[1], linear programming in Z3, gradient descent in ML, calculus & vectors in graphics programming etc.
[1] https://norvig.com/sudoku.html
hyperbovine|4 years ago
grumpyprole|4 years ago
Most computer science graduates are well schooled in algorithms, complexity, linear algebra etc. However, the vast majority of their time commercially will not be spent implementing some core algorithm, but instead gluing systems together and consuming/writing APIs. Any mathematics that helps them better accomplish this would be good to add to their toolbox.
62951413|4 years ago
esarbe|4 years ago
Category theory gives you the precise language to talk about such stuff and helps to communicate and talk about more complex concepts and that's always a plus.
AzzieElbab|4 years ago
ackfoobar|4 years ago
1. How easy it is for an average programmer to understand the code.
2. How understandable the code actually is if the reader is fluent with that prior knowledge.
If one finds the latter is much better than code written in conventional techniques, but people are scared off by the former, it can get pretty frustrating, and "people are stupid" may be a conclusion drawn.
---
But my experience is that those techniques do not yield better code.
And "Educate yourself" made me barf.
the_only_law|4 years ago
I hate how “best tool for the job” really just means “what we can have a large hiring pool of people who’ve only ever learned one programming style and possibly only one language of” or “for legacy reasons this is the only choice”.
Yes I’m familiar with reality, I just hate it.
philipkglass|4 years ago
- Engineers had written higher abstractions seemingly just because they could. When I audited how internal libraries were used across our services, calling applications weren't making use of the advanced abstractions. I'm talking about things like using Cats to abstract across AWS S3 error handling. Cool, except that it wasn't needed because we never actually encountered the exotic compositions of failures anticipated by the libraries.
- The abstractions written for our own business logic were worse than abstracting over S3. They were premature. Our business logic had to change frequently because the end user experience was still evolving rapidly. Changes that violated previous assumptions and their corresponding abstractions took longer than they should have and/or led to very awkward code.
- At least at the time, tooling had more problems with the "advanced" code. The IntelliJ IDEA Scala plugin could not yet show how implicits were used. It couldn't find senders sending to an Actor the way it can easily find plain callers of a function. You would need to manually force a "clean" in certain modules before code changes would compile as expected. IDEs would also fail to flag code that couldn't compile, and incorrectly flag code that would compile, at a higher rate compared to plainer Scala.
I'm still glad that I have access to Cats, Akka, and other advanced parts of the Scala ecosystem. They're still used in a few places where their value is greater than their cost. Even in the plain code, I'm still very glad I have pattern matching, immutability-by-default, rich collections, map, flatMap, filter, fold, scan, find, etc. I have no plans to transition our company off Scala internally. If I were starting a greenfield project with myself as the sole developer, I'd probably be using Scala for that too. But I prefer to write a bunch of simple repetitive code first, then develop abstractions after it's clear what the commonalities are.
Zababa|4 years ago
caoilte|4 years ago
There are plenty of companies where everyone has a very mature understanding of monads and where new joiners are coached to reach that level of understanding.
And there are plenty of places that aren't using source control, CI and unit tests.
_vertigo|4 years ago
In other words, do you solve problems that actually require category theory or are you just navel gazing?
hocuspocus|4 years ago
You don't need more than highschool maths to understand the category theory behind common typeclasses.
LandR|4 years ago
Instead of bringing the juniors up, we drag the more experienced guys down to their level.
The result is really shit basic code everywhere. or you end up using a language like go where there's basically only for, if and arrays and writing code in it is miserable and tedious.
ngc248|4 years ago