top | item 10559776

Five years of Scala and counting: debunking some myths

250 points| logician_insa | 10 years ago |manuel.bernhardt.io | reply

152 comments

order
[+] leothekim|10 years ago|reply
Although I agree with most of what the author says here about scala and its benefits and features, this is hardly a debunking and more of a strongly opinionated defense. Some of the arguments are along the lines of "hey SBT isn't just a bunch of symbols" or "I upgraded my MacBook and compilation times are not a problem" or "tooling is better now". Reassurances yes, debunking hardly.

Scala compilation time is probably the worst problem I have with scala, and it still is frankly. I have seen no evidence that this has improved from personal experience and after talking with devs I know working in large scala code bases (Foursquare, Twitter).

Even if there are speed improvements in the scala compiler in future versions, the author doesn't address binary incompatibility between major scala versions either which, along with slow compilation times, makes it a serious PITA to upgrade to a scala version that might actually compiler faster.

[edit] hit submit too early last time, re-edited after regathering my thoughts.

[+] gilbertw1|10 years ago|reply
I'll address the binary compatibility concern. I think at this point for the average scala developer, binary compatibility isn't really a big deal anymore. At this point the releases require very few (if any) changes to the codebases and all the library maintainers have stepped up their game with releases corresponding with new versions.

Sure, it'd be great to have binary compatibility, but I think the ability to make changes to the language unhindered by supporting byte code from previous languages is more valuable.

For example with the 2.11 release, I had three different non-trivial sized projects and for each I was able to simply change the scala version in my sbt file and everything worked just as it had before with no library or compile issues.

I will agree that it was a huge issue in the past with 2.7 - 2.9, but in my world it hasn't bitten me in a long time. Definitely not trying to say it isn't an issue or something that wouldn't be great to be fixed, but at this point I have a tough time seeing it being considered a barrier to entry in any way.

Also, I just throw in that for me compile time hasn't been an issue for me lately either. I work on reasonably sized projects, but definitely nothing on the twitter / foursquare scale.

[+] bad_user|10 years ago|reply
Scala developers end up on the defensive because many of the claims are either because of misunderstandings or are fashion driven, being repeated ad-nauseam many times by people with only a superficial experience of the language.

For example I'm going to make the claim that SBT indeed isn't just a bunch of symbols, but rather one of the best build management tools in existence. For anybody going to challenge that I'm going to start mentioning the complete clusterfuck of other platforms and languages, like that of C/C++, or that of Python, or that of Javascript, or that of C#/.NET, or that of Haskell. SBT not only does demonstrably a better job than most, but works well out of the box and is relatively pain free. It has its own set of problems of course, but what tool doesn't.

Compatibility issues are real. But you know, the nice thing about those compatibility issues is that it forced the community to come up with solutions. At least Scala and SBT are addressing compatibility issues, whereas the transition from Java 6 to later versions is proving to be next to impossible and I don't think I have to mention Python 3 or ECMAScript 6.

Compilation times haven't been that much of an issue for us because our codebase is split into services and libraries that have a reasonable size. And on the switch from Scala 2.10 to 2.11 I definitely saw an improvement, with the claims of 30% being about right.

I do agree that they can do better, taking a look at similar languages with a slightly better story, like Haskell. However what people are usually missing is that this is usually a trade-off with compile-time safety.

The more static a language is, the harder it is to compile it. The compilation cost for a dynamic language like Javascript is zero, the cost being completely amortized with everything happening directly at runtime. When going from Javascript to Java or C#, for people that prefer Java or C#, people go from zero to a very noticeable and resource intensive AOT compilation step and many times this tradeoff is worth it (depending on whom you ask) for safety, performance or tooling reasons. Well, Scala is a more static language than Java, it infers a lot of things, it has better generics, it does implicits which can then model type-classes and even the infamous CanBuildFrom pattern and so on, giving you a nice balance between productivity and safety in a static language. And that doesn't come for free.

[+] whateveracct|10 years ago|reply
Full-pass Scala compilation is slow, true. It takes my codebases at work a couple of minutes to compile currently. But

1) sbt incremental compilation is basically immediate. This facilitates tight feedback loops for running tests and using the repl

2) Scala's type system does so much more than fast-compiling languages (Go, Java, etc) that it's definitely worth it. What you lose in compile-time, you gain in speed to a correct and sound solution.

[+] tonetheman|10 years ago|reply
Compilation time is still a problem though it is not as big of a deal to me personally.

Complexity or learning the language is still the biggest issue I see. That has not changed in 5 years.

[+] jinst8gmi|10 years ago|reply
I've been using Scala primarily for a few years now after using Java since the 90's. Well written Scala code is much more pleasant to write and maintain than Java (true even for JDK 8). Scala's feature set is great, but you need to have good taste to use it effectively. It's a bit like the gang of four design patterns which some developers, especially Java developers, go looking for places to apply as many of them as possible. Rather consider where the features add value - they are very useful when used appropriately. When used sensibly Scala is amazing. You just have to keep an eye on the new guys so that they don't just use advanced features all over the place where they aren't appropriate. Fortunately that type of behaviour isn't that common but does feed the Scala is complex meme. After a few years with Scala I consider my time spent primarily with Java as lost years.
[+] yashap|10 years ago|reply
Agreed, love Scala, and I think it can lead to super maintainable code bases, but you do have to be careful. The worst offence is probably when people start to get comfortable with implicits, and start throwing them everywhere, definitely need a good code review process to stop that.

Also, the article left out probably my two favourite parts about Scala - awesome language level support for concurrency, and pattern matching. Whenever I go to languages without these features, I really miss them.

[+] joslin01|10 years ago|reply
Scala is my favorite language at the moment. I've used (professionally) Java, C#, Perl, Python, Ruby, Objective-C, and Swift. I played with Go and started writing a simple library (https://github.com/joslinm/validate) to get acquainted with it.

Above all else, what always gets to me with Scala is its powerful generics. It truly is first-class. I'm excited for macros as well.

[+] k__|10 years ago|reply
> Scala has so many complex features

I think operator overloading and implicit conversions are the biggest problem.

You look at Scala code and first don't get anything because it's full of library specific symbols.

Then these implicit conversions mangle your objects behind your back and you don't even know what's happening at run time.

I like the ideas behind these features, but as a beginner, they always scared me away after looking into "production code" :\

[+] hibikir|10 years ago|reply
The problem is not "production code", but library code. I write a lot of business code in Scala, and a large majority of it is very easy to read for the newcomers we have. What is absolutely terrifying is what happens when you look under the covers: If you look at the implementations of libraries, as a newbie, you will want to run away, because you need to understand many language features before you can even understand a thing.

This is why I ended up writing a series of articles on practical implicates. Implicits are pretty much unavoidable, but the scale's website documentation is appalling. But if they are explained through examples, and shows how, and when, it makes sense to use them, we can deal with them reasonably.

[+] incepted|10 years ago|reply
Somebody recently made the acute observation that they love their own Scala code but that they always find other people's Scala code very difficult to read, and that it's probably the reason why Scala never imposed itself.
[+] tormeh|10 years ago|reply
Note that unconventional symbols are very rare in the standard library. Third-party libraries, especially those made by the scaskell community, are where the problem lies. If one uses scalaz one gets what one deserves.

Edit: replaced "you" with "one" to clarify

[+] jinst8gmi|10 years ago|reply
> You look at Scala code and first don't get anything because it's full of library specific symbols.

I've been using Scala for years and I've had minimal contact with libraries using symbols in any significant way (most libraries I commonly depend on use no operator overloading at all). The only ones which come to mind are some of the linked list operations and Map(foo -> bar, abc -> def) stuff.

[+] dudul|10 years ago|reply
Implicit conversions are now considered an anti-pattern and not recommended.
[+] phamilton|10 years ago|reply
I've come to realize that there's no intuitive meaning to the infix tilde operator. Therefore, assigning it meaning is asking for trouble.

EDIT: added infix

[+] merb|10 years ago|reply
> I think operator overloading and implicit conversions are the biggest problem.

Not everytime. Sometimes it could be great. Just look at some BigDecimal Code of Java and other languages.

in Scala (and Python) it's:

    BigDecimal(1.00) + BigDecimal(2.00)
in java it is:

    new BigDecimal(1.00).add(new BigDecima(2.00) and so on
[+] phamilton|10 years ago|reply
I find that Scala is pretty successful if you are already a seasoned Java developer. Coming from other ecosystems, simple questions like "How do I use an execution context other than the global one?" are answered in terms of established Java practices. ("Just call this with an Executor.") And down the rabbit hole you must go. That plus a ton of leaky abstractions (also requiring a knowledge of standard Java practices) has been a big source of frustration for me. But I suppose that if you were already dealing with these quirks in Java, Scala is a nice improvement.
[+] muraiki|10 years ago|reply
This was one of my largest barriers in learning Scala. It seems that one needs to learn Java first because you're going to have to understand how it and Scala interact at some point.
[+] anentropic|10 years ago|reply
I had a similar experience learning a bit of Groovy recently

on the surface it looks quite Ruby-esque, but a bunch of surprises lurking in wait because it's Java underneath

[+] jcadam|10 years ago|reply
I really don't foresee any of these alternative JVM languages getting much traction in my industry (Defense/Aerospace).

I use Scala where I can at work (mostly smaller projects where I am the only developer and can thus get away with it), but every place I've worked over the last few years has been staffed with developers who are actively hostile to the thought of using any language other than Java. And right now, Java is king in the Defense industry. It's the new Ada, except I'd rather be coding in Ada than Java.

On my own side projects, I don't use Scala, because I would never choose to target the JVM for greenfield development. But that's just me.

[+] incepted|10 years ago|reply
The Defense industry is probably a very tough nut to crack for any language, but right now, I'm seeing Kotlin picking up some good momentum. I hope it can take off, it's a very decent alternative and improvement over Java.
[+] Irish|10 years ago|reply
european space agency is/was actively hiring scala devs specifically
[+] theseatoms|10 years ago|reply
What do you use instead? Just curious.
[+] eranation|10 years ago|reply
Using Scala for about 5 years after being a Java dev for another 5 years. This post captures my view pretty accurately as well. Including book recommendations. I want to believe that most software engineers don't avoid checking scala just because of some "one star" post about it or other rants.
[+] oelang|10 years ago|reply
I would add 'Scala in depth' to the book recommendations, it's similar to 'Effective java', the book you need after you have used the language for more than one year.
[+] kafkaesq|10 years ago|reply
But seriously now: don’t let yourself be scared off by “category type theory” and other fancy-sounding words. You do not need all of this theoretical background in order to learn and become a fluent Scala developer. I know, because I have not bothered to look at those things until 4 years with the language. And when I did it’s not like I had an epiphany, throwing out all the code I had written until then. In fact, I was monading all over the place, just had no idea how the things I was building and using were called.

Works for the author, I suppose. Me, I'd prefer to have a solid grasp of the abstractions I'm using -- and an equally solid understanding of how the programming constructs I'm using correspond to those abstractions -- before using them to crank out production code.

[+] whodatninja|10 years ago|reply
great read! I'm 3 years into the language and brought it into my company. As part of mentoring other devs coming from a web background, I kept repeating the mantra, "make it work the simplest way first" then explore the options scala gives you. As long as there were units tests for the logic, the details could change. The more I use the language, the more fascinated I become. Heck, I just learned about Bottom types last week!
[+] oxplot|10 years ago|reply
I've recently started on Rust. Can someone who knows both Rust and Scala draw comparison on the myths the author is talking about?
[+] steveklabnik|10 years ago|reply
I don't really know Scala, but...

Rust doesn't have inheritance, let alone multiple inheritance. We do have traits, which I assume work similarly, it looks like.

On to the ones that are listed under "myths":

IDE support is coming along: https://www.rust-lang.org/ides.html

I rarely hear people say that Rust has too many features, or if they do, they're comparing it to a language with a GC.

Rust's compiler can be faster, and it's getting better all the time. https://raw.githubusercontent.com/sanxiyn/sandbox/master/rus...

Rust doesn't really have traditional OOP, so yeah, a lot of it does go out the window.

Rust doesn't have some Scala features like Higher Kinded Types that lead to this kind of perception, so it's not quite the same. Our type system can be hard for totally different reasons ;). The borrow checker stuff.

[+] ape4|10 years ago|reply
In Java its such a pain to always account for possible exceptions. You have explicitly catch it or explicitly pass it on. Scala frees you of this hassle by not enforcing anything. So handy.
[+] JackFr|10 years ago|reply
And yet with the ease and availability of Try, Either and Option combined with fluent pattern matching I find myself writing much more robust code, especially when using legacy Java libraries.

(My real fear is that Scala is just a gateway drug and if I don't take it easy I'll be main lining Haskell eventually. )

[+] catnaroek|10 years ago|reply
Explicit exceptions in method signatures are a language design mistake. If you need to handle multiple classes of return values, that's what sum types are for. Exceptions exist to communicate secret messages from a raiser to a handler, preventing anyone in the middle from intercepting them.
[+] incepted|10 years ago|reply
Sure, it's much easier to write code that doesn't handle errors, but you're creating buggy code.

Checked exceptions force you to think about error cases upfront, that's their value. More pain for developers, better software for users.

[+] ww520|10 years ago|reply
You can always use unchecked exception like RuntimeException and others to silently pass the exceptions back.
[+] graffitici|10 years ago|reply
I had read somewhere that there were talks of creating a smaller subset of Scala, one that's easier to learn. I believe it wasn't going to have implicit arguments (or conversions?).

I think Odersky had mentioned something of this kind. Anybody else remembers something of this sort?

[+] virtualwhys|10 years ago|reply
That would be Dotty[1]. It's the same language as Scala but with union and intersection types, and some extraneous features stripped out (e.g. procedure syntax, XML, and `for some` types).

Under the hood it's a complete overhaul, based on the DOT calculus[2][3]. Already compile times are faster than current Scala (without any optimizations applied), and tooling will improve as well (Dotty fuses current Scala's type parameter and type member implementation into one, thus less work for IDE presentation compilers to do, for example).

Implicits aren't going anywhere, thankfully, how else could Spark et al offer such wonderful DSLs ;-)

[1] https://github.com/lampepfl/dotty [2] http://www.cs.uwm.edu/%7Eboyland/fool2012/papers/fool2012_su... [3] http://arxiv.org/pdf/1510.05216v1

[+] flavor8|10 years ago|reply
They could call it Java.
[+] nileshtrivedi|10 years ago|reply
> Play has both APIs in Java and Scala, so you can slowly transition from one to the other (provided that you architect your application correctly especially in terms of database access).

What does he mean by "architecting the application correctly"? I have a project in production which uses Play framework in Java and I'd be interesting in transitioning this to Scala? What should I look out for?

This is the only discussion I could find for such a migration: http://grokbase.com/t/gg/play-framework/151wb9eebq/migrate-s...

[+] icedchai|10 years ago|reply
The problem with Scala is it that it's too complicated. This results in slow compile times, poor IDE support, and on and on. I worked with Scala for about 10 months back in 2011. Never again.
[+] joslin01|10 years ago|reply
I would probably be prone to agree with you in 2011. Things are very different in 2015 and it wouldn't hurt you to try. My devs all use IntelliJ with it and have never complained. I use vim personally :)
[+] Babylonian|10 years ago|reply
I've found IntelliJ support for Scala to be excellent. I can only guess that you were using Eclipse? IntelliJ does incremental compilation so most compiles take about 1 second typically.
[+] emergentcypher|10 years ago|reply
None of these things are really a problem any more. Scala has come a long long way since 2011.
[+] piokoch|10 years ago|reply
I am curious to what extent Paul Phillips lecture [https://www.youtube.com/watch?v=4jh94gowim0] is just a rant or in-depth Scala criticism. If he's right then seems that we should not expect bright future for Scala.

I am wondering what Scala experts would say (I don't know Scala sufficiently to trust my judgment)?

[+] hcarvalhoalves|10 years ago|reply
I can see how Scala is an improvement over, e.g., Java, and other languages with weaker type systems and bad abstractions.

Since it's often the case big and complex languages succeed, for some definition of success, I'm sure it will be as successful as Java.

[+] hibikir|10 years ago|reply
Scala is my favorite language. I use it every day. And yet, the language, and its ecosystem, have big problems nobody wants to tackle. They have nothing to do with the myths people talk about though.

For me, the biggest problem is how fractured the community is. Scala is not a very opinionated language at all, so work from different subcultures barely looks like the same language. Working with scalaz? You won't touch the standard library at all. Finagle? Twitter came up with a Futures library before it came standard with Scala, and there seems to be no interest in migrating out of that, so you have multiple libraries for basic concurrency (along with the scalaz version of course). Those groups, that use the language so differently, do not share many interests when it comes to the evolution of the language, what parts are good and bad. This leads to much infighting, and an environment where instead of learning Scala, what people learn is that the community is divided, and often very rude. It makes learning a challenge. Each community also has a different idea of what it is considered acceptable behavior to other human beings. The scala mailing lists has postings that make your average Linus Tovalds post seem sensible and polite.

Another part that makes learning a challenge is how different application level Scala and library level Scala are. In a language like Java, it's easy to go into the standard library, or into whatever popular library you want to use, and look under the covers, even and a newbie, and learn how to write idiomatic Java (whether that's good or not). In Scala, the standard library, and pretty much every major library out there, use language features that are not needed at all for regular use, but are very valuable for building libraries, and yet make the code very hard to understand for someone new. For the worst case of all, look at the source code of Shapeless: Even very seasoned Scala programmers find it difficult to understand, because it's attempting to do things that the designers never even expected people to do with the language.

Another problem, related to the previous one, is that since it's so easy to build DSLs in Scala, pretty much every major library does it. Internal DSLs, implemented around a bunch of implicit conversions that you need to understand pretty well to solve syntax errors. For instance, say you want to use json in your spray web service. There are two ways to do it: Follow a tutorial without understanding it and hope you do not have errors, or spend the time to understand the type class pattern.

And that leads to the last, problem: There are basic intro articles out there. There's information for experts. But the route from basic knowledge to mastery is a big cliff with very little information, and it's easy for people to give up. As it is now, Scala is a language better learned tribally: You have a few gurus, and when complicated questions happen, go ask one of the gurus to see if he can teach you about whatever problem you are having, or about how a feature really applies to your problem.

I love Scala, I really do, but if the community problems are not resolved, the rest of the problems will not get resolved. It will remain a hard language to learn, and other languages that right now do not offer all the rich features of Scala will just steal what they can and replace it instead, just because they have better, less fractured communities.

[+] AhtiK|10 years ago|reply
There's just one sad trouble with the current state of tooling: Scala and SBT in IntelliJ IDEA do not go together.

Each Scala run inside SBT-enabled project takes >10 secs minimum to run. Every single run. SBT make is invoking the full SBT startup sequence and is unable to reuse the session. This comes from the context of using Play Framework where you can't substitute SBT with something else.

I do hope that SBT server becomes a reality at some point and IntelliJ Scala plugin is able to fully use it.

[+] mclifton|10 years ago|reply
Totally agree. I don't understand what the Scala compile server does. It's far slower than using SBT (or Activator, when working on a Play project).

I always just keep a console open and compile from the command line, because IntelliJ compilation is huge waste of time. I rarely debug because that means I'd have to wait 2 minutes for IntelliJ to do it's thing before it launches.

Oh, and the test output is inconsistent. It often reports that all tests pass even when they are failing.

Other than that - and it is a huge inconvenience - IntelliJ is still awesome for Scala development. Everything else the editor is capable of easily justifies it's place in Scala development.

[+] uberdog|10 years ago|reply
I've been developing a Play app in IntelliJ for the past year. When run in development mode ($ activator run) it recompiles only the necessary files when you make a change, save, and reload the page in the web browser. I can even switch branches to another developer's work and don't have to restart the server. I'm not sure what you're talking about. Maybe you have something set up incorrectly.
[+] vbezhenar|10 years ago|reply
An interesting note is that new language Kotlin which aims to the similar niche has many things mentioned in the article. It handles NPE a bit differently and I'm not sure about multiple inheritance, but compilation is definitely fast, IDE support is wonderful, no need to learn another build tool and no monads allowed (joke, but language is more practical, than academical).

For me it's a strong competitor to Scala, if you want to use something better than Java.