Open-sourcing Sorbet: a fast, powerful type checker for Ruby

[the_duke]

It's been funny to watch how more and more static type systems are getting bolted on to dynamically typed languages in recent years.

Typescript (with stellar adoption), native type annotation support in Python, Sorbet, PHP 7, Elixir + Dialyzer, ...

I wonder why there isn't a popular gradually typed language that natively allows writing both dynamic and type-safe code, allowing quick prototyping and then gradual refactor to type safety.

I guess in part because it's a big challenge to come up with a coherent type system that allows this, the bifurcation in the ecosystem, and often a somewhat leaky abstraction. Eg in Typescript you will often still run into bugs caused by malformed JSON that doesn't fit the type declaration, badly or insufficiently typed third party libraries, ....

Google's Dart is the only recent, somewhat popular language (only due to Flutter) that allows this natively - that I can think of right now.

I do think such a language would be very beneficial for the current landscape though, and projects like this show there is a clear need.

Edit: just remembered another option: Crystal. Also Julia, as pointed out below.

[tptacek]

For whatever it's worth and without wanting to start a language war (I like Python just fine), I think Python/Ruby-style typing is a false economy for prototyping. There are a lot of things that make Go slower to write than Ruby, but mandatory typing isn't one of them. Rather, Ruby's total lack of typing makes it harder to write: you effectively end up having to write unit tests just to catch typos.

I wonder whether the perception that type safety slows down Ruby (or ES6) development comes from the fact that the type systems are bolted on after the fact.

[_hardwaregeek]

Unless you've used a language with a high level of strictness, i.e. OCaml/Haskell/Rust, it can be hard to see the sheer power and utility of typechecking. If someone has only used Java, they may not understand the true power of types. But if you've familiar with OCaml/Haskell/Rust, why bother writing dynamically typed code? Sure there's some niche usecases where it's more powerful, but generally you can do as much with say, Rust, or even more pragmatically, C# 8/Kotlin.

While if you've only used dynamic languages or badly typed languages, then having to deal with this stupid naggy compiler is just annoying. A big part of learning a strongly statically typed language is learning that the compiler is your friend, and that errors are good. I've noticed that a lot of people new to TypeScript try to get the compiler to shut up, often resorting to any or @ts-ignore, while more advanced users will see it as a dialogue. The compiler complains? Okay, something's wrong: Let's find the root cause here.

TypeScript took off because people had no choice but to write JS, so any benefit was better than no benefit. Sorbet was also borne out of an existing codebase. But a new language wouldn't have this lock in factor.

[robacarp]

> a popular gradually typed language that natively allows writing both dynamic and type-safe code, allowing quick prototyping and then gradual refactor to type safety.

You mention it in your edit, but Crystal has been exactly that for me. A rubyist for a decade I found Crystal to have the type system I was expecting all along.

[simplify]

A fascinating part about Sorbet is it did not have to introduce any additional syntax to Ruby (unlike TypeScript, for example). This really speaks to the expressiveness of Ruby. Very cool.

[SomeOldThrow]

The type signatures are pretty noisy to read, though, some syntax can definitely help. Maybe with Ruby 3?

[est31]

Was that additional syntax in TypeScript actually neccessary for type inferrence? Or is it rather to avoid API hazards when you change some internal code and suddenly the API of your library breaks because the inferred type has changed.

[castwide]

Coincidentally, I announced the beta version of a Ruby type checker in Solargraph two days ago: https://github.com/castwide/solargraph/issues/192

It has a few overlapping features with Sorbet, with one major difference being that Solargraph type checking relies on YARD documentation instead of annotations.

[hit8run]

Love the work you’re doing on Solargraph! Thx for it.

[brigandish]

I wondered why no one had tried this instead. It's certainly a (much needed!) incentive to document methods.

I'm going to give Solargraph a look-see.

[anonova]

> To enable static checking with srb, add this line (called a sigil) to the top of your Ruby file:

> # typed: true

Isn't this called a directive/pragma? A sigil is a symbol on a name.

Either way, I'm excited to see this finally out after seeing the past presentations on it.

[ptarjan]

Thanks for pointing that out. It can be called all of those. We liked sigil from its connotation:

> Google defines sigil as, “an inscribed or painted symbol considered to have magical power,” and we like to think of types as pretty magical

https://sorbet.org/docs/static#fn1

[cperciva]

The terms all overlap a bit, but I would interpret "directive" or "pragma" to indicate that it's conveying meaning to the Ruby interpreter. This is exactly the opposite -- it's a comment as far as the Ruby language is concerned while conveying meaning to an external tool.

[FpUser]

I've never understood the so called advantages of dynamic typing. To me it looks like a land mine in one's project waiting to blow at run time. And what for? Do developers code so fast that the time spent on typing something like "int i" will provide any real savings? Now vendors are trying to patch those with bolted on top syntax extensions/derived languages that need to be transpiled. What a mess.

[mberning]

Most people fixate on the terseness that it can afford in a language. I suppose I do like that but for me it is not a huge deal.

What I think is more important is the flexibility that it brings to express design patterns that in other languages, like Java for example, can become very cumbersome. I can’t tell you how many times I have been in the bowels of some Java code and found some method that takes a concrete implementation of something that could or should be an interface when I really want to pass in something different. Then you are like “let me extend and fix this class” and then you end up just extending and fixing 1/2 the code base to get done what needs to be done. In a language like Ruby I would just pass in an object that responds to all the needed methods and it would happily work. Ideally you wouldn't get into these type of messes in statically typed languages because people would follow good design principles all the time. But people are fallible and in reality messes are everywhere in statically typed languages.

So I think the approach of adding type enforcement if desired is a nice approach considering there is a large amount of code out there that probably doesn’t benefit much from it.

[jeremycw]

When you're consuming JSON that has deep nesting, arrays that contain multiple types, etc. Something that may be two lines of code in a dynamic language could be as much as 100 lines in some static typed languages.

[SomeOldThrow]

It makes unit testing much easier. That’s the best explanation I’ve found. Rapid prototyping too, but that just means you’re backloading tech debt so that’s at best neutral in pure technical terms. In a startup context backloading tech debt is deeply desired.

[mruts]

Type checking isn’t really related to typing “int.” Many languages infer types. In fact, Hindley Milner type systems should be able to infer all types without explicitly specifying any of them.

[camjohnson26]

Sometimes I do code that fast. When you’re messing around just trying to find out if something is possible you want to write as much code in as short a time as possible, and Python shines for that. Every second wasted typing is a second that could have been spent moving forward.

Of course the problem is that the prototypes are terrible to maintain and eventually need unit tests and typing. But you don’t want to waste time adding those things if you’re not even sure your idea will work. I use strongly typed languages in production and couldn’t imagine using Python for that.

[hartator]

Awesome work.

    sig {params(person: Person).returns(Integer)}
    def name_length(person)

Not sure if I dig the syntax. Furthermore arguments seems to be the official names for method arguments, not parameters. eg, `ArgumentError`. `params` also feels like it's linked to Rails `params` variable in controllers. It can be confusing.

Something like this will also feel more Rubyist:

    def name_length person: Person, return: Integer

But it probably requires a deeper hack or a change in MRI.

[whycombagator]

Related, Square wrote a great article: "RubyKaigi and the Path to Ruby 3"[0]. The section titled "Static Analysis" high level compares Sorbet to Steep

[0] https://developer.squareup.com/blog/rubykaigi-and-the-path-t...

[sickcodebruh]

It’s my understanding that the Sorbet team is involved with bringing types to Ruby 3. I’m unclear on whether it will be Sorbet itself or if it’s elements of it. Can’t dig up the source right now, maybe someone can corroborate this?

[jtms]

Though I haven’t yet used it for anything in production, I think if I were starting something greenfield and wanted “Ruby with static types” I would go with Crystal. I really enjoy writing it and the performance you can get is quite a significant boost over Ruby.

[sickcodebruh]

I’d still go Ruby. A language’s ecosystem and community are as much factors in why someone should choose or avoid it as its syntax. Both of those things are fantastic for Ruby — I’d argue that they’re some of its best features, in fact. Crystal? Not so much.

[dajonker]

I have mixed feelings about adding type annotations to an existing project. IDEs become easier to use, you can avoid certain bugs, refactoring becomes a bit less error-prone. But this comes at a cost: you need a very high type coverage, which means you need to rewrite a lot of code to deal with the different style of polymorphism. It's very likely that you end up with code that looks as if it were written in a statically typed language but without any of the performance benefits of such a language.

[mark_l_watson]

Thanks for this. Major contribution to the Ruby community!!

[ptarjan]

Thank you! Ruby has been kind to us, we'd like to be kind back.

[hderms]

I'm interested in whether `.rbi` files are going to be the only official route for typing in Ruby, and if so, how that would end up impacting Sorbet?

[hirundo]

It'd be nice to have an option to put that data inline.

[bakery2k]

Does anyone know what Matz thinks of Sorbet? He has previously been opposed to adding type annotations to Ruby [1].

This is in sharp contrast to Python, where Guido has overwhelmingly embraced type annotations.

[1] https://bugs.ruby-lang.org/issues/9999#note-13

[mbell]

I wonder what the reason for not supporting structural typing is. It seems like a very natural fit for Ruby.

[itake]

The "Getting Started" link at the bottom of the page is broke

https://sorbet.org/blog/2019/06/20/docs/adopting

[poorman]

I've used contracts any time this type of thing was necessary. https://github.com/egonSchiele/contracts.ruby

[wpride]

We use Contracts too and are in the process of transitioning to Sorbet. In addition to the same runtime type checking as Contracts, Sorbet offers static type checking (and will re-use your runtime signatures in its static analysis).

[imhoguy]

Excellent work! I wonder if somebody already tried to run it against Rails codebase.

[hdoan741]

Original author of sorbet-rails here. I tried and it did take some work to integrate with Rails, because how dynamic Rails code can be.

But it's pretty useful once setup. It can know when an attribute is nullable or non-nullable, which is a big deal. We even use Sorbet to limit the usage of some dynamic Rails API so that it's more sane.

[darkdimius]

Companies that use Rails have also started working on https://github.com/chanzuckerberg/sorbet-rails, worth checking out!

[dwheeler]

While we're on the topic of static analysis of code that uses Rails... I suggest also using other static analysis tools as they make sense. I lead the Railroader project, an open source security static analysis tool for programs built on Rails. It doesn't guarantee finding all security problems, but it can help. More info here: https://railroader.org/

[darkdimius]

Yes, join the slack! While Stripe doesn't run rails all other companies in private beta did!

[ksec]

Both Shopify and Kickstarter are on Rails.

I wonder why Github didn't join the private Beta?

[poorman]

The dependency on bazel is very off-putting to me. After having tried it for other projects and watched as rapid breaking backwards incompatible changes were made to the tool, I'm opting out of anything requiring it.

[darkdimius]

We only use it at build time, as for a user, it should be invisible for you

[maxfurman]

I'm having trouble finding the implementation of `sig` - could someone please point me to the right file? Thanks. I'm very curious how they pulled this off.

[sebastianconcpt]

I still wonder what problem exactly static typing fixes. Is is only me that I'm too used to dynamic tech without correctness issues?

[SmooL]

Well one thing: when you work on enough code for long enough you start to forget what's/what. Static typing let's you jump in and immediately know the shape of your data at any point in the code, without having to re-trace execution manually

[lacampbell]

I tend to think in terms of static types, so I prefer it, but a lot of people seem to be very productive with dynamic typing.

What's your workflow like? Strict TDD? Runtime contracts? How do you feel when you need to use a statically typed language?

[sebastianconcpt]

Dude, I asked a genuine question and I get downvotes? WTF? Downvoters, I didn't asked anything off topic, can you elaborate why the downvotes?

[jedisct1]

Very cool stuff!