The benefits of static typing without static typing in Python

[hibikir]

Optional typing like the one we see here is not uncommon in other dynamic languages: For instance, Clojure has core.typed and prismatic schema, which approach the problem in ways related to what the article shows.

However, while optional typing gives you some benefits over purely dynamic typing, the fact that it's all bolted-on causes a variety of problems. First, there's the fact that you'll have code with types interact with code without them. This eventually causes more trouble than it solves.

IMO, the biggest issue though is that what we really see from most of these systems is to add optional type systems that are comparable to very simple type systems, like Java's. But those strongly typed systems are not really that powerful! The real power of static typing doesn't come from being able to make sure we don't mix strings and integers, but in doing type checking for much more complex abstractions. Type systems like Scala's, or Haskell's. Creating an optional typing linter that looks at that high a level, and doesn't cause much of pain, is not something I've ever seen. Type inference with generics, existential types, higher kinded types, algebraic types. That's where the real value is, and where modern typed languages are.

Aiming optional typing at where typed languages were 20 years ago is not going to help bridge the gap. If anything, I think it makes it wider, because then fans of dynamic languages think they understand the position of proponents of type systems when, in fact, they are looking at a strawman from the past.

[zimbatm]

It's more helpful to think about classes of errors and what the type system prevents.

A simple type system prevents type mismatch errors. A more complex type system like Haskell's might be able to encode interfaces and other rules but might also has it's limits.

The trade-off is usually that the more classes of errors you remove, the more complex the type-system becomes. Over the lifetime of your program, how much time did you spend hunting bugs vs time spent in long compilations or encoding all the rules.

I guess my point is that a lite-weight type system might also be enough.

[matt_kantor]

I haven't had an opportunity to actually use it, so take this with a grain of salt, but the optional/gradual type system of TypeScript looks decently expressive (http://www.typescriptlang.org/Handbook).

[incepted]

The main advantage of static typing to me is that it enables automatic refactorings.

Without types, it's impossible for tools to refactor your code safely without the supervision of a human. This leads to developers being afraid of refactoring and, ultimately, code bases rot and become huge piles of spaghetti that nobody wants to touch.

With a statically typed language, I'm never afraid to refactor whenever I see an opportunity to do so.

[ptype]

I hear this argument a lot, and I'm sure static typing is helpful when refactoring, but I have found that nothing is as important as tests when refactoring. I'd rather refactor dynamically typed code with good test coverage than statically typed code without good tests.

[japhyr]

I haven't done large-scale refactoring work before. How does typing help with refactoring? Is it because you know exactly what's being passed into a function?

[lispm]

The SBCL Common Lisp compiler detects some of those type errors at compile time - especially for declared types:

    ;     (> SECRET GUESS)
    ;
    ; caught WARNING:
    ;   Derived type of GUESS is
    ;     (VALUES STRING &OPTIONAL),
    ;   conflicting with its asserted type
    ;     REAL.
    ;   See also:
    ;     The SBCL Manual, Node "Handling of Types"
    ;
    ; compilation unit finished
    ;   caught 1 WARNING condition
    ;   printed 8 notes

Common Lisp allows optional type declarations. SBCL (this feature it has inherited from CMUCL) uses those as compile time type assertions.

[Animats]

The Python PEP 0484 approach, unchecked type hints, is strange, and probably a bad idea. There are good arguments for entirely dynamic typing, or entirely static typing, or optional static typing. Those have all been used successfully in other languages. But optional static typing without checking is new. (It may have been tried in some forgotten language, but it never made it into a mainstream one.) This is likely to create bugs, rather than fix them. The type you see looking at the code may not be the type being used there. This will confuse maintenance programmers. Worse, the compiler itself can't rely on the type info for optimization purposes. This limits optimizations in PyPy. Type checking is supposed to be performed by third party programs.

The syntax is backwards compatible, and this doesn't fit the language well. Forward references to types are handled via a really tacky kludge: "When a type hint contains names that have not been defined yet, that definition may be expressed as a string literal, to be resolved later." So, sometimes you write

    def foo(item: 'Tree'):

instead of

    def foo(item: Tree):

It's not stated in what scope 'Tree' is evaluated. So don't reuse type names.

Some type info is in comments:

    with frobnicate() as foo:  # type: int
        # Here foo is an int

Also, Python is getting header files (called "stub" files), like C. In most modern languages, such as Go and Rust, that's all handled automatically by storing type info in the object files. But in future Python, that will be manual.

There's function overloading, sort of. There's "@overload", but it doesn't really do anything at run time yet.

This whole thing is a collection of hacks in search of an architecture. This is the messiest type system of any mainstream language. Well designed type systems are hard enough. This is not one of them.

If this hadn't come from Python's little tin god, it would have been laughed out of the Python community. Stop him before he kills again.

[ubernostrum]

Also, Python is getting header files (called "stub" files), like C. In most modern languages, such as Go and Rust, that's all handled automatically by storing type info in the object files. But in future Python, that will be manual.

Stub files are not mandatory; type annotations can be in the same file as the code. This is perfectly legal, for example:

    def add(first_value: int, second_value: int) -> int:
        return first_value + second_value

Stub files exist because the syntax support didn't exist in earlier versions of Python, so if you write a library that supports older versions of Python you ship a stub file rather than causing syntax errors for a subset of your users.

[hahainternet]

> Probably this will still not be enough for Python enemies though.

Because it doesn't appear to actually do anything. It's a joke to call this 'static typing'.

[dbyte]

Well, it won't take much to have tools like Numba (http://numba.pydata.org/) taking advantage of such "type info" during optimization phase. On the other hand IDEs are already taking advance of those (see PyCharm).

And by the way the future looks bright as interesting PEPs are coming: PEP509, PEP510 (https://www.python.org/dev/peps/pep-0510/) and PEP511 for instance.

[mcguire]

What does mypy do if you make a call from code with type annotations into code without? Or vice-versa?

Has anyone ever gotten paid to add annotations to code that works?

Personally, I view type systems as like a safety line when doing work on a roof, and optional typing as having a line that might or might not be tied off.

[dzbarsky]

> What does mypy do if you make a call from code with type annotations into code without? Or vice-versa?

It's not able to analyze across such boundaries, but as you expand the set of typed code you get better and better coverage.

> Has anyone ever gotten paid to add annotations to code that works?

At Dropbox, we're starting to add type annotations to code. We're confident it will catch many bugs at lint time. Source: I work at Dropbox.

> Personally, I view type systems as like a safety line when doing work on a roof, and optional typing as having a line that might or might not be tied off.

At least it has the possibility of being tied off and catching you. Better than definitely not having a line.

[TazeTSchnitzel]

Having type annotations that are ignored at runtime would cause problems when you interact with code without type annotations, no?

PHP has a (unfortunately quite limited) set of type annotations, but the interpreter actually enforces them.

[ubernostrum]

Having type annotations that are ignored at runtime would cause problems when you interact with code without type annotations, no?

I seem to be doing this a lot in this thread, but...

This is your obligatory reminder that several statically-typed languages actually run on dynamic-language runtimes, because programmers don't like to think about the cost of polymorphism/generics, but implementers have to think about that.

[randallsquared]

How could adding ignored type annotations cause problems? Any problems that arise would have to be there without the type annotations...?

[tcopeland]

Here's a longer article (http://codon.com/consider-static-typing) with more background / history about static typing and Ruby.

This topic has been knocked around in Ruby-land for a while; I remember seeing Michael Edgar's LASER (https://github.com/michaeledgar/laser) static analysis tool including some work around optional type annotations, but seems like development there has stopped.

[infraruby]

InfraRuby is a statically-typed Ruby (compiles to the JVM). InfraRuby code runs on Ruby interpreters without modification: http://infraruby.com/blog/why-infraruby

[zeckalpha]

Re: Ruby: There's http://crystal-lang.org/, too.

[such_a_casual]

Aren't type annotations in python just documentation that's designed to look like it's not? Seems like it would make more sense to just create a documentation standard than develop a brand new syntax for documentation. Another idea that makes more sense to me is to use @decorators. I really don't understand why this syntactic hack is supposed to be a good idea.

[hardwaresofton]

Has anyone ever tried to implement optional typing in python with just generators?

It seems like a generator like @Signature(...input types, output type) would solve this problem with limited language changes, and would work in python 2/3?

[justusw]

Do you mean decorators? A solution like that exists and has been used for a while. It really is only usable for run-time type-checks. See here: https://github.com/dobarkod/typedecorator

With type annotations you can have static guarantees, which decorators will not be able to provide, as decorator methods will be called and resolved only during runtime. Objects imported from `types` hopefully will not.

[qwer]

Since I unit-test the heck out of my code, this doesn't really do much for me. Unit-tests test actual values (which is where the interesting bugs come from IMO) and give me more powerful refactoring capabilities than an IDE.

The real benefit I'd be looking for is the chance to give the compiler hints to speed up execution times.

[echelon]

I don't think this is true at all. You cannot write unit tests for all plausible values sent to your functions. If you omit manual type checking in your code or in the respective unit test, you may miss some subtle failure scenarios. Undergoing a refactor, maintenance, or change from other people (or even yourself at a later point in time) only makes this more possible.

I personally find that type safety cuts down the number of unit tests I write by half and makes refactoring work an order of magnitude easier to perform.

[dzbarsky]

Even if you ignore all the other benefits of type annotations, merely being explicit about the types of parameters your function accepts helps people reading your code.

If you're going to make the argument that you can just add comments with the types - well then that's exactly what mypy is doing, but if you do it in the mypy format you get static analysis of your types for free.

Static typing and unit testing aren't mutually exclusive.

[vinceguidry]

Ugh. I have hardly any type problems in my Ruby code. I've gotten very good at recognizing implicit state and capturing it in a properly instantiated object with a well-named class. I daresay that if you don't have this skill, a type system isn't going to help you much and you're going to get nasty bugs anyway.

The problem in Ruby is nils, and you'd have the same problem with the same solution in a static language; creation of a duck type. You can't get away from duck types, whether it's a maybe type or whether you perform nil-checking at the earliest possible opportunity. You learn with time and experience how to deal with inconsistent data. Ruby gives me the flexibility to do it without a lot of boilerplate.

[pka]

Yes, a type system is very much going to help you, even if you have that skill of "capturing implicit state". Because then a function you call, written by somebody else without that skill, may rely on global state in unexpected ways and still blow up in production.

A good type system doesn't allow you to call impure code from a pure function. It just won't compile. So your skill becomes useless because it's automatically verified by the compiler.

[actsasbuffoon]

Nils are a problem in languages like C and Java, but more powerful type systems completely eradicate the issue. You might enjoy taking a look at Crystal. It has Ruby-like syntax and a type system that makes nil-checks completely unnecessary.