top | item 17416276

SICP Goodness – Why you don't need looping constructs

93 points| lvguowei | 7 years ago |lvguowei.me | reply

126 comments

order
[+] sametmax|7 years ago|reply
Actually you don't need another contruct than goto. You don't even need functions.

But for some reason given the choice, people use them.

It makes things easier to reason about for most day to day problems.

I train people in programming for a leaving. "Map" is harder to understand than "for" to most of my students. "Reduce" harder than manual looping. Recursion harder than iteration. Immutable harder than mutable.

Now I always end up teaching the entire toolbox. Those are all very useful concepts that I use in production and I do want my students to have a large solution set to their future problems.

But there is a clear pattern in my teaching experiences.

[+] CodeArtisan|7 years ago|reply
>Actually you don't need another contruct than goto. You don't even need functions.

>But for some reason given the choice, people use them.

There is no choice when it come to functional programming because the paradigm is based on Alonzo Church model of computation known as Lambda calculus which is declarative: Everything is an expression or, more specifically, a function. Because of that, statement like goto are forbidden. Goto is from the Alan Turing model of computation known as the Turing machine which is imperative; everything is a statement.

Guy Steele said that the lambda expression in Scheme is the ultimate GOTO because it's like an unconditional jump while remaining declarative and functional.

[+] pdpi|7 years ago|reply
> It makes things easier to reason about for most day to day problems.

> I train people in programming for a leaving. "Map" is harder to understand than "for" to most of my students. "Reduce" harder than manual looping. Recursion harder than iteration. Immutable harder than mutable.

Careful! Tools that are easy for a trained expert to use and tools that are easy for a newbie to grasp are not only different sets of things — I'd go as far as to say that overlap between the two is very rare indeed.

[+] YeGoblynQueenne|7 years ago|reply
Beware the conclusions drawn from the first read through the writings of our functional programmer forefathers (and the second, and the third), for "a little knowledge is a dangerous thing". The unprepared mind of the imperative programmer is especially prone to following the revelations of the old masters before achieving real understanding.

A while ago, I was working in company X, that sold a web platform for job boards, with multiple big and famous clients. One time, there was a bug in our platform, that had to do with a for-loop with an always-true condition. The loop -which was in our core platform code and so deployed in every single website we maintained- caused untold havok, as resources disappeard down a black hole and all our servers dropped to their knees and asked for mercy.

In the aftermath of the incident, when an explanation had to be given to those UpStairs running the show, it was put to them that the cause was an (effectively) unconditional loop. Now, as a lowly junior dev, I was not present in the deliberations but, given what followed, I can imagine the conversation:

  Those UpStairs: "How can we ensure that the same thing never happens again?"

  Senior Software Person: "Well, there is no way to ever be 100% safe. A for-
  or while-loop can always go wrong."

  Those UpStairs: "Is there any alternative to for- and while-loops?"

  Senior Software Person: "Not really. I mean, there is recursion of course..."

  Those UpStairs: "Recursion? What is that?"
Shortly after, we had a big emergency meeting to discuss What Should be Done to Avoid the Same Problems in the Future, where we were instructed to not use the looping constructs of the langauge in which our platform was written, anymore. Instead, we should only ever use recursion to write loops.

The language of our platform was C#, mind. A language that (to my knowledge, and certainly at the time, around 4.0) does not even support tail-call optimisation. Not to mention, it's just as easy to go into an infinite recursive loop as it is to go into an infinite imperative loop, if not easier.

Suffice to say that nobody followed the instructions given from On High, either because they found it too much work, or because they found it too stupid, or because they were just busy pulling their hair out.

[+] discreteevent|7 years ago|reply
It's just as easy (if not easier) to make a mistake with a recursive call such that it never terminates. Why did your team suggest it as an alternative and why was it accepted? TBH the whole story seems very hard to believe.
[+] acjohnson55|7 years ago|reply
I wouldn't replace loops with recursion, generally speaking. It's probably at least as hard to write and understand explicit recursion as it is a for loop, and it's still fairly prone to off-by-one errors.

Explicit loops have a use, as does explicit recursion, but in the vast majority of cases, I use higher-order functions, like map, filter, flatmap, and fold.

The real problem is choosing tools that have more power than you need. This increases the surface area for potential bugs and it requires future reader to parse more code that's just there for plumbing.

[+] Const-me|7 years ago|reply
C# has foreach loops, available for decades i.e. since very beginning.

C# also has LINQ, available for many years i.e. since 3.5.

Also, when management is asking “How can we ensure that the same thing never happens again?” they rarely interested in language level. In my experience, they don’t care whether the problem was caused by for or any other f-word in the code. This question is usually about QA and deployment workflows.

[+] calebh|7 years ago|reply
C# 4.0 had Linq, which can replace loops in most cases.
[+] bjoli|7 years ago|reply
I would say that schemes looping constructs are too limited for comfortable use, but that tail call elimination makes implementing your own constructs very easy. I have reimplemented racket's for loops for guile using named let loops [0] which was not only pretty easy, but it was also easily understood and optimised by the guile compiler. The end result is as fast as a hand rolled named let for almost all cases (I still haven't found one where it is measurably slower, but I have only tested it in the repl where the compiler can make lots of assumptions)

For that reason alone I think tail call elimination is a must, especially in today's transpiling world.

[0]: https://bitbucket.org/bjoli/guile-for-loops

[+] soegaard|7 years ago|reply
"For that reason alone I think tail call elimination is a must, especially in today's transpiling world."

This is an important insight.

I feel the JavaScript implementors are dragging their on getting tail call elimination widely supported.

[+] kccqzy|7 years ago|reply
ES6 used to mandate TCE but it was removed. No browser other than Safari implements TCE for JavaScript. Apparently the main reason is an inaccurate stack trace on exception.
[+] goalieca|7 years ago|reply
I made it a goal of mine about 10 years ago to avoid explicit for-loops. It turns out using declarative style makes things more readable and safer and better optimized by the compiler. I can usually tell the purpose of the loop by just reading the function name (map, filter, ..) and the lambda. Sometimes parallel comes for free. Plus, all those off by one errors, iterator errors from erasing elements, etc. are all eliminated :)
[+] jstimpfle|7 years ago|reply
> It turns out using declarative style makes things more readable and safer and better optimized by the compiler.

I don't think most of my loops are straightforward maps or filters of a single iterable. I need to reference various data sources and execute actions with side-effects. To do that using a functional framework, something like Haskell's mapM ("monadic map") is required. But that brings quite some mental overhead and I find it way too constraining for the typical case.

> more readable and safer and better optimized by the compiler.

I doubt these are in general qualities of non-builtin constructs vs builtin ones. But yes, too much builtin and non-orthogonal functionality leads to detrimental complexity.

In any case I don't think it matters much to most compilers whether you write a loop using syntactic sugar or as labels and jumps.

[+] sametmax|7 years ago|reply
For loop don't have to mean counters. We had automated for loops for years, generators and comprehensions that are basically map/filter integrated in syntax. E.g you have map and filter in python but they are rarely used for that reason.

No need to be dogmatic.

[+] pjc50|7 years ago|reply
However, if iteration and tail-recursion are equally powerful constructs, the implication that recursion is morally superior to iteration seems misplaced.
[+] noelwelsh|7 years ago|reply
You can use tail recursion to create abstractions in a compositional manner that cannot be created compositionally with iterative constructs.

For example, if you can express a finite state machine (FSM) as a number of mutually (tail) recursive functions. Then you can reuse this FSM as part of a larger FSM via a tail call. You can't do this with the iterative constructs in C et al---the whole FSM must be defined in one spot.

[+] willtim|7 years ago|reply
I think the point was that recursion makes a better foundational primitive. For example, with Java and C#, their (somewhat arbitrary) Iterator and IEnumerator library interfaces are baked into the syntax of the language (e.g. the foreach loops). This is not a good position to be in.
[+] tspiteri|7 years ago|reply
And when parallelization and vectorization come into play, iteration is a much better fit, at least in my intuition.
[+] rs86|7 years ago|reply
Easy to prove termination and analyse recursion in general via structural induction.
[+] Const-me|7 years ago|reply
I prefer looping constructs over recursion, almost every single time.

Loops are closer to what CPU actually does. A compiler is therefore simpler to implement and will likely contain less surprises.

It’s easier to use a debugger on procedural-style or OO code (step in/over/out, local variables) compared to recursive functional style advertised by SICP.

In most runtimes, call stack is limited to a small size, couple of megabytes. Stack overflow exception is hard to handle in reasonable way. When instead using a loop with a stack container for state, it can be much larger, also easier to handle overflows i.e. implement soft and hard limits.

[+] akashakya|7 years ago|reply
> Loops are closer to what CPU actually does. A compiler is therefore simpler to implement and will likely contain less surprises.

I dont find the argument convincing. By that assembly is closer to what CPU actually does, why not write code using goto (or binary)? We all can agree on its easy to write reasonably good code high-level languages. I think it because high-level languages are closer to the problem domain (or our thinking), hence it's easier to define the solution in these languages. same for recursion.

> It’s easier to use a debugger on procedural-style or OO code (step in/over/out, local variables) compared to recursive functional style advertised by SICP.

I feel exactly opposite. OO by definition have state bound to object outside the method (not actually local variable), so you have to watch all the objects method is affecting. Not pleasant experience when you have hundreds of objects. A recursive code on the other hand usually depends only on the local arguments so you can only focus on current step.

> In most runtimes, call stack is limited to a small size, a couple of megabytes. Stack overflow exception is hard to handle in a reasonable way. When instead using a loop with a stack container for the state, it can be much larger, also easier to handle overflows i.e. implement soft and hard limits.

Well, depends on the platform you are working on. For desktop at least it's definitely not "a couple of megabytes".

I feel that most of the time choice is between readability and performance. Sometimes maybe the solution is inherently iterative/recursive making choice easier. ymmv though.

[+] hajile|7 years ago|reply
Closer to the CPU isn't a valid argument. Most major programming concepts don't exist in a CPU.

* CPUs are dynamically scoped

* CPUs are weakly typed

* CPUs are dynamically typed

* CPUs only compare to zero

* CPUs have no way to represent OOP

and so on

All that said, tail calls are very close to how the CPU works. They are just GOTO with additional parameters. If you implement tail calls with dynamic scoping, the parameters would be registers and the tail call would be a change to the instruction pointer. That would be very close indeed.

[+] willtim|7 years ago|reply
Explicit recursion is really a foundational primitive upon which other abstractions should be built, including looping constructs. It is similar to goto, in that it is a form of unstructured programming. Functional programmers prefer using maps and folds.

Loops are not necessarily close to what a CPU actually does either, especially one supporting vector operations or out-of-order execution. Loops imply an ordering which may or may not be actually exist and many compilers are complicated by their presence.

Debuggers are a problem for any suitably high-level abstract code. Would a step-by-step word-at-a-time debugger help to debug a SQL query or even a complex arithmetical expression from Fortran? There is no getting away from the fact that high-level domain-specific code needs high-level domain-specific debuggers to be built alongside.

[+] mrami|7 years ago|reply
To be fair, Scheme implements tail recursion, which converts properly prepared recursion into an iterative loop. Usually converting the recursive call into a GOTO. So if used correctly, recursion in scheme isn't a stack issue.

That said, I love my for loops, too.

[+] mattfrommars|7 years ago|reply
Does anyone know a good time in one's career to take time out and spend it on going through SICP and videos lectures with it?

I have a STEM (non-CS) degree but very recently, started working as a programmer. Doing my first internship at a 'techy' corporate company. One of the things I'd be doing on the side is to go through https://mitpress.mit.edu/books/elements-computing-systems. The video introduction to it just gives me goosebumps to this day. I don't know why. It's fascinating.

Than there is other paradigm books such as "pragmatic programmer", "Thinking Like A Programmer", "Design Patterns: Elements of Reusable Object-Oriented Software", etc.

So far, I've done plenty of reading on Algorithm & Design Structure and leetcode here and there. In the past, I didn't qualify for internship at Google, Amazon or M$ as a software developer/engineer. I feel the reason why I didn't was because I didn't have a "computer science" keyword mentioned anywhere on my resume. Hence, my hope is after getting through "Element of Computing System" & Design Pattern, I'll be able to shove in "computer science" keyword one way or another on my resume.

[+] paidleaf|7 years ago|reply
> Does anyone know a good time in one's career to take time out and spend it on going through SICP and videos lectures with it?

I guess any time is good. It's generally a class you take as a freshman or sophomore. People who attend top engineering schools generally played around with these topics in high school or even in elementary school.

Also, it looks like MIT stopped teaching SICP.

https://news.ycombinator.com/item?id=11628080

> In the past, I didn't qualify for internship at Google, Amazon or M$ as a software developer/engineer. I feel the reason why I didn't was because I didn't have a "computer science" keyword mentioned anywhere on my resume.

Or you are competing against the best of the best CS students from around the world. People who aren't blown away by introductory stuff like elements of computing systems or who are just looking into SICP. I really don't think missing a keyword is why you didn't get the internship at google, amazon or microsoft.

> Hence, my hope is after getting through "Element of Computing System" & Design Pattern, I'll be able to shove in "computer science" keyword one way or another on my resume.

I've never had computer science on my resume though I majored in it. For my internships, I just listed the courses/projects I worked on. For my first job, I just listed the degrees I had ( BS and BA and the gpas for both ), along with the projects I worked on and the internships I had. After that, I just listed the university and degrees and my work experience. "Computer science" isn't something I ever listed in any resume. But then again, I was never really good at writing resumes.

I think you are really overthinking things but feel free to do what you want. If you are so intent on putting computer science on your resume, why not just major in computer science?

[+] davidgrenier|7 years ago|reply
I believe this is the place to ask a question I've been wondering about. WebAssembly currently doesn't support Goto labels and several people are pushing for it: https://github.com/WebAssembly/design/issues/796

Now I'm aware that mutually recursive functions can be gracefully used to implement state machines and that the issue of jumping into the middle of a loop from a goto seems to be prized.

The former (mutually recursive function) seem to solve that problem, not necessarily in almost a general senses, however perhaps preventing some of the most nasty things Goto can do.

I'd like to know if mutually recursive function requires the underlying machinery to provide gotos. In my view, that would fully justify them. If not, then tail-recursion might be the only thing missing from WebAssembly.

[+] tonyg|7 years ago|reply
It's often said that a tail call is a "goto with arguments" [1]. It's also often said that "lambda is the ultimate X," for many different X [2]. Just adding proper tail calls to WebAssembly would do wonders for its expressiveness [3], in ways that adding lesser forms of goto would not.

[1] I can't find (or remember) who or where this originated! It seems to be folklore now, but it must have started somewhere.

[2] Many examples here: http://library.readscheme.org/page1.html

[3] Felleisen, Matthias. ‘On the Expressive Power of Programming Languages’. In Proc. ESOP. Copenhagen, Denmark, 1990. https://pdfs.semanticscholar.org/2798/5884a1947b656424b0956f...

[+] simias|7 years ago|reply
Why would you be worried about the "nasty things Goto can do" in an assembly language? IMO assemblies are about exposing the underlying machine's capabilities (be it physical or virtual) in the most straightforward way possible. I don't expect an elegant abstraction, I leave that to the higher level languages.

I don't know much about WebAssembly so I don't know if gotos have a place or not but if they're useful and the only reason for not adding them is cargo-culting "gotos are teh 3v1l" it's a bit silly.

[+] MaxBarraclough|7 years ago|reply
> nasty things Goto can do

It's not a source language. It should be elegant, but not in the same way. An intermediate language should lend itself to mechanical synthesis and analysis (in whichever balance is most appropriate), and shouldn't worry about how writeable and readable it is to humans.

Java bytecode, for instance, is very 'simple' in a way, but no-one in their right mind writes code in Java bytecode assembly language.

There is such a thing as a bad intermediate language. These things need careful and skillful design, but shouldn't be mistaken for source languages.

When Dijkstra wrote "Go To Statement Considered Harmful", he was referring to source languages, not to machine languages or intermediate languages.

[+] madmulita|7 years ago|reply
Luckily I had been exposed to recursion previously, what blew my mind while watching the SICP videos was the implementation of car and cdr using only lambda and the realization that the language could go without variables.
[+] LfLxfxxLxfxx|7 years ago|reply
Lisp only replaces looping with recursion, it doesn't eliminate it. APL does.
[+] kazinator|7 years ago|reply
Lisp idioms like (mapcar function list) replace both looping and recursion with an aggregate operation, similar to APL; it's just not obsessively condensed into a single character syntax.
[+] erikb|7 years ago|reply
The title should be "why you don't need looping constructs if your language supports tail recursion".
[+] tzahola|7 years ago|reply
Technically you don’t need loops, that’s true.

But from a pragmatic standpoint, it’s just too easy for someone to turn a tail-recursive function into non-tail recursive, causing a stack overflow eventually.

So when working in a large team, with run-of-the-mill programmers, I’d stick with loops.

[+] chriswarbo|7 years ago|reply
Would an annotation help, e.g. a `tailcall` keyword, which will trigger an error for non-tailcalls?
[+] Double_a_92|7 years ago|reply
Yes, you can use recursion to loop... What was this post trying to tell me?