[Tevo]

>recursion

I think one of the reasons recursion is often emphasized in relation to Lisp is because one of Lisp's core data structures, the linked list, can be defined inductively, and thus lends itself well to transformations expressed recursively (since they follow the structure of the data to the letter). But recursion in itself isn't something particularly special. Though it is more general than loops, and so it is nice to have some grasp on it, and how looping and iteration relate to each other, and it is often easier to reason about a problem in terms of a base case and a recursive case rather than a loop, at a higher level you will usually come to find bare recursion mostly counterproductive. You want to abstract it out, such that you can then compose your data transformations out of higher level operations which you can pick and match at will, APL-style. Think reductions, onto which you build mappings and filters and groupings and scans and whichever odd transformations one could devise, at which point recursion isn't much more than an implementation detail. This is about collections, but anything inductive would follow a similar pattern. Most functional languages will edge you towards the latter, and I find Lisp won't particularly, unless you actively seek it out (though Clojure encourages it most explicitly, if you consider that a Lisp).

>the pleasantness of being able to get 'inside' the program

Indeed, that's one of the things makes Common Lisp in specific particularly great (and it is something other contemporary dialects seem to miss, to varying degrees). It lets you sit within your program and sculpt it from the inside, in a Smalltalk sort of way, and the whole language is designed towards that. Pervasive late-binding means redefining mostly anything takes effect pretty much immediately, not having to bother recompiling or reloading anything else depending on it. The object system specifies things such as class redefinitions and instance morphing and dependencies and so on, such that you can start with a simple class definition, then go on to to interactively add or remove slots, or play with the inheritance chain, and have all of the existing instances just do the right thing, most of the time. Many provided functions that let you poke and prod the state of your image don't make much sense outside of an interactive environment.

There is a point to be made about abstraction, maths, and giving instructions to silicon (and metaprogramming!), but I'll have to pass for now. I apologize if this is too rambly, I tend to get verbose when tired.

[lispm]

> I think one of the reasons recursion is often emphasized in relation to Lisp is because one of Lisp's core data structures, the linked list, can be defined inductively

Lisp was used in computer science education to teach "recursion". We are not talking about software engineering, but learning new ways to think about programming. That can be seen in SICP, which is not a Lisp/Scheme text, but a computer science education book, teaching students ways to think, from the basics upwards.

Personally I would not use recursion in programs everywhaere, unless the recursive solution is somewhat easier to think about. Typically I would use a higher order function or some extended loop construct.

[troad]

Not at all too rambly, very interesting, thank you. Your answer makes intuitive sense to me; I'll ponder over it.