Declarative Languages
Lecture #5

Purpose: Functions, recursive and iterative: the sordid truth and other stuff

5.1 Let's talk recursive

Suppose we are (still) trying to write a function which takes two arguments - a list and a number - and if the number is = to the any element of the list then return the position of that element in the list. (So if the number matches the first element in the list return 0, if it matches the second return 1, and so on.) If the number isn't found in the list, we'll return nil.

Here's one solution to the problem:

(defun position-one (list number)
(if list
(if (= (first list) number)
0
(let* ((pos (position-one (rest list) number)))
(if pos
(1+ pos)
nil)))))
And here's another:
(defun position-two (list number &optional (pos 0))
(if list
(if (= (first list) number)
pos
(position-two (rest list) number (+ 1 pos)))))
Both these solutions are recursive - the functions call themselves. There are two official tips for how to write recursive functions (these apply to any language, not just lisp):
• you need to find a way to reduce the problem to a slightly simpler case, in the event that this time round wasn't the last pass, and
• you must ensure that whatever the data to your function it will terminate, i.e. not keep calling itself forever.
Also there is one unofficial tip:
• keep it as simple as you can possibly get away with. In my humble opinion position-one above is somewhat on the hairy side.
• the equivalence of the empty list and logical false - handy for tests with if
• the use of &optional to allow functions to take a variable number of arguments
• tail recursion...
What is tail recursion? Answer: a tail recursive function is one in which the recursive call is the last thing it does. Why should that matter to anybody? Answer: because (a) the compiler can transform it into a loop, which will run faster and (b) if the compiler can write it as a loop then so can you, which will save you all the aggro of having to think recursively in the first place. :-) More on this later.

5.2 Sinking without a trace

If you're stuck on seeing how these recursive functions work (and there are plenty of further examples on the solution sheets), you should consider tracing function calls. For example, if you evaluate
(trace position-two)
- after defining the function! - and then call it with test data, you will see a printout of all the recursive calls and their return values.

• Turn tracing off with (untrace position-two) or even (untrace) to untrace everything.
• Be warned that tracing Common Lisp functions (as opposed to tracing your own code) can get a little bit hectic. The reason for this is that there is a strong possibility that the lisp system will be calling these functions anyway, e.g. to handle redisplay of the listener brought on by printing out all that tracing info.
Sometimes tracing won't be appropriate, and you may find yourself peppering your code with print statements to get it debugged. If you need to do this, you might as well make it quick'n'dirty rather than bothering with the finesse of format.  The function print takes a lisp object and prints it (as the read-eval-print loop would, i.e. as if with the format directive ~s as opposed to the ~a which we've been using and which leaves output more human-readable). The printing is preceded by a newline and followed by a space. [See also definitions of prin1 and princ.]

5.3 Divertimento: several functions, a macro and a special operator

5.4  Let's get iterative

Lisp supplies you with a whole load of macros and functions to make iteration simple. These range from fairly basic "loop forever until I tell you to stop" constructs to highly complex packages supporting specialized data structures, or lazy evaluation (see appendices to Cltl2). Let's start with the macro dolist:

(dolist (variable expression &optional result-form)
form-1
...
form-n)
This macro is used for iterating along a list. The expression is evaluated, and it should come out as a list. The variable variable is bound to the first element in the list and the forms form-1 ... form-n are evaluated as an implicit progn. Then variable is bound to the second element in the list and the forms are evaluated again, and so on down the list until it runs out.

If result-form is present it is evaluated (with variable bound to nil, fwiw) and its value is the return value of dolist, otherwise dolist returns nil.

If you want to bail out of a dolist loop before the list has been used up, use the macro return. This takes an optional return value (to be evaluated), otherwise dolist still returns nil.

Examples:

(defun my-reverse (old-list)
(let* ((new-list nil))
(dolist (elt old-list)
(push elt new-list))
new-list))
(my-reverse '(alpha beta gamma))  =>  (gamma beta alpha)

(defun position-three (list number)
(let ((count 0))
(dolist (var list)
(if (= var number)
(return count))
(incf count))))

A similar macro is dotimes:
(dotimes (variable expression &optional result-form)
form-1
...
form-n)
This macro is used for iterating while incrementing a counter. The expression is evaluated, and this time it should come out as an integer. The variable variable is bound to 0 first time round the loop, then 1, etc until the loop has been been executed expression times. If result-form is present it is evaluated (with variable incremented one final time to give you the number of times the body of the loop was executed); return values and use of the macro return are exactly as for dolist. Examples:
(dotimes (foo 10 foo))  =>  10

(setf bar 0)
(dotimes (foo 10)
(incf bar))           =>  nil
bar                     =>  10

(let* ((list nil))
(dotimes (i 5)
(push i list))
list)                 => (4 3 2 1 0)

There is a fun example with palindromic strings on the HyperSpec page on dotimes.

5.5 Functions can iterate too

If you need to bomb down a list in a hurry (and it's your valuable time as a slothful typist I'm thinking of here), you should consider the following incredibly neat function.

(mapcar '1+ '(1 4 9 16))  =>  (2 5 10 17)
(mapcar 'evenp '(1 2 3 4 5))  =>  (nil t nil t nil)
(mapcar 'type-of (list "my" (+ 1 2) 'sons))
=>  (simple-base-string fixnum symbol)
(mapcar 'length '(() (()) (()()) (()()()) (()()()())))  =>  (0 1 2 3 4)
So what does mapcar do? At its simplest it takes a function and a list. The function is passed members of the list, one at a time, and its return values are collected into a fresh list which is itself returned from mapcar.

Hang on. Takes a function? How do you pass a function around? Answer: simple, just quote its name. As above.

Another example:

(defun my-copy-list (list)
(mapcar 'identity list))
Stuck? Try tracing (see 5.2 above) the function which is being passed to mapcar, i.e. 1+, evenp, ..., identity in the above.

5.6 More on mapcar

In all the examples above, mapcar was passed a function which itself took one argument. Now, this need not be the case. More examples:

(mapcar '+ '(1 2 3) '(4 5 6))  =>  (5 7 9)
(mapcar '< '(1 2 3) '(3 2 1))  =>  (t nil nil)
(mapcar 'list '(one two three four) '(1 2 3 4) '("I" "II" "III" "IV"))
=>  ((one 1 "I")
(two 2 "II")
(three 3 "III")
(four 4 "IV"))
The function handed to mapcar should take as many arguments as there are lists. First time around, it is called with the first element of each list, and so on. If the lists turn out to be of different lengths, mapcar halts as soon as any of them run out:
(mapcar '/ '(0 1 2) '(3 2 1 0))  =>  (0 1/2 2)  ; no division by zero
Finally, if you want to do iteration like this without the overhead of collecting results into a list, call mapc instead:
(mapc 'print (list foo bar baz))
will print out the values of foo bar and baz. For want of anything better, mapc returns the first list passed to it, and programmers generally throw this value away. In other words, mapc is called for side effects only.

5.7 Practical session

1. Use dolist to write a function which adds together (i.e. sums) all the elements in a list.
2. Use dotimes to implement the factorial function.
3. Implement the following Common Lisp functions recursively (n.b. give them new names or you will upset your lisp session)

4.          length copy-list reverse
5. Repeat the above (apart from reverse whose solution was given above) with dolist. Which ones get easier? Which get harder?
6. Rewrite the function
1. (defun multi-type-of (list)
(if list
(cons (type-of (first list))
(multi-type-of (rest list)))))
2. with mapcar
3. with dolist
Which of the three approaches do you prefer?
7. Use mapc to write a function which takes a list and prints a dot for each member of the list:
1. (dotty '(a b c d e f g h i j k l m n o))
...............
=>  nil ; actually you need _two_ functions to do this ;-)
5.8 Suggested activity / exercises etc

MAKE SURE YOU UNDERSTAND IT ALL, NOW

1. Go through your lecture notes and check you are happy with everything to date. Fire up a lisp session and experiment with every function etc introduced so far (one-liners will do) to check your understanding. Anything you don't understand from the notes: look it up in Graham, Cltl2, the HyperSpec. Come and ask me if you're still stuck. See my office door for consultation times.
2. Make very certain that you can by now write simple lisp functions. Examples:
1. take two numbers (i.e. write a function of two arguments) and sum them
2. take a list of two numbers (i.e. write a function of one argument) and sum them
3. take an arbitrary list of numbers and sum them
4. take a list of numbers and return a list of all the numbers which were negative
I would hope by now that you can do all four of these; I would accept it if you could produce working versions of the first three; I would be worried if you can't get the first two right. Come and ask me if you're stuck. See my office door for consultation times.
3. Complete any exercises from previous weeks and any unfinished practical work. I have deliberately set lots so that there is more than one example of each concept. Try without the solutions first but - failing that - look them up. Come and ask me if you're stuck.
4. Go through the above notes and answer all the rhetorical questions.
5. Experiment with different arguments to type-of to see how many different return values you can get it to produce. (Rate yourself:- 4: good, 5: very good, 7 great, 9: excellent)
6. Implement my-consp in terms of type-of and my-listp in terms of my-consp.
7. Why would
1. (defun last (list)
(nthcdr (1- (length list)) list))
be considered inefficient?
8. What's wrong with
1. (defun copy-list (list)
(reverse (reverse list)))
as a means of copying lists?
9. The body of the macro defun is an implicit progn. Name another operator whose body has this property.
10. Both dolist and dotimes have an optional result-form. Programmers rarely use this feature. Why do you imagine this is?
This week is your best chance to sort out lingering problems. Assignment coming next week.