SPECIAL NOTE:
I am extremely new to Common Lisp, and macros. I’m willing to bet that anything shown here can be improved.
Been reading through Paul Graham’s On Lisp which is probably not a real “Intro to Lisp”, but well within the grasp of my super interlect. Well at least it was today when I had a breakthrough on macros. I had some introductory stuff on macros when I was perusing through various ancient texts of the language of Clojure. However, I never really looked into to macros much since I was trying to figure out things like how to set up a web server in Clojure. For whatever reason, call it divine intervention or maybe vision inducing exhaustion, it dawned on me to read On Lisp. I figured that if I were to learn macros, I might as well learn them from a person who I can only guess is Lisp: Paul Graham. Turns out, this was a good choice.
Now to start, understanding what a macro is, I have to introduce the word “homoiconic”. It’s a really fancy word that means that a Lisp code is compromised of data structures. In simple terms: Lisp is built on lists. For example:
(+ 1 2 3)
Is that code or a list? Yes. Why is this important? Because any code can be from combining lists together, or by taking a smaller list and expanding it out into code. Unlike other languages that might use a method to evaluate a string representation of code, or some kind of preprocessor directive, you are creating real code that needs nothing special to read/evaluate it. A kind of useless example of this is taking a method, and making it into a macro:
(defun average (x y)
(/ (+ x y) 2))
And the macro:
(defmacro average-macro (&rest args)
`(/ (+ ,@args) ,(length args)))
Completely obvious, am I right? Nothing seems odd or foreign… HRAHARAHRAHAR
Ok sooo… The main thing about macros is what is supposed to be evaluated at the time the macro is expanded, and what to ignore. Essentially when the macro is expanded, you can have method calls to create code to add to the final expanded code, or code that will remain untouched to become a part of the final product. It’s kind of an odd thing to understand. After all, it means you have to think in more than just one dimension. Think of it like you are concatenating a string. In something like c# you might have a string built like this:
string function ReturnFinalString(name) {
return CreateFormalGreeting(name) + ". It is nice to meet you on this fine " + RetrieveTheDayName(DateTime.Now);
}
Ignore that I am using the + string concatenation junk instead of string.format, or StringBuilder. Just wanted something easy to read.
As you can see, there are parts of the string that don’t need any changing, because they are in a final state. (Meaning that’s exactly how they will appear in the final string) However, there are other parts that need the result of an evaluated method. Once those method returns are resolved, you get a single string. A macro isn’t too much different in concept. Instead of creating a list of characters, you are creating lists of code.
Ok so now it’s time to break that macro down. Remember, macros are about what to evaluate, and what to ignore.
(defmacro average-macro (&rest args)
First two parts should be easy to understand, but the third one is misleading somewhat. What looks like two parameters is really just one: args. The &rest is just syntax for “What ever arguments are left.” For instance, say this macro was more generic where it would take in a function (add, subtract, whatever), and also some numbers. The syntax would be:
(defmacro do-something (methodToUse &rest args)
The call would be like:
(do-something add 1 2 3 4)
“add” would be the “methodToUse”, and 1 2 3 4 would be “args”. An important point is that anything passed the named parameters gets lumped into a list. So if I was completely stupid (I’m only partially), and didn’t know what I was supposed to sent into “do-something”, I might do this:
(do-something add 1 "a" someVariable derp)
The “args” would be (1 “a” someVariable derp). Most likely the macro would throw an error if you tried to use it in the REPL, but the point is just to realize how “&rest args” works.
That was the easy part. Now for the weird part.
`(/ (+ ,@args) ,(length args)))
Looks like a dragged my face across the keyboard, or PERL. Either way, the syntax is pretty goofy looking at first… And even after that. However, the idea behind it isn’t that bad.
Remember how I said something about it all being about what to evaluate, and what to ignore? Well right off the bat, the macro is told to ignore everything. Why? The back quote.
`(... stuff here...)
Without any special characters, the macro will just ignore everything within the parenthesis. So if I did this:
(defmacro add-this ()
`(+ 1 2))
Everything in there will not change when the macro is expanded. So basically that’s a really stupid macro. What’s the best thing to do with a useless macro? Make it more complex.
(defmacro add-this ()
`(+ ,(+ 1 2) 2))
New syntax, the comma. The comma is the opposite of the back quote. It basically says not to ignore what’s there, and to evaluate it. Before the final product, the (+ 1 2) will be evaluated, and that result will end up in the final expansion. Without that comma, the final product would be:
(+ (+ 1 2) 2)
But it really is:
(+ 3 2)
The difference is subtle, but it’s everything. What this means is that in the version with the comma, the (+ 1 2) is being evaluated, and its result (3) is then applied to the macro expansion. Given this example, it’s hard to understand what use that could be, but it will be completely obvious when the last part of the original macro is explained.
`(/ (+ ,@args) ...
So the comma is used to make sure that parts are evaluated for the final expansion, but what is @? @ is an interesting concept that also has large implications, despite the textbook explanation being underwhelming. The , followed by the @ means that it will strip the outermost parenthesis off. Yeah, boring BUT critical. I had explained that &rest args dumps all remaining arguments into a list, right? Well what if you needed to use the values in “args”, but not as a list.
What if I created a simple macro like:
(defmacro add-that(&rest args)
`(+ ,args))
And the call would be:
(add-that 1 2 3)
Now + can’t be used with list, because it expects single values like:
(+ 1 2 3)
Now in this macro, I did this:
(defmacro add-that(&rest args)
`(+ ,args))
Which says to ignore the +, and evaluate “args”. Well we know that the macro call stuff 1 2 3 into one list. So as is, if this is expanded, it will look like this:
(+ '(1 2 3))
Bad. How can that be fixed? The @ sign.
(defmacro add-that(&rest args)
`(+ ,@args))
This will expand to:
(+ 1 2 3)
As you can see, the ‘(1 2 3) “args” was evaluated to just “1 2 3” before the final product.
With that all in mind, time to go back to the original semi-useless macro:
(defmacro average-macro (&rest args)
`(/ (+ ,@args) ,(length args)))
Read out loud it says:
1) ignore the /
2) ignore the (+
3) remove the parenthesis from args
4) ignore the )
5) get the length of args
6) ignore the )
The interesting part to note is step 5. This shows the power of macros, even in a lame example. To make sure the average of x number of er numbers, there has to be a total count of numbers to divide by. When looking at the original function that this macro was based on:
(defun average (x y)
(/ (+ x y) 2))
If you want to find the average of 3 numbers, you hosed. BUT since the macro is able to take the length of the args list, and then apply that value to the final expansion; You can easily find the average of however many numbers you want. Remember, the “,(length args)” part is able to resolve that count BEFORE the final expansion is evaluated. This can be shown using macroexpand-1:
;; When the macro is expanded
(macroexpand-1 '(average-macro 1 2 3 4 5))
;; Becomes:
(/ (+ 1 2 3 4 5) 5)
As you can see, the final product has 5 in place of ,(length args). Magic.
;; When the macro is expanded
(macroexpand-1 '(average-macro 1 2 3 4 5))
;; Becomes:
(/ (+ 1 2 3 4 5) 5)
This macro is kind of pointless, because it could be done with a better function than the original one. However, the concepts are what matter.
Now for something more interesting. What if you don’t like the way that functions are declared? I kind of do. I started with Racket (Scheme) before I ended up at Common Lisp. In Racket, there is a uniform definition.
(define (name argument) ...)
But with Common Lisp, the “name” is outside the argument list. Well with macros that can change. And rather easily:
;;(DEFUN X () (+ 1 X))
(defmacro define (head &rest body)
`(defun ,(car head) ,(cdr head) ,@body))
And it would be called like this:
(define (test-it x y) (+ x y))
Which is syntactically different than the Common Lisp way:
(defun test-it (x y) (+ x y))
How did I do that? There is a little trickery involved, but nothing hackish. Just opinionated. (Or Convention over Configuration as some might call it) Time to break it down.
(defmacro define (head &rest body)
Here’s the start of the trickery. I know that anything that is in “body” is what’s left over after “head” is set. Right off the bat, that could be an issue. After all, “test-it” not only needs the function name, but has two parameters in “x” and “y”. Doesn’t take long to realize that “x” and “y” will somehow have to be accounted for. This means they will end up in “head”, or “body”. In this case, “head”.
`(defun ,(car head) ,(cdr head)....))
And that’s what I mean about opinionated. I expect that “head” is actually a list. A list that starts with the function name, and then has all the parameters. That’s where car and cdr come in. (car means grab the first item in the list, and cdr means grab all BUT the first item in the list) My macro expects that the first item in the “head” list is the function name: “test-it”. It then expects anything else in “head” is a parameter. When the part above is evaluated, it will ignore “defun”, get the first item “test-it”, and the get the list of remaining items. Since cdr returns a list, the function definition doesn’t need anything special done to the parameters.
`(defun ,(car head) ,(cdr head) === (defun test-it (x y )...
After that, the rest is easy.
,@body === (+ x y)
Ok, how about something even more interesting. Say you wanted to completely change the syntax for lambda expressions. This is the normal way:
(mapcar (lambda (x) (+ 1 x))...)
I say, “To hell with that.” I also say, “I want something like this:”
(-> x y z (+ x y z))
Not only did I remove the “lambda” keyword, I removed the parentheses around the parameters. HOW CAN THIS BE?!?! More trickery.
(defmacro -> (&rest arguments)
(let ((reversed (reverse arguments)))
(let ((last (car reversed))
(undone (reverse (cdr reversed))))
`(lambda ,undone ,last))))
And that’s it. Good night.
Oh, you want an explanation, so be it. The idea behind this is that I will expect the call to have this: n1 n2 n3 … nx (method call). This means that when all items smashed into “arguments”, everything but the last item in the list is a parameter. So no matter how long the “arguments” list is, the most important thing to remember is that ONLY the last item is the body of the function. I think I just wrote the same thing twice, but whatever. Now to break it down.
(defmacro -> (&rest arguments)
Just the typical start to a macro.
(let ((reversed (reverse arguments)))
Here’s something a bit new. As the other macros I showed didn’t need any fields, I didn’t use let. This time the “arguments” field is taken, and then reversed. Why? So that I can do this:
(let ((last (car reversed))
(undone (reverse (cdr reversed))))
I took “reversed” from the original let statement, and then retrieved two things from it: The head (Which is actually the last item of the original “arguments” list, but now it the first in the reversed list), and all the rest re-reversed.
arguments == ‘(x y z (+ x y z))
reversed == ‘((+x y z) z y x)
last == ‘(x y z)
undone == (x y z)
The reason why “undone” has to be reversed it that when the “arguments” list was reversed, the “x y z” ended up as “z y x”. If I hadn’t reversed it, the parameters in the method would be reversed. This would be a large issue.
(z y x (+ x y z)) ;;BAD
(x y z (+ x y z)) ;;GOOD
Last part:
`(lambda ,undone ,last))))
Oh yea, there was an actual part to be expanded in the macro. Why wasn’t ,@ used? Remember that the lambda syntax is:
(lambda (x y z) (+ x y z))
Meaning that the parenthesis that surround “undone” and “last” are OK to keep.
Done yet? Too bad, because here’s one more. The last macro has room to improve from a structure stand point (as I am new to Lisp), but also on a conceptual level. The next big thing would be to not have to bother with declaring the method parameters for the lambda clause. I want to turn something like this:
(mapcar #'(lambda (x y z) (+ x y z))
Into:
(?> (+ z y z) '(1) '(2) '(3))
Where the ‘(1), ‘(2), and ‘(3) are the values that will be passed into “mapcar”. As you can see, I have no declaration of the lambda or its parameters. Crazy, I know. The idea is to have a macro not only create the needed mapcar and lambda syntax, but to generate one parameter declaration for every argument… no matter how many arguments there are. The caveat here is that the generated argument names shouldn’t be something so common as x, y, or z. For this demo I’ll be using a list of numbers that have the “%” character appended to them. So instead of the “x y z” above, it will be “1% 2% 3%”. The final expansion should be:
(mapcar #'(lambda (1% 2% 3%) (+ 1% 2% 3%))
Before I get to the macro, I need a couple of helper methods.
First I need something that will generate a list of numbers as string. The reason why is that later I will concatenate each number string with the “%” character to have a basis for the created parameter names.
;;This creates a ascending list of numbers as string.
;; (numbers 5) == ("1" "2" "3" "4" "5")
(defun numbers (count &optional (finalList '()))
(if (= 0 count)
finalList
(numbers (- count 1) (append (list (write-to-string count)) finalList))))
Now I need something to concatenate the number strings with “%”, AND turn them into legal parameter names.
;;This is used to take two concatenated strings, and turn them into potential parameter names
;; (create-parameter-name "1" "%") == 1%
(defun create-parameter-name (x y)
(read-from-string (concatenate 'string x y)))
Finally I need a method to use the other two method to create the needed list of parameters:
;;This is used to dynamically create the needed parameters for a lambda expression
;; (create-placeholders "a" "b" "c") == '(1% 2% 3%)
(defun create-placeholders(arguments)
(let ((argumentCount (length arguments)))
(mapcar #'create-parameter-name (numbers argumentCount) (make-list argumentCount :initial-element "%"))))
And here is the macro:
(defmacro ?> (conversion &rest args)
`(mapcar #'(lambda (,@(create-placeholders args)) (,@conversion)) ,@args))
;;Expanded
(macroexpand-1 '(?> (+ 1% 2% 3%) '(4) '(5) '(6)))
(MAPCAR #'(LAMBDA (1% 2% 3%) (+ 1% 2% 3%)) '(4) '(5) '(6))
1) Ignore (mapcar #'(lambda(
2) Create the parameters based on the length of args
3) Ignore ) (
d) Evaluate the contents of conversion (the passed in method), and remove the surrounding parenthesis
5) Ignore the )
VI) Evaluate args, and remove the surrounding parenthesis
7) ignore )
Yay. Time to stop for now.