Common Lisp and Reader Macros… You Better Sit Down For This

Fair warning: I only learned about reader macros today, so my terminology may suck. Well, most likely it would still even if I had a couple weeks. So it will suck more. Say, 20% more suck.

In case you didn’t get a chance to read my primer on macros (Let’s be honest, you didn’t. It’s ok, I’ll be fine.), this next macro may not look familiar. This is the post of which I type. A quick recap is that the macro would take this:

  (?> (+ 1% 2% 3%) '(4) '(5) '(6))

And create this code:

  (MAPCAR #'(LAMBDA (1% 2% 3%) (+ 1% 2% 3%)) '(4) '(5) '(6))

Essentially it took the original expression, and created a lambda with dynamically created parameters. (The 1% 2% 3% part) The macro “keyword” if you will is the “?>”
Here’s the macro minus the helper methods. (The whole thing is the last section of the same post from above.)

  (defmacro ?> (conversion &rest args)
   `(mapcar #'(lambda (,@(create-placeholders args)) (,@conversion)) ,@args))

Now that’s pretty cool and all, but it still has to follow the general rule of Common Lisp: Every statement is encased in parenthesis. After all, the macro starts with “(?>”. What if there was a way to take that and remove any need for outer parenthesis? Turns out there is, and the term is “reader macros”. What’s the big difference between the two types? Glad you asked. Would have been a long, uncomfortable pause if you hadn’t.

Unlike normal macros that are expanded AFTER post read. That is, taking what is on the text file (.lisp file), and converting it to be compiled. So basically, read is pre-compile time. Why does this matter? Because you can instruct the reader on how to “digest” stuff from the .lisp file, and then prepare it for compile time. So like a normal macro that tells the compiler how to expand itself to compile ready lisp code, a reader macro tells the reader how to transform the text into useful pre-compile code. And yes, a reader macro can create a normal macro. In fact, that’s what this example will show.

Ok, so back to the normal macro above; It still has the normal {} encasing. What if I wanted to encase it with something else? Say something like #| |.

 #| (+ 1% 2% 3%) '(4) '(5) '(6) | 

Normally this would cause all sorts of problems, a linguistic middle finger if you will. Don’t worry, lisp won’t get away with that.

(set-macro-character #\| (get-macro-character #\)))
(set-dispatch-macro-character #\# #\|
  #'(lambda (stream char1 char2)
      (let ((pair (read-delimited-list #\| stream t)))
        (cons (read-from-string "?>") pair))))

So as far as I understand it, somewhere there is a table of symbols that are reserved by Common Lisp. For instance, the right parenthesis or ). The “set-macro-character” basically allows the addition of new symbols like ]. It is telling the reader that ] is ).

Next part is the actual macro. If you take away the #\ characters from “#\# #\|” you’re left with “#|“. This is what the reader will look for in order to read the code that follows the “#|“. Essentially this has set up the ability for the reader to take in “#| |” without throwing an error. The in between is what the rest of the code handles.

For whatever reason (Again I’m new to this) the “set-dispatch-macro-character” needs a method that takes in three parameters: “stream” “char1” “char2”, the significance for this I have yet to see. However, the “stream” parameter is important to this macro. The stream is the code that is read in after the reader finds “#|“. The needed code is the start of the stream until it hits “|”. In this example that would be:

  (+ 1% 2% 3%) '(4) '(5) '(6)

To match the original macro, there still needs to be the leading “?>”:

  (?> (+ 1% 2% 3%) '(4) '(5) '(6))

And that’s where the “cons” comes in. I simply appended “?>” to the read in code “(+ 1% 2% 3%) ‘(4) ‘(5) ‘(6)” to give me:

  (?> (+ 1% 2% 3%) '(4) '(5) '(6))

Which is exactly the code I needed to call the original non-reader macro. So not only was it possible to completely ignore lisp syntax, it was also possible to nest a normal macro within the reader macro.

 #| (+ 1% 2% 3%) '(4) '(5) '(6) |
    -> (?> (+ 1% 2% 3%) '(4) '(5) '(6))
      -> (MAPCAR #'(LAMBDA (1% 2% 3%) (+ 1% 2% 3%)) '(4) '(5) '(6))
        -> 15