Note: This post had to be update, WordPress seemed to mangle some of my writing making it less then readable (deleted some text and merged a lisp macro and a c++ function).
Whats the special sauce of Lisp? Macros! And Code as Data etc…
What is a macro? It’s a way to write code that generates more code, or it’s a way to improve/increase the syntax of your language. This confused me for quite a while, so lets go with an example.
Recently, I was writing code that contacts a database in C++. First it creates a connection to the database, runs a query on said database, does some processing collected to the database (the database sends back only a row at a time), and then closes the connection. Since my query is slow (the table is HUGE) it frequently times out giving a retry-able error. The code looks like:
void QueryDatabaseWithWork() {
DatabaseConnection conn;
RunQuery(&conn);
DoWork(conn.Result());
assert(conn.Finish() == 0);
}
So we create a database connection, run a query, do some work with the returned rows and then call Finish(). Finish() will return either 0 for success, 1 for retry-able error or -1 for non-retry-able errors. So you tell me just wrap this in a loop. (Note wordpress keeps undoing changes to the below code…)
void QueryDatabaseWithWorkRetry() {
char status = 1;
for(int attempts = 0;
attempts <= 10 && status == 1;
attempts++) {
if (attempts > 0) {
sys.sleep(attempts * Milliseconds(10));
std::cout << "On retry: " << attempts;
}
RunQuery(&conn);
DoWork(conn.Result());
status = conn.Finish();
}
assert(status == 0);
}
This code is fine… I even submitted a CL that looks suspiciously like this. The issue is you must do this EVERY place you have a a query with a bit of work. Of course you could parameterise the query and put the work in a lambda, a runnable or another construct, but this separates your query from your work, and causes a level of indirection that makes code hard to read.
Instead we can extend our language to create a control flow syntax (like while or for) that will give us retries.
(defmacro retry-query (status &body body)
`(loop :for attempts from 0 to 10
:while (= status 1)
:do
(when (> attempts 0)
(sleep (* 10 ,attempts))
(format nil "On attempt ~a" (+ attempts 1)))
,@body))
Note: Before we discuss this macro note this is not “production” grade, the macro has variable capture and there are plenty of arguments that should be modifiable, but that would make our example more complex then it should be.
What this will do is allow the user to specify a “status” variable that, if equals 1, will retry the body. We see in the loop body that it will retry up to 10 time and log after every failure. We have also added a back-off in the form of a sleep as before.
Now lets use this macro:
(defun query-database ()
(let ((conn nil))
(retry-query
status
(setf status (run-query 'my-query)))))
First, we require the user to update status. This makes this macro more useful, as long as we set status to 1 for retry anything can have a retry-able back off, not just this kind of query. Another option is to require body to return the status (retry-able or not) and put the setf in the macro. I’ll give this in the bottom of the page.
I said macros generate code. We used the retry-query macro to allow us to get retries for our code, but what lisp will do is expand
(retry-query
status
(setf status (run-query 'my-query)))
So after macro-expansion Lisp will compile:
(defun query-database ()
(let ((conn nil))
(LOOP :FOR ATTEMPTS FROM 0 TO 10
:WHILE (= STATUS 1)
:DO
(WHEN (> ATTEMPTS 0)
(SLEEP (* 10 ATTEMPTS))
(FORMAT NIL "On attempt ~a" (+ ATTEMPTS 1)))
(SETF STATUS (RUN-QUERY 'MY-QUERY)))))
Which is much like the C++ code from above. Instead of having to deal with worrying about abstract classes or interfaces to pass through functionality, or lambda function for dealing with work, we just use our retry-query macro.
I know it’s hard to understand the use of macros for new Lispers, or non-Lispers. I hope I’ve made you a little interested!
I leave you with Lyra and Stitch.
P.S.
Here’s a macro that at least removes variable capture and adds at least minimal flexibility:
(defmacro retry-query ((status &key
(log-stream nil)
(num-retries 10))
&body body)
(let ((attempts (gensym "attempts")))
`(loop :for ,attempts from 0 to ,num-retries
:while (= ,status 1)
:do
(when (> ,attempts 0)
(sleep (* 10 ,attempts))
(format ,log-stream "On attempt ~a"
(+ ,attempts 1)))
,@body)))
or if you know that your body will return 1 for retriable:
(defmacro retry-query ((&key (log-stream nil) (num-retries 10))
&body body)
(let ((attempts (gensym "attempts"))
(status (gensym "status")))
`(loop :for ,attempts from 0 to ,num-retries
:while (= ,status 1)
:do
(when (> ,attempts 0)
(sleep (* 10 ,attempts))
(format ,log-stream "On attempt ~a"
(+ ,attempts 1)))
(setf ,status ,@body))))
UPDATE: You can also write macros in a language with C-Style syntax in Dylan (example given by @cgay):
define macro with-retry
{ with-retry (#key ?log-stream:expression,
?num-retries:expression = 10)
?:body
end
}
=> { block (exit)
for (attempt from 0 to ?num-retries)
if (attempt > 0)
sleep(attempt * 10);
format(?log-stream | *standard-output*, "Attempt %d",
attempt);
end;
if (?body ~= -1)
exit()
end
end
end block
}
end macro with-retry;
experimentalprogramming 