I’m one of the main maintainers for the Google Common Lisp protocol buffer library:
I’ve been meaning to do a write up on it because it’s a heavily used library for my team, developed at Google. Internally we only use the SBCL Compiler (Steel Bank Common Lisp), while other compilers are also available for Common Lisp. In this post I will describe the benefits of getting Common Lisp libraries such as cl-protobufs compiling on multiple Common Lisp compilers.
Recently I’ve been working on getting cl-protobufs to compile with CCL and ABCL. Most of my coworkers have been wondering why I bother. It seems fairly unlikely people outside of Google will use cl-protobufs and we only use SBCL. I hope this is wrong; it would be great to see more adoption of protocol buffers in the Lisp community, and thanks to Ben Kuehnert we have the only fully proto2 and proto3 compatible protocol buffer library in Common Lisp. But I digress.
Why would I bother getting protobufs to compile in both CCL and ABCL?
The answer is fairly simple; we want cl-protobufs to be available to a wider array of Common Lisp users; and SBCL allows certain constructs that other compilers don’t, so having cl-protobufs run in CCL and ABCL we can find more bugs.
A clear example, and the majority of the bugs I found in cl-protobufs, is the use of make-instance for creating structure-classes. SBCL will allow this — you can’t set the slot values with make-instance, but you can create the instance. In CCL, make-instance expects all of the init-forms of the structure variables to be static, i.e. not functions, so fails. For example, if you have the struct
(defstruct my-struct
(foo (make-array ‘(3)
:element:initial-elements ‘(1 2 3))
:type (simple-vector))))
You can call (make-instance ‘foo) in SBCL but not in CCL or ABCL. Calling make-instance on a structure-class object is undefined behaviour, so this is a bug!
CCL, like SBCL, compiles down to machine code, but ABCL compiles down to JVM bytecode. Why would I care that cl-protobuf can run on ABCL? We find lots of interesting bugs by using ABCL.
Here’s a simple example:
(defun positive-p (my-int)
(declare (optimize (speed 3) (safety 0))
(type fixnum my-int))
(> my-int 0))
What should (positive-p nil) return? In SBCL we expect my-int to be a fixnum, and we tell the compiler to believe us. Speed 3 safety 0 means just treat it as a fixnum. The output on my laptop is T. In ABCL nil is not a fixnum, and there’s no way to cast it to a fixnum, so this is a type error.
Note:
(defun safe-positive-p (my-int)
(> my-int 0))
(safe-positive-p nil) throws an error in SBCL.
Why is this a problem? Why are you telling the compiler to compile at speed 3 safety 0? When code has to be fast, this makes it fast. I’ll discuss using the debugger in a later blog post.
So why try to get cl-protobufs working on different Lisp variants? It finds lots of bugs and undefined behavior!
Special thanks to Ron Gut and Carl Gay for looking over this post.
experimentalprogramming 