I’ve done several programming posts back-to-back, and I think it’s time to take a fun break. Today we are going to talk about the Banana bike, you can find it on Amazon.

The Banana bike is a balance bike with an aluminum body and air filled tires. It says it’s suitable for a toddler between the ages of 2 to 4, but I would think that is probably too large of a range, so lets says 2-3. It has a movable seat. I would say it’s best attribute, especially for the price range, is air filled tires. More expensive options like the Strider have foam tires, something that doesn’t roll well with me.

I got this bike for my daughter around 14 months. She wasn’t really able to ride it until around 20 months so you may want to wait for your toddler to be tall enough to ride it. On the other hand, she continually wanted to try to ride it, so maybe having a challenge like this is a positive.

Pros:

• The weels are air filled. This is something bikes at this price point rarely have.
• Very light. My daughter can easily move it around.
• Good size.
• Great price.

Cons:

• The handlebar swivels a full 360 degrees. A steering limiter would be appreciated.
• No breaks. At this age level thats probably fine.

Final opinion:

This is a fantastic bike for your toddler. At first I was concerned about the difference in quality a 60$ bike would be from say the 200$ Woom 1. I can tell you that your toddler won’t notice the difference. I might suggest the 30$ upgrade to a Schwinn balance bike to get the steering limiter, but it’s definitely not required. IF you have a toddler, you should definitely get out there and bike!

Note: A Woom 1, while expensive, may still be worth it. One can often resell them for more than the initial cost. The Banana Bike will probably have no resale value. You should however be able to use iether bike with multiple children.

Last post we created a server using the Lisp web server Huntchentoot and the cl-protobufs Lisp protocol buffer library. In this post we will discuss making a client to contact our web service, sending protocol buffer messages over HTTP. We will be using the HTTP client package Drakma. We will end the post by discussing improvements that should be made in future iterations.

The reader should find the code in my hello-world-client github repo. It contains three files:

1. hello-world-client.lisp
2. hello-world.proto
3. hello-world-client.asd

If you haven’t read the previous post please take a look here as we will be connecting to the service discussed therein.

Updates to the HTTP web server

In our last post our hello-world-server accepted a string and we directly read the string in as an octet-buffer. We did this to ease our testing, we could manually use a GUI web client such as Postman to send rest calls to our web server, inputting the octet-buffer we get from calling print on the octet buffer returned from serialize-proto. I would like to avoid any read calls in my Common Lisp code for security. Instead we will take in a base64-encoded string containing the octets, decode the base64 encoding with cl-base64 and use flexi-streams string-to-octets and octets-to-string to read and write the octet-buffer as a string.

The updated code can be found in hello-world-server/hello-world-server.lisp starting at line 19.

Code Discussion

I will omit the discussion of the hello-world.proto code and how it is compiled with the cl-protobuf asd additions. For a discussion on this please refer to my previous post here.

Hello-world-client.asd:

The useful information in this file is:

• defsystem: The Lisp system is called hello-world-client. 
• defsystem-depends-on: To load the system you will need to load cl-protobufs so we can generate Lisp code from the proto file.
• depends-on: We will use Drakma as our http client library.
• module: We have one module src.
• protobuf-source-file: This is an asdf directive given to us by cl-protobufs. It will look for a file hello-world.proto in our current directory and call protoc-gen-lisp on this file.
• file: A lisp file hello-world-client.lisp

Hello-world.proto:

The request and response schema definitions for the hello-world-server. This is copied from hello-world.proto in hello-world-server. 

Hello-world-client.lisp:

This is where the real work is done. We are starting the hello-world server locally and default to port 4242 with handler hello so we will set those as globals. We define a function call-hello-world which is what the client is attempting to do. It takes a name as either nil or string and address, port, and handler as optional keyword arguments defaulting to the aforementioned globals.  

We create the proto and then serialize the bytes using the cl-protobufs serializer:

(cl-protobufs:serialize-object-to-bytes proto-to-send)

Next we use flexi-streams to turn the bytes into a string octet, cl-base64 to base64 encode the string, and send it to the server with drakma.

(drakma:http-request
  (concatenate 
    'string address ":" port "/" handler)
    :parameters 
      `(("request" . ,(cl-base64:string-to-base64-string
                        (flexi-streams:octets-to-string
                          serialized-req)))))

The drakma library blocks until it receives a response, which will contain a base64 encoded stringified proto message. We simply reverse the base64 encoding, then call string-to-octets to get our octet buffer. We deserialize the proto message with cl-protobufs and print the response to the repl.

(print
     (hwp:response.response 
       (cl-protobufs:deserialize-object-from-bytes
          'hwp:response
          (flexi-streams:string-to-octets
            (cl-base64:base64-string-to-string response)))

We see

(call-hello-world “foo”)
=> “Hello foo”

as all good hello-world calls should show.

Final Remarks

The hello-world-server and hello-world-client code work as one would expect. There is however, too much boilerplate that has to go around this code. Having to manually call octets-to-string and string-to-base64-string and its reverse is cumbersome. What one should really do is have a function for the client that does this for you.

On the server side it is equally onerous to have to call base64-string-to-string and string-to-octets, and at the end it’s the reverse for every proto parameter. This should really be a macro that takes as an argument a list of (parameter-name . proto-type) and does the serialization for you. You could add an optional output proto-type/parameter-name to do the base64 encoding then string-to-octets at the end of the call. This would equate to having to add one macro call per handler.

In the next cl-protobuf hello-world post we will try to add these!

I would like to thank @rongut, @cgay, and @benkuehnert for their edits and comments.

Introduction

Recently I announced the release of CL-Protobufs, a full featured protocol buffer library for Common Lisp. It does not include a gRPC package. In this post, I will make an example hello-world server that will take a protocol buffer message as input and respond with a protocol buffer as output. I will describe some of the limitations of the given code and some ways to improve it. 

I will expect a basic understanding of the ASDF build system; if you don’t have one please see an example such as this.

The reader should find the code in my github repo. It contains three files:

1. hello-world-server.lisp
2. hello-world-server.proto
3. hello-world-server.asd

The .asd file is for integration with the ASDF build system. The proto file is where we define the request and response. 

Code Discussion:

The actual implementation is in the hello-world-server.lisp file. Let’s briefly go over the files.

hello-world-server.asd:

The useful information in this file is:

• defsystem: The Lisp system is called hello-world-server. 
• defsystem-depends-on: To load the system you will need to load cl-protobufs first. This is so we can compile the proto file into a file that Lisp will understand. Call this the Lisp proto schema file.
• depends-on: We will use hunchentoot as our web server.
• module: We have one module src.
  • protobuf-source-file: This is an asdf directive given to us by cl-protobufs It will look for a file hello-world.proto in our current directory and call lisp-protoc on this file.
    • Note: You can specify a different directory to look in with proto-pathname. 
    • Warning: You must have the lisp-protoc plugin compiled and in your $PATH to call this directive successfully. Please go to CL-Protobufs and follow the protoc installation directions.
  • file: A lisp file hello-world-server.lisp

If you have a background using asdf that may have been obvious.The defsystem-depends-on call to cl-protobufs and the protobuf-source-file call may be non-obvious to newer lispers.

hello-world.proto

This is where we place the request and response definitions. This will be compiled into a lisp file by protoc that lisp will be able to interpret with cl-protobufs. Note the package name is hello_world. To read more about protocol buffers please visit the Google developer documentation and to learn more about the API for cl-protobufs please visit the CL-Protobufs readme.

hello-world-server.lisp

This is where the application code lives. The defpackage tells us mostly what we already know, we will use Hunchentoot as our web server library and the cl-protobufs library which will allow us to serialize and deserialize our protocol buffer objects. We will also be using a package: cl-protobufs.hello-world. This package is defined as part of the code generated by the Lisp protoc plugin and contains everything we need to work with the native lisp protocol buffer objects:

• cl-protobufs.hello-world:request
• cl-protobufs.hello-world:response

A good tutorial on Hunchentoot can be found here. We define a handler for our server that will be at /hello and will be available on port 4242. We give a parameter request of type string. 

Inside of the request body is where the real protocol buffer work is done. The handler will take in a request which will be of type string. This string will actually hold a simple array, i.e. #(values) which will be the serialized protocol buffer request message ‘cl-protobufs.hello-world:request. As an example: 

(cl-protobufs:serialize-object-to-bytes
  (cl-protobufs.hello-world:make-request :name "fish"))

Will give us a serialized request object whose string name entry is “fish”. When run in the REPL we get:

#(10 4 102 105 115 104) 

We call read-from-string to get the array as an array and then call make-array to get an octet array. In cl-protobufs:deserialize-object-from-bytes we expect an octet-array so this is required. We deserialize the message with deserialize-object-from-bytes. If we received a message with a set name we return a response with a response string “Hello {name}”, otherwise we just return a response with response string “Hello”. Finally we serialize the created response and return it as a string.

Calling with my “fish” request I get the response:

#(10 10 72 101 108 108 111 32 102 105 115 104)

Then calling 

(cl-protobufs:deserialize-object
  'cl-protobufs.hello-world:response
  (make-array 12 :element-type '(unsigned-byte 8) 
                 :initial-contents 
                 #(10 10 72 101 108 108 111 
                   32 102 105 115 104)))

I get 

#S(CL-PROTOBUFS.HELLO-WORLD:RESPONSE :%RESPONSE "Hello fish" 
                                     :%BYTES NIL 
                                     :%%IS-SET #*1) 

As should be expected.  

Limitations and Extensions

The first limitation is the work we have to do to read the request. We shouldn’t have to call read-from-string at all, and then we shouldn’t have to make a new octet array. What we should do is make our call with octets-to-string and then call string-to-octets on the received message. This can easily be done by importing trivial-utf-8. I was using Postman to make the calls in debugging this simple hello-world-server. This should be easy to fix for the next post.

We shouldn’t even have to do that. We should be able to set the message request in the handler and the octets-to-string then deserialize call should be handled for us. The final serialize and utf-8-string-to-bytes call should also be handled like that. This is easily handled by around methods. Given the ingenuity of Lispers, there’s probably even better answers.

Final Remarks

I hope you found this example interesting. This is only a simple server. Also, I haven’t used Huntchentoot very much so I probably wrote some terrible server code. I 

1. Hope to improve my lisp server code.
2. Hope to get a gRPC server out someday.

Thank you for reading. I hope to see some interesting lisp code using cl-protobufs!

I would like to thank @rongut, @cgay, and @benkuehnert for reviewing this document.

I’m one of the main maintainers for the Google Common Lisp protocol buffer library:

https://github.com/qitab/cl-protobufs

I’ve been meaning to do a write up on it because it’s a heavily used library for my team, developed at Google. Internally we only use the SBCL Compiler (Steel Bank Common Lisp), while other compilers are also available for Common Lisp. In this post I will describe the benefits of getting Common Lisp libraries such as cl-protobufs compiling on multiple Common Lisp compilers.

Recently I’ve been working on getting cl-protobufs to compile with CCL and ABCL. Most of my coworkers have been wondering why I bother. It seems fairly unlikely people outside of Google will use cl-protobufs and we only use SBCL. I hope this is wrong; it would be great to see more adoption of protocol buffers in the Lisp community, and thanks to Ben Kuehnert we have the only fully proto2 and proto3 compatible protocol buffer library in Common Lisp. But I digress.

Why would I bother getting protobufs to compile in both CCL and ABCL?

The answer is fairly simple; we want cl-protobufs to be available to a wider array of Common Lisp users; and SBCL allows certain constructs that other compilers don’t, so having cl-protobufs run in CCL and ABCL we can find more bugs. 

A clear example, and the majority of the bugs I found in cl-protobufs, is the use of make-instance for creating structure-classes. SBCL will allow this — you can’t set the slot values with make-instance, but you can create the instance. In CCL, make-instance expects all of the init-forms of the structure variables to be static, i.e. not functions, so fails. For example, if you have the struct

(defstruct my-struct
    (foo (make-array ‘(3)
                     :element:initial-elements ‘(1 2 3))
                     :type (simple-vector))))

You can call (make-instance ‘foo) in SBCL but not in CCL or ABCL. Calling make-instance on a structure-class object is undefined behaviour, so this is a bug!

CCL, like SBCL, compiles down to machine code, but ABCL compiles down to JVM bytecode. Why would I care that cl-protobuf can run on ABCL? We find lots of interesting bugs by using ABCL.

Here’s a simple example:

(defun positive-p (my-int)
    (declare (optimize (speed 3) (safety 0))
             (type fixnum my-int))
    (> my-int 0))

What should (positive-p nil) return? In SBCL we expect my-int to be a fixnum, and we tell the compiler to believe us. Speed 3 safety 0 means just treat it as a fixnum. The output on my laptop is T. In ABCL nil is not a fixnum, and there’s no way to cast it to a fixnum, so this is a type error.

Note: 

(defun safe-positive-p (my-int)
    (> my-int 0))

(safe-positive-p nil) throws an error in SBCL.

Why is this a problem? Why are you telling the compiler to compile at speed 3 safety 0? When code has to be fast, this makes it fast. I’ll discuss using the debugger in a later blog post.

So why try to get cl-protobufs working on different Lisp variants? It finds lots of bugs and undefined behavior! 

Special thanks to Ron Gut and Carl Gay for looking over this post.

First, I hate the new WordPress wysiwyg editor, so apologies.

My wife is a college lecturer, she usually teaches 3-4 classes per semester. I am a software engineer, I work 40 hours a week, usually more. During this time of Covid it’s difficult to find childcare. Do you even want someone else to take care of your child? With my wife teaching this opens up a problem.

Previously I posted thanking Google for giving me time off to take care of my daughter. This is a great benefit, and I’m very thankful to Google for giving me this benefit. That being said, it has one really big downside. A large part of performance is based on impact, and it’s hard to have next level impact when you’re only working 60% of what everyone else is working.

I’ve been working at Google for almost 4 years. I got to L4 in about a year and a half to two years. It was a fairly easy promotion process, just sure you’re a relative mature programmer who can do some smaller level system design. It’s probably the level most engineers are at, and it’s the level we hire PhD grads at. I was happy to get the promotion, it came with nice benefits and all is great.

Two years later I have a daughter and another one on the way. I want to get a nice house for my family, a yard to play in and a garage to work on my motorcycle (and learn some woodworking). But in the area I live, it’s hard to afford a decent house with the salary an L4 at Google makes. One of my friends went to Mountain View when they got hired and were told it takes an L5 to buy a decent house within an hour commute, now it’s L6…

So I’m left with seemingly two possibilities. I can take the extra time off to spend with my daughter, or I can try to find the extra time and work for that promotion. It’s not a hard choice for me, my daughter will win out 100% of the time every time. That being said, it still makes me wonder. Would the extra work, the promotion, be worth it?

Thankfully the problem is simple, if you can spend more time with your family, you should do it.