Doing Lisp stuff

I am documenting (for myself at least) the things I found with Lisp programming of particular documentary interest.

SLY

I am using SLY. I haven't used SLIME, but SLY was just easier out of the box with the typical creature comforts (code completion, etc.) In the World Wide Web, when slime is mentioned, like an interactive function in Emacs, I could just replace it with sly and be on my merry way.

SLY implements the most portable aspects of Common Lisp. That doesn't a clear stepping debugger. The one who maintains SLY also implemented a form of stepping using SLY's really nice Stickers: https://github.com/joaotavora/sly-stepper (Stickers are like hyper-printf. You highlight a sexp and it will trace it and you can tell it to drop into a debugger at that point.) SBCL does implement a stepping debugger actually. It sort of works, but it's not the first thing I am thinking of.

Debugging CL

So, it might be hard to think about how one would debug without stepping. Other Lisps even allow stepping. Each and every time though I read about this 'defciency', seasoned Lispers said it's not something to be missed. It does help of course; Emacs Lisp has a stepper and it has helped me a bit. But what other languages lack though is a running environment sitting next to your code. The challenge for me at least (and I don't see it yet), is to exploit this when errors do occur. (Errors do not mean necessarily "unroll the stack", but an opportunity to fix something live.)

Refactoring

I came across this problem after I wrote some functions with really long names. SLY doesn't have refactoring facilities. It's probably possible, but I think this goes back to the fact that CL developers might just not care enough. This is particularly true since you have something like rg + wgrep. wgrep in particular has something I wanted for awhile. I can search for a symbol and then start editing it in the ripgrep buffer. The astute thing about this is that I probably have a lot more control over renaming. IDE renamers might not rename documentation.

There is always gotchas to this type of task, so I am going to not say much more until I did more.

CLOS

I started with a rough sketch of an idea around my toy vthreads. The idea was to bash a few functions together and see how far I can do without locks and such. I'll say "sort of" but not entirely there. I started to read the documentation a little deeper of bordeaux-threads and have a better idea what to do. This library is a wrapper around different CL implementations, providing a single API across those implementations

In the first commit, it implements it with a bunch of global variables. While OK to start, I might want to have independent sets of threads handling different workloads. I can put them into wrapper data structures.

defclasss in CLOS seem like more advanced defstructs. It's probably important to get most of the idea of what traditional classes look like and instead see that classes here are different. They provide a type, they have instance variables, they can spit out accessors, they can be reflected on, have their instanstiations controlled, be controlled by metaclasses (the "Meta-Object Protocol"). But they don't smash functions with data.

I started with this:

(defclass virtual-thread ...)
(defclass supervisor-thread ...)
(defclass worker-thread ...)

and then made generic functions ... think of it as a protocol over objects:

(defgeneric add-thread (target thread))
(defgeneric start-thread (thread))
(defgeneric stop-thread (thread))

(defmethod add-thread ((target supervisor-thread) (thread virtual-thread))) ;; I use this, the supervisor uses the one below.
(defmethod add-thread ((target worker-thread) (thread virtual-thread)))

(defmethod start-thread (thread supervisor-thread)) ;; maybe i don't need these - threads should start right away
(defmethod stop-thread (thread supervisor-thread))
(defmethod stop-thread (thread worker-thread))
(defmethod stop-thread (thread worker-thread))

What's fun is that there is no locus of attention. This is called "multiple dispatch" and the methods are called "multimethods" (my add-thread can be implemented in probably two clear ways in a single-dispatch approach - that is, like in Javs, Python, etc. This gives one clear place to send one into another without declaring "responsibility", methinks.)

What's also fun in CLOS is that you are really not limiting yourself to just the types you write. If it makes sense to add your virtual thread to a system created thread, just do it (but in the SBCL implementation, it starts it right away; I don't know how it would make sense.) Let's say you could:

VTHREADS> (defmethod add-thread ((target sb-thread:thread) (thread worker-thread)))
#<standard-method virtual-threads:add-thread (sb-thread:thread worker-thread) {1003856BB3}>

... it defined it! ¯\_(ツ)_/¯ Note that sb-thread:thread is a struct, not a class.

CLOS and visitor patterns

I started to watch Peter Seibel's talk in relation to his book (halfway done right now.) He talked about visitor patterns. If we judged through CLOS itself, it would make instance sense. You be the judge. I took a different approach to visiting from here: https://www.baeldung.com/java-visitor-pattern:

;; Do in lisp what is in this tutorial https://www.baeldung.com/java-visitor-pattern
(defclass element ()
  (
   (more-elements
    :initform nil
    :accessor element-more-elements)))

(defclass json-element (element) ())

(defclass elt-visitor () ())

(defun test-visiting ()
  (let ((base-elt (make-instance 'element))
        (new-elt (make-instance 'json-element))
        (base-visitor (make-instance 'elt-visitor)))
    (push (element-more-elements base-elt) new-elt)
    (accept base-elt base-visitor)
    (dolist (elt (element-more-elements base-elt))
      (accept elt base-visitor))))

(defgeneric accept (elt visitor))
(defmethod accept (elt visitor) (visit visitor elt))
;;(defmethod accept ((elt element) (visitor elt-visitor))
;;  (visit visitor elt))
;;(defmethod accept ((elt json-element) (visitor elt-visitor))
;;  (visit visitor elt))

(defgeneric visit (visitor elt))
(defmethod visit ((visitor elt-visitor) (elt element))
  (format t "Generic element"))
(defmethod visit ((visitor elt-visitor) (elt json-element))
  (format t "Json element"))

REPL

In the above example, I just tried if adding a method over a completely different data type was possible.

If anything, this has highlighted for me how nice it is to just toy around with stuff quickly. I feel like I need to learn some more Python tooling, but I know for a fact that quick redefinitions in the plain REPL is just not a thing. What is stopping me from finding out is a lack of interest at this point.

Also, what's relevant is that classes are redefineable, along with their existing instances. Now, I did run in a tricky corner, where it thought it was using a class definition of the same class in a different package (the package imported the symbols from the first project.) But otherwise, it just mostly works...

I also leave my computer on; just put it to sleep. This is the first case where I have a programming environment still running for days, forgetting about all the stuff I was smashing into it sometimes :D

smartparens

In Emacs, there are a lot of 'structural' editing tools. I tried a few, but I am going back to smartparens. I do other things in Emacs and I don't like anything too fancy for my non-modal head (very much the reason I don't like VIM.) There are others: paredit, lispy-mode, parinfer. But at least with smartparens I can used it with other languages. I did have to tweak the defaults. Slurping and barfing by default was bound to keys that I didn't like.

I also use rainbow-delimiters-mode. With brightly-colored parentheses (or anything that needs to be balanced), it's really easy to see.

But with that in mind: I got used to working with Lisp code quick. My own, like any language, gets dense. But it doesn't bother me as much. With some smartparens tools, it's actually quite manageable to rewrite code.

Wrapping up

I made this Emacs Lisp function for myself:

(defun bpo/insert-shrug ()
  (interactive)
  (insert "¯\\_(ツ)_/¯"))