unCLog

I'm continuing to muddle through various ideas for Enterprise Lisp. One thing I think that the Lisp community needs is better library management. This means:

System definition
New System installation
System maintenance

ASDF (and MK-Defsystem and a few other defsystems) serve the system definition role admirably. My own defsystem-compatibiliy aims to make it easier to use multiple defsystems simultaneously (*). ASDF-Install does a great job grabbing new systems. The missing piece is system maintenance: making sure you have the latest of every and that it all works together. What follows is my proposal for improving the situation.

Desiderata

System maintainers want to do a good job but managing version numbers is difficult, time consuming and often goes undone.
System consumers just want things to work
If the process that handles system maintenance runs on the client, then you add another layer of things that must be maintained. Therefore, put as much computation on the server. (**)

The view from above

The system maintainer consists of the following processes:

Checker - monitor known systems for changes. When changes occur, generate tickets for Tester.
Tester - Test system installation (similar to ASDF-Install-Tester). Note that Tester is distributed across different machine architectures and Lisps.
Reporter - Use the results from Checker and Tester to produce pretty pictures, update RSS feeds, send text messages, generate press releases, call out the national guard, etc.
Clients (that's us!) - communicate with Webber to see if systems need to be updated
Webber - Communicate with clients (generally ASDF-Install) to determine if systems need to be updated.

Questions and Answers

This is the section where careful readers get to catch my errors and e-mail me as to how to make things better... So pay attention.

Q: What do you mean by known systems? A: Initially, Checker will work with ASDF-Installable systems. The main point of this, however, is to let Checker know where a system is to be found so there is no reason that other systems could not be registered for the service.

Q: How does checker know when a system has changed? A: Checker does two periodic checks on known systems. First, it compares the :last-modified date of an HTTP head request. If the last-modified date of the systems tarball is greater than the date Checker saw before, then the system may have changed. Secondly, Checker uses the system definition to build a system signature: a list of system files and file-write dates. Checker can compare the signature it has with the new signature to see if files have been added or removed and to see if file dates have changed. Note that occasional false positives are OK.

Q: What about *features*? Won't they mess with this signature you're talking about? A: Yes, features are a problem. For the curious, there are 52 systems on the Cliki that contain features in their system defs (out of about 250). These systems contain 40 different features. Most of them are operating system or Lisp implementation related. A few are more specialized. I propose to get around the features problem by using a custom reader to grab every file in a system definition.

Conclusion

The system described above is being actively developed and I hope to have something beta-testable sometime this work (oh, oh, that sounds like a promise. I have to stop writing now and get back to coding). Please let me hear from you if I've missed something obvious or subtle.

(*) To be fair, defsystem-compatibility (DSC) currently only supports ASDF and EKSL's system definer: Generic Load Utilities. However, only time and lack of clamor prevents DSC from working with others.

(**) Yes, this gives a central point of failure but web server and hardware is up to the task.

Home | About | Quotes | Recent | Archives