| |||
|
opening it up with Common Lisp |
|||
|
|||
|
Favorite weblogs Lisp Related Politics Other home Recent Readings
Book review: Darwinia
Summer reading: Spin
Runner
the Omnivoire's Delimma
the Golem's Eye  
|
|
RDF Triples in XML (After all, where else would you put them?). Even after everything is in RDF, you still need to find someplace to put it and a way to write it down. Big disks answers the first question but the second has turned out to be surprisingly hard. Since XML has become the one (markup language) to rule them all, it was no surprise that people turned to it for the answers. Unfortunately, dealing with everything has a way of pulling in competing constituencies and those pesky subgroups tend to pull things into a muddle. This left RDF/XML of several diverse forms each very successful but each with its own problems. This paper presents another way to look at stuff that, as far as I know, has gone on to become quite successful.
IT Conversations
more "real" C++ macros via template metaprogramming If it didn't end up being so ugly, it would be cute.
This is from a longer article by Todd Veldhuizen referenced by Scott Meyers (it's on the web somewhere but I seem to have misplaced it Someone should create a language that lets you do this without having to jump through so many hoops. It could work per taking source code as inputs and writing out new source code with the whole power of the language behind the transformations. A language like that would blow C++ out of the water in terms of popularity. What's that. Oh, sorry. I went off my meds again.
SPAM on the lam? Not for me! It used to be that most of my SPAM went into junk. Lately, however, a great deal of stuff that looks as if it should be easy to categorize is ending up in my inbox. Grrrr. I'm guessing that I should try retraining my SPAM filter (in OS X's Apple Mail) or switch to something like Michael's Tsai's SpamSieve. I used SpamSieve once before and liked it -- but why pay when Apple Mail seemed to be doing as good (or at least nearly as good) of a job? Is anyone else experiencing this?
Feedback: good and bad It's ironic I received both high praise and an indictment for my Lisp software on the same day! Both, however, were deserved. Well, I know I deserve the indictment and I'm pretty sure that the praise is justified too. First, I heard that CL-Markdown's new extensions mechanism
That cheered me up. Later in the day, however, I heard that:
Which is quite true. I suppose it would be sort of OK if I didn't list any dependencies but only listing some of them really opens the door for frustration. Besides, it's easy to pull all of this stuff and include it in the web pages automatically. Things to do:
Thanks for both the kind words and the less kind ones. I want Lisp to succeed and that's only going to happen if the barriers to entry become ever lower. I just heard about RIFE which sounds like RAILS only it's Java, etc... The nice thing is that Lispers could start working on LIFE which has gotta be good, right? The linked interview has an interesting comment differentiating Rails from RIFE:
Update: CL-Markdown again Thanks for Frank Schorr for noticing that CL-Markdown sometimes failed to produce output. Actually if it failed once, it never stopped failing but when it succeeded, it never failed. It was a file compilation order dependency problem that didn't occur in my usual development environment. I'd been thinking that having a system compile an ASDF system in every order allowed by the dependencies would be a good way to catch certain bugs... it would have caught this one (assuming that a lot of additional infrastructure was in place <smile>). When, on when, will someone kick me in the butt so that I can get back to Enterprise Lisp? New files are up on Common-Lisp.net.
the Human Condition I like conditions. I like them a lot. I like to use them to describe bad program states instead of strings. (e.g., (defmacro defcondition (name (&rest super-conditions)
slot-specs format &body args)
(flet ((massage-slot (slot-spec)
(cond ((atom slot-spec)
`(,slot-spec
:initarg ,(read-from-string (format nil ":~a" slot-spec))))
(t
slot-spec)))
(massage-format-arg (arg)
(cond ((atom arg)
`(slot-value condition ',arg))
(t
arg))))
`(progn
(export '(,name))
(define-condition ,name ,super-conditions
,(mapcar #'massage-slot slot-specs)
(:report (lambda (condition stream)
(format stream ,format
,@(mapcar #'massage-format-arg args))))))))
and this lets us write (define-condition record-number-too-large-error
(invalid-record-number-error)
((record-count :initarg :record-count))
(:report
(lambda (condition stream)
(format stream
"Record number ~a is too large for this store. Store size is ~a."
(slot-value condition 'record-number)
(slot-value condition 'record-count)))))
as (defcondition record-number-too-large-error
(invalid-record-number-error)
(record-count)
"Record number ~a is too large for this store. Store size is ~a."
record-number record-count)
(the record-number slot is inherited). It's not much but its enough to make writing simple condition like this almost as easy as writing an error message as string. Programming environments are (in part) about reducing impedance and about making it easy to do the right thing.
Book review: Darwinia I very much enjoyed Robert Charles Wilson's Spin and I was looking forward to reading Darwinia. I'm glad I read them in the order I did because I thought Spin was superb and Darwinia both trite and silly. The characters are well drawn and relationships matter but the central premise -- that earth (and the rest of the universe) is essentially a library book come to life and is rewriting itself because of the attack of computer viruses on steroids; that everything we know is composed of a vast cosmological battle between sentience and these viruses... well, it's an interesting idea. Darwinia feels as if it could have been great but it isn't. It feels tacked together from several weaker ideas and grand notions and never quite evolves beyond itself. Oh well, you can't always write a masterpiece.
Update: CL-Markdown I've corrected another few bugs in CL-Markdown and improved the extensions a bit.
It actually works pretty well (I suppose I shouldn't be surprised! Smile)
:explain that :box, will you? One of Common Lisp's darker arts is optimizing functions. It's a beautiful idea: don't worry (too much) about performance until you've found the critical sections of your code; then tell the compiler what you'd like and, voila, faster code. Since compilers differ, however, and since you're never quite sure what incantation is going to convince Lisp to do "the right thing" (*), you're often left flipping declarations around like hot potatoes. Enter Allegro Common Lisp, version 8.0 I've been working lately (for money even!) in ACL and one of the wonderful new features is that you can ask the compiler to tell you what it thinks and what it wants. I've wanted this ability for a long time (I even talked about it at ILC 200? in NYC). Let the programming environment notice that such and such a function is suboptimal and tell me that if I scratch it's back (with a little more information), then it can scratch mine (with a little more performance). ACL's explain feature isn't quite that, but it's a great step in that direction. (*) I.e., what you want it to. Joke.
Cellular signposts I suffer from psoriasis (and it's not just a heartbreak!). One of the many interesting questions about this disease is why this little bit of skin is undergoing massive immune mediated wonkiness (MIMW) and this bit of skin next to it thinks that everything is normal. This article (which I found via the American Scientist) gives part of an explanation. Cool.
Perception, reality, etc. Java, Python, Ruby guru Mark Watson points out that he feels more productive using LaTeX rather than Word and this reminds me of the whole "is incremental spell checking really a good thing" debate (that I've had with myself annually since 2003; don't be too surprised if you haven't heard about it; my ex-agent failed to get the big network contracts he promised. Which reminds me, isn't stream of consciousness interesting. I suppose it depends on the consciousness... <smile>). At question is what work style and tools help humans find maximum productivity and creativity. Is is better for a writer to work with pencil and paper -- forced to produce marks on paper at a rate slower than thought -- or with a word processor that highlights spelling, grammar, and logical inconsistencies (I made that last one up) on the fly? Is it better for a programmer to submit batch jobs to a mainframe or have background processes constantly checking the source for problems? The truth is probably a muddy thing. My opinion, however, is that the best (in terms of productivity, creativity, and flow) is highly context dependent and is often at odds with what feels psychologically most productive. We need time to think; we need (metaphorical) quiet to get into the flow. A tool that shouts "spelling error" with every typo is providing more distraction that help. Yet it feels good to make those red squiggles disappear -- "We're making progress!" -- it feels good to play with fonts and shift the format. A computer is a far different beast that Heidegger's hammer: it can be at hand in many ways simultaneously. It's up to us to ensure that the tools we use are optimizing the important tasks, not the trivial ones.
Summer reading: Spin I'm sure many of my readers are saying to themselves (quietly so as not to attract attention to themselves) "It's summer, why isn't Gary reading?". Well say no more, squire, say no more. I have been reading, I've just been keeping quiet about it. Spin is a wonderful book by the novelist Robert Charles Wilson. Normally, I worry about people with multiple first names -- seems to me that it must leave them confused on some deep, deep level. Of course, I used to think that the group Tony Orlando and Dawn was composed of three people named "Tony," "Orlando," and "Dawn"... it made sense to me. Enough about my neuroses, however, let's talk about the book. Spin is science fiction in the best sense -- interesting science (yes) and interesting fiction (of course) but the heart of the book is the characters; not the plot nor the gizmos. The "spin" is an event caused by forces unknown for reasons unknown that leaves the earth wrapped in a shell. The stars and moon are no longer visible and satellite communication is impossible. The sun, however, still appears to rise and give warmth. Life goes on. Eventually, it's discovered that time on earth is moving more slowly than that of the universe outside the shell; much more slowly. Weeks on earth correspond to millennia. This means that the sun will consume its supply of hydrogen fuel, enter senescence and grow to consume the earth in less than a century. But why wrap the earth at all? Who is doing this? Is it the end of the world? Is it the rapture? The book tracks the responses of three friends and the rest of the world to these questions. Wilson writes elegantly and eloquently; beautiful and at times heart rending phrases enliven the interplay of plot and character. In the end, science matters less than people and the hardest gap to cross is not the one between the stars but the one between self and other and, sometimes even less permeable, that between self and self. Highly recommended for enjoyable science, wonderful writing and beautiful story.
Functional programming fun from Joel Joel Spolsky writes well. He writes like a fish swims. Though not as glib as Paul Graham he still leaves an earnest yet unctuous film around the eyeballs and a bit of a grin around the mouth. In any case, he provides a great lead in for why MapReduce is way cool.
CL-Markdown extensions - first steps I've made a little time to push on the CL-Markdown extension mechanism I wrote about a week or two ago. I've added a few new glitches but the change was less painful that I thought it might be (CL-Markdown swings far towards the organic and messy side of the code I write!) In any case, the text below (written in Markdown format and parsed by CL-Markdown) describes how to write and use CL-Markdown extensions. The output isn't perfect (there are some glitches with quotations marks in code blocks for one thing), but it's pretty darn good if you ask me... (I know, biased, biased, biased... just like that SCLM). Let me know what you think!
Update: ASDF-Binary-Locations ASDF-Binary-Locations now has another way to customize output locations thanks to Erik Enge. Erik's patch recognizes that sometimes you want to specify output location based on who is currently using the machine. Thus we have the new variable: *include-per-user-information*.
Enjoy.
Links to things I thought were cool After all, isn't that what the web is all about?
Some CL-Markdown design questions for public review I've been using Peter Seibel's Markup (see also here for a fork by Cyrus Harmon) lately for a project. If you've seen LaTeX then Markup will be familiar. The nicest thing about it is that Peter has made it really easy to extend the language with your own commands. It's also very well integrated with Lisp and with Peter's other tools. On the other hand, it's not Markdown. This means that Markup documents tend to look like documents with Markup and that there is no convenient round trip from Markup to HTML and back again. Now I'd like to be able to say that CL-Markdown provided Markdown's extremely readable syntax with Markup's cool customization but I can't. At least, not yet. Here then, are my thoughts on the matter. Please let me know if anything resonates positively or, perhaps even more importantly, negatively with you. First, what do I want to support?
Second, what do I propose:
I'm not sure when I'll get to this... Soon, I hope but let me know (gwking@metabang.com) when you see something I've missing. Thanks. William C. Dietz's Runner is decent science fiction: decent plot, decent characters, decent technology, decent writing. Everything about the book was, well, decent. If it sounds as if I'm damning with faint praise it's because I am. This was a book I finished only because it's the sort of book I can read very fast. There was nothing particularly bad about it but also nothing particularly good. The characters were mostly believable but there was nothing organic about the. The writing was passable but surely not memorable (no phrase savoring here, move along). The plot was a standard "make a journey and pick up allies and enemies and learn things about yourself on the way" and the science was neither deeply explored nor central to the plot. Maybe you'll love it, but there are better books out there to read.
Update: trivial-shell Trivial Shell now works with CMUCL. Thanks to Satyaki for the patch.
Logo Wiki / Code Wiki (Via Paolo Amoroso) The Logo Wiki is a great example of a code wiki. It needs some better version control of the code (in case of mistakes and typos) and I'm not sure how it deals with spam, etc. Still, it's fun and it makes it really easy to get a feel for Logo itself -- it helps that one of Logo's was (is?) teaching programming and that the interface is primarily a turtle carrying a pen! Paolo Amoroso mentions that a Lisp variant would be even cooler and he's right in a way. I'm not sure that the examples would be as fun though.
the Omnivoire's Delimma
I'll try to avoid the obvious food puns but Michael Pollan's the Omnivore's Dilemma is a rich dessert of elegant verbal treats, sweetened with thoughts both philosophical and political.
Pollan's four meals (the industrial, the agrarian, the organic and the gathered) clearly delineate some of our possible relationships with food; what's more, it's clear that the most common relationships are the worst for us and the health of our shared biosphere.
This lack of knowledge -- and indeed the (apparent) strong desire of the powers that be to deny knowledge, crush and impede the Freedom of Information Act, pretend that processed food is just like its real counterparts, distract us with (wonder) bread and circuses, and with fear -- is what worries me. I believe that people will do the right thing (most of the time, by and large) if they know what that thing is and if it's not too hard to do it and if doing it is visible and engenders positive feedback loops. Mass industrial capitalism does not support a human economy of scale. Regardless of that, however, this is a great book and I enjoyed every minute of my reading!
Update: ASDF-Binary-Locations Thanks to Joshua Moody, ASDF-Binary-Locations now knows something about 64-bit Allegro Common Lisp.
Update: CL-Markdown I've made several small tweaks to CL-Markdown in an effort to bring it a bit closer to its Perl cousin. In particular:
As usual, CL-Markdown has a host of dependencies. You may want to try System-check to see what needs to be updated.
Mark Watson on Global Warming Lisp and Java consultant Mark Watson on global warming:
He's right. But America is addicted to oil, air conditioning and cheap corn so we know it won't happen.
Another step towards ASDF system understanding As usual, progress is slower than desired or expected and my reach seems to ever exceed my grasp (I think it has something to do with the anatomy of my hand...). Nonetheless, there are now some spiffy system dependency graphs up at enterpriselisp.com. Not surprisingly, CL-PPCRE's is one of the most complex. In case it's not clear (and it probably isn't!), you can navigate between systems by clicking on the graph vertexes. Also, the graphics are all SVG so you'll need a browser that knows how to speak it. More web phun is in the pipeline.
ASDF turns 100 I just had the honor of checking in the 100th revision of ASDF. Cool! The newest feature is the addition of load-preferences and preference-file-for-system/operation. These generic functions can be specialized on the system and the operation. The out of the box behavior of load-preferences is to do nothing except in the case of a load-op or a load-source-op. For either of those operations, load-preferences calls preference-file-for-system/operation to get a pathname. If the pathname returned exists, load-preferences loads it. By default, preference-file-for-system/operation returns ~/.asdf/<name-of-system>.lisp. It's been said that an example is worth a 5.19 pictures so here are my preferences for asdf-binary-locations (in /users/gwking/.asdf/asdf-binary-locations.lisp): (in-package asdf)
(setf *default-toplevel-directory*
"/users/gwking/temporary/fasls/")
(setf *centralize-lisp-binaries* t)
;; force SBCL things to stay in SBCL
(setf *source-to-target-mappings*
'(("/usr/local/lib/sbcl/" nil)
("/usr/local/lib/sbcl0.9.9" nil)
("/usr/local/lib/sbcl0.9.7" nil)))
These put all of my compiled files into sub-directories of ~/temporary/fasls (except for SBCL stuff which stay where they are expected). The nice thing about this is that the preferences are loaded after the system whose preferences are being set is loaded. This is nice because it's hard to set preferences for a system that doesn't exist yet (because, for example, the home package of the variables you'd like to set isn't there). (Thanks to the CCLAN and especially Christophe Rhodes who provided valuable feedback! Any remaining errors are probably the fault of either the Bush or the Clinton administrations. Or both of them together).
Those wacky biologists...
Actually, I think that they had a great idea! Calling it "counting", however, reads more into the data than is available. Many animals are known to count accurately and to keep track of things like rate, etc. (Charles Gallistel's book is a wonderful read if you're interested in learning much, much, more!) but just because a human would solve a problem using solution X doesn't mean that an animal uses X too (even if we can't think of any other method to use than X...).
Updated: trivial shell I fixed a typo in trivial-shell (sb!ext instead of sb-ext... go figure).
July Fourth in America With apologies in advance for the politics... Howard Zinn says it for me:
There's probably a bad joke here about Lisp being marginalized and America becoming increasingly marginalized -- or something -- but I don't want to think that hard.
Transparency. Openness. Aim for the heights. Think the best of people. Do good and be nice. keep peace.
Updated: CL-Graph Thanks to Attila Lendvai for some new patches to CL-Graph. He improved the GraphViz support and added some &allow-other-keys to avoid some warnings. I also just added the #'delete-all-edges generic function.
Two papers on Racer The semantic web is one of those great ideas that hasn't quite jelled. Eventually, the infrastructure, specifications and plans that are welling up out of the web's interstices are going to come together into something beautiful and strange. One of the droplets slouching towards us is Racer: a reasoner that takes all the little factoids expressed in RDF, DAML+OIL, OWL and so forth and derives consequences. These two papers (Racer: A Core Inference Engine for the Semantic Web and Racer: An OWL Reasoning Agent for the Semantic Web) share much of the same text so I'm describing them together. They provide a very high level overview of the various languages used to express semantic web knowledge (the acronym salad I listed above) and how Racer can use them to explore consistency, sub-classing (and other relationships), role filling and so forth. The papers describe several actual systems (e.g., RICE: Racer Interactive Client Environment, OilEd, and DIG). Neither paper is particularly useful in terms of giving compelling examples of how the semantic web is going to make our lives better (they give features not solutions) but they're a start.
#+Ignore is fine, #+(or) is bliss Although Common Lisp has a multiline-comment facility (that even handles nesting), many Lispers tend to use the #+ / #- reader macros to temporarily remove chunks of code from the view of the evaluator (note that these do not remove the code from the view of the reader, an issue which often leads to confusion the first time you can't load code that you think is 'commented' out — which is why it's important to remember that #+/#+ is not a commenting facility even though you may be using it like one...). The reason for this in my case is simple: laziness. I'd rather do something in one place than in two and it's easier to place (and remove) #+ignore in front of a form rather than wrapping the form in #| and |#. Frankly, I think that this sort of laziness is part of the human condition. If we really want most people to do the right thing most of the time, then we need to make it easy and convenient to do the right thing (e.g., recycle, carpool, keeping documentation up to date, and so on). Using the reader macro to hide code isn't a bad practice as practices go (it's certainly healthier than coding in C or C++ <smile>) but lurking in the back of every coder's mind is that horrible question: "What if," they say to themselves in the still hours of the night, "What if someone pushes :ignore onto the *features* list?" (or :no, :later, :old, :new, :nil, :never, :not-yet, ...)? Well, I was looking at some of Peter Seibel's code recently and found a very nice way to have my cake even after eating it. There is form that is guaranteed to return nil regardless of what is on *features*? That form is #+(or) and it's a very nice thing indeed.
Microsoft is sharing code I think that it will be interesting to see how this functions. Microsoft certainly has the resources to make it a useful code garden but I'm not sure if the political will is really there (shared source isn't open source). Coincidentally enough, one of the features I'd like to support at Enterprise Lisp is the ability to share Lisp code easily and with structure. The CLiki and Lisp Paste supply a good base but I think that a bit more infrastructure could make a bit difference in usability and utility.
Updates; E8EL is live Thanks to the Tech Co-op, Enterprise Lisp is live and on-line in what I hope will be it's permanent home. System-check has been updated to version 0.1.6 to accommodate the new home.
Pretty wonderful (if you ask me...) Little brown dress:
I heard the artist interviewed last night on WAMC. I think it's great!
Ain't soccer grand Dynamic, fluid, a constant reinterpretation of tactics and strategy... like Life, like Lisp.
Sparklines + AJAX = nice Sparklines are little graphs that run in-line with your text. I believe that Edward Tufte invented them and I know that he talks about them in his upcoming book, Beautiful Evidence. Several folks have written code to create them (I think that there is even a Lisp implementation out there somewhere for CL-PDF?). Joe Gregorio has a nice Python AJAX web application if you want to play with them yourself.
How come Skype and Apple's Address Book don't play nice? Skype stores it's own contact data. How 1990s.
Common Lisp and RSS I've just started to start to work on an RSS feed for the Enterprise Lisp System Checker. The thing is, I can't find any Lisp libraries that make generating RSS feeds as easy as I think it should be (Kevin Rosenberg's CL-RSS is still available but seems unmaintained and only generates and parses RSS 0.92 feeds). So, in typical lisp hacker fashion, I wondered if I should first write a simple RSS library; something that can parse and generate RSS 0.92, RSS 1.0, RSS 2.0 and Atom feeds. If I did, there are two obvious designs: a object based one and a language based one. In the former, there would be classes for feeds and items and the usual things would happen to create, parse and generate feeds. In the later, RSS feeds would like like an XML template language (e.g., LML2, YACLML, and so forth). The language approach would be conceptually simple but provides no useful abstractions (and no help in parsing). I think that templates make sense for a language like HTML because HTML content can be just about anything. RSS, on the other hand, always looks like a channel description and a list of items. The template just doesn't help that much. So on to objects and classes and such...
System Checker The Enterprise Lisp system's checker has reached a stable point and I'm also happy with system-check. Unless I hear otherwise, the next steps will be (in no particular order):
Once these features are stable (and once I've moved operations to a stable system), then I'll get back to ASDF-Install-tester / ASDF-Status and begin integrating them into Enterprise Lisp.
more various updates ASDF-Binary-locations now knows about 64-bit OpenMCL (thanks to Joshua Moody for the heads up and the patch). I've been having some troubles keeping the system currently hosting Enterprise Lisp up and running (those darn kids <smile>). I'm also ready to move over onto a virtual Xen thing run by the nice folks at GrokThis. I've also improved the systems report slightly so that it is clearer what the systems checker found amiss. There are now five kinds of errors:
Note that signature problems can be restarted around (especially if you have the latest ASDF-Install) so it is possible to have a valid system and a valid signature even if the system status is not OK.
various updates
(merge-pathnames
(make-pathname :name "asdf-binary-locations"
:type "lisp"
:directory '(:relative ".asdf"))
(truename (user-homedir-pathname)))
More why? (not Y) I've received some follow-up on my "Why isn't Lisp da bomb for web development" post from last week. Since the CLiki isn't the place to talk about this kind of thing, I've made a little wiki at Infogami for it: lispandwebdev. Without going into details, the nub of the problem may be that Lispers are busy writing great stuff for the few whereas the Ruby on Rails folks et. al. are catering to the plebeian masses. I don't want to get sucked into language wars or anything like "my language is better than your language" or even the wistful "if only they could see it, then they would understand that Lisp is the way"... I also don't want to disparage the existing Lisp frameworks like BNKR, Lisp on Lines, UCW, the others I've forgotten and so forth. At issue, I think, is the sort of vitriol that leads to disasters like this (also noted by Stefan Scholl). This is happening very publically on Reddit and many people will have their prejudices created or confirmed that Lispers are arrogant smug weenies. I find this almost unbearably sad.
ASDF-Install 0.5.5 Version 0.5.5 of ASDF-Install is ASDF-Installable and also available in the usual Darcs and tarball formats. This version starts to rationalize some of the where to look for system files registry management (i.e., there is now a function asdf-install:add-registry-location). It also adds a new restart so that packages can be installed even if there is no GPG signature file available (e.g., trivial-gray-streams). See the change log for all the gory details)
two quick notes I bumped up ASDF-Install to 0.5.4. The changes are very minor and serve only to make processing putative future keyword arguments a bit more consistent. I also found a bug in system-check. At one point I had remembered to wrap my HTTP POST request in a with-standard-io-syntax but then I ran into some confusion with SBCL and I think pretty printing... To be honest, I never figured it out (slinks away with shame faced admission of guilt...). In any case, without the with-standard-io-syntax, it was quite possible for your personal print settings to muck things up badly (e.g., I have *print-length* set to 10 and the server didn't know what to do with an ellipsis). So, if you have already downloaded system-check, you might want to do so again. If you haven't, why, then, you should -- how is that supposed to be punctuated anyway? It makes sense in a conversation but I'm not sure how to write it... Anyway, please do check out system-check and let me know how to make it suit your needs. Joel Reymont is leaning toward Ruby on Rails over Lisp for web application development and I don't get it. Joel is very smart and on top of things and I figure that he's right: Lisp just isn't there yet when it comes to building web applications quickly and easily unless you want to make it half your own private research project. But Why? Lisp is good. Lisp excels at Domain Specific Languages. Lisp has the REPL for goodness sake. It can't just be that there are many Lisp dialects / implementations and that it's a bit hard to get, say, a socket library that works on all of them... can it? I mean, the MediaWiki runs on PHP (I think) and everything I've seen or heard about PHP is "...ick...". So why does the MediaWiki have all the loving goodness and we have the CLiki? (This isn't to diss the CLiki by the way, it's cool; I like it but as wikis go there isn't much there and nothing much has happened to it for ages.) So bloggers on Planet Lisp, start your engines please... I'd like several essays from people who know the skinny on web dev and Lisp. Is Joel wrong? Is there a missing secret ingredient? what does Lisp need to come from behind in the web application development language race? After all, Ruby was a dark horse until Rails came along. The best essay gets a prize (to be announced later once I think of something).
That was fast I bought a new hard drive for my laptop yesterday afternoon (the 8th) from PBParts.com. They promised that the drive would ship sometime before the 13th which seemed fine to me. Guess what arrived today via FedEx. That's right, I've got the little thing already. It's always good to lower expectations but I'm very impressed. It looks like a hardware kind of night tonight.
ASDF Checker report Here's the first ASDF Checker Summary report from Enterprise Lisp. The checker process runs daily and checks all of the known ASDF installable systems to see if they seem to work (i.e., that they download, have GPG signatures, that their system files seem to make sense and so on). The data from checker will be grist for tester once I finish refactoring ASDF-Install-tester to create it. There is much room for improvement in the report (and feedback is welcome!). The last three columns show the system's status, whether we have a valid system file and whether or not we were able to create a system signature. The current system column isn't too useful; I need to spend a bit of time looking at the data and creating informative codes... A valid system is one whose system definition the checker could load -- it currently conveys no more information than whether or not the system has a status of error. Finally, the system signature is a list of the direct files of the systems and their universal times. Here are two examples: ? (dsc:system-signature 'moptilities)
(("moptilities.asd" . 3345750061)
("dev:moptilities.lisp" . 3356703878))
? (dsc:system-signature 'anaphora)
(("anaphora.asd" . 3287421247)
("packages.lisp" . 3287421247)
("early.lisp" . 3287421247)
("symbolic.lisp" . 3287421247)
("anaphora.lisp" . 3287421247))
?
These signatures are part of defsystem-compatibility. Checker needs to take more pains than it yet does to ensure that features and such don't influence signature computation. That's another one of the seemingly endless list of next steps!
On average I'm in the middle of a hot bed of Lisp activity Unfortunately, the distribution is highly bimodal If wishes were fishes... we'd all cast nets.
announce: system-check (beta) Enterprise Lisp has been rumbling and grumbling slowly to life around me. Today I'm announcing the first beta of System-check, an asdf-upgrade like facility, that will eventually integrate with an improved ASDF-Install-tester and other cool things. The idea behind system-check is that it's easier to centralize the testing of Lisp libraries that it is to get all of the library developers to agree on versioning schemes and the like. Therefore Enterprise Lisp will regularly check on all ASDF installable systems and
This will be coupled with an improved and distributed ASDF-Install-tester that can go on to make sure that systems actually do install on a variety of Lisps and environments. System-Check is the client that you can run to see if your systems are up to date. Here is an example session (in OpenMCL). First, we load system-check. It displays instructions when it finishes loading: ? (asdf:oos 'asdf:load-op 'system-check) ; loading system definition from user-home:darcs;asdf-systems;system-check.asd.newest into #<Package "ASDF0"> ---------------------------------------------------------------------- ;; System-check helps keep your ASDF-Installable systems up to date ;; by communicating with enterpriselisp.com via HTTP. Enterprise Lisp ;; checks ASDF-Installable systems regularly to make sure that they ;; work properly and install correctly. ;; ;; System-check has three main entry ;; ;; 1. update - checks all of your ASDF-Installed systems against the ;; most recent available version and lets you select which ones to ;; ;; 2. gather - performs the same check as update but returns the ;; results as a ;; ;; 3. check-system - performs a check on a single ;; ;; More information can be found in the documentation or at ;; ;; System-check can look for systems in several different ;; ;; * installed-systems (used by default) - returns a list of systems ;; that seem to have been ASDF installed (see its documentation for ;; ;; * installable-systems - returns a list of all of the available ;; systems (using ;; ;; Both of these functions take the keyword argument ;; :only-asdf-installable?. If this is true, then System-check will ;; query the CLiki to see if the system is ASDF installable. ;; We cut to the chase and call the update function. ? (system-check:update) Searching for systems. ; loading system definition from /Users/gwking/.asdf-install-dir/systems/cl-difflib.asd into #<Package "ASDF1"> ; registering #<SYSTEM :CL-DIFFLIB #x84E1786> as CL-DIFFLIB ; loading system definition from /Users/gwking/.asdf-install-dir/systems/uffi.asd into #<Package "ASDF1"> ; registering #<SYSTEM UFFI #x8514DE6> as UFFI Checking 59 systems........................................................... Results ============================================================ 4 we-are-latest (local system has more recent changes than the remote) ------------------------------------------------------------ xmls mk-defsystem cl-prevalence cl-html-diff 6 both-changed (system has modifications locally and remotely) ------------------------------------------------------------ system-check metatilities lml2 defsystem-compatibility cl-graph cl-containers 8 error (the server was unable to check the system) ------------------------------------------------------------ split-sequence net-telent-date ironclad clx cl-store asdf-install asdf-binary-locations arnesi 11 need-update (remote system has changes) ------------------------------------------------------------ trivial-http tinaa moptilities metacopy metabang-bind lift cl-variates cl-mathstats cl-html-parse asdf-system-connections araneida 30 ok (local system is up to date) ------------------------------------------------------------ xlunit wilbur-ext uffi trivial-configuration-parser s-xml-rpc s-xml s-utils s-sysdeps s-http-server s-http-client s-base64 rt puri md5 lw-compat kmrcl html-encode diff contextl clsql closer-mop cl-utilities cl-fad cl-dot cl-difflib cl-base64 cl-ajax cffi asdf-upgrade anaphora Now that we've seen the report, we're asked what we want to update and ASDF-Install takes away the show. We'll now ask which kinds of systems you want to update. You will be able to confirm before the update process begins. 11 systems need to be updated. Do you want to update them? (y or n) y Marking these systems for update: araneida, asdf-system-connections, cl-html-parse, cl-mathstats, cl-variates, lift, metabang-bind, metacopy, moptilities, tinaa, trivial-http. 6 systems are changed locally and remotely. Do you want to update them? (y or n) n Updating 11 systems Install where? 0) System-wide install: System in /Users/gwking/.asdf-install-dir/systems/site-systems/ Files in /Users/gwking/.asdf-install-dir/systems/site/ 1) Personal installation: System in /Users/gwking/.asdf-install-dir/systems/ Files in /Users/gwking/.asdf-install-dir/site/ 2) Abort installation. --> 1 ;;; ASDF-INSTALL: Downloading 531636 bytes from http://common-lisp.net/project/araneida/release/araneida-latest.tar.gz to araneida.asdf-install-tmp ... "gpg: Signature made Fri Dec 2 12:55:13 2005 EST using DSA key ID 5E55AFEB" "[GNUPG:] ERRSIG 19876FCE5E55AFEB 17 2 00 1133546113 9" "[GNUPG:] NO_PUBKEY 19876FCE5E55AFEB" "gpg: Can't check signature: public key not found" > Error in process listener(1): No key found for key id 0x#1=(19876FCE5E55AFEB 17 2 00 1133546113 9). Try some command like > gpg --recv-keys 0x#1# > While executing: #<Anonymous Function #x8426CE6> > Type :POP to abort. Type :? for other options. 1 > :r 0. Return to break level 1. 1. #<RESTART ABORT-BREAK #x2947FE> 2. Don't check GPG signature for this package 3. Retry GPG check (e.g., after downloading the key) 4. Return to toplevel. 5. #<RESTART ABORT-BREAK #x294CBE> 6. Reset this process 7. Kill this process 1 > (:c 2) Invoking restart: Don't check GPG signature for this package ;;; ASDF-INSTALL: Installing araneida.asdf-install-tmp in /Users/gwking/.asdf-install-dir/site/, /Users/gwking/.asdf-install-dir/systems/ araneida-version-0.90.1/ ;;; and so on... We can also check the status of a single system. I'll turn on the verbose mode so that you can see what gets passed back and forth. ? (system-check:check-system 'anaphora :verbose? t)
Signature: (:SYSTEM :ANAPHORA :SIGNATURE (("anaphora.asd" . 3287421247)
("packages.lisp" . 3287421247) ("early.lisp" . 3287421247) ("symbolic.lisp" . 3287421247)
("anaphora.lisp" . 3287421247))
:FEATURES (:KMR-MOP :CLX-ANSI-COMMON-LISP :ARANEIDA-THREADS :CL-FAD :CLOSER-MOP :ASDF-INSTALL :ASDF :GWKING :PRIMARY-CLASSES :CCL :CCL-2
:CCL-3 :CCL-4 :CORAL :COMMON-LISP :MCL :OPENMCL :ANSI-CL :PROCESSES :UNIX :OPENMCL-NATIVE-THREADS :OPENMCL-PARTIAL-MOP
:MCL-COMMON-MOP-SUBSET :OPENMCL-MOP-2 :POWERPC :PPC-TARGET :PPC-CLOS :PPC32-TARGET
:PPC32-HOST :DARWINPPC-TARGET :DARWINPPC-HOST :DARWIN
:POWEROPEN-TARGET :32-BIT-TARGET :32-BIT-HOST :BIG-ENDIAN-TARGET
:BIG-ENDIAN-HOST :OPENMCL-PRIVATE-HASH-TABLES)
:IMPLEMENTATION "openmcl-1.0-darwin-powerpc"
:VERSION "0.1"
:PATHNAME-SEPARATOR "/")
Response: 200
Headers: ((:DATE . "Wed, 07 Jun 2006 16:24:23 GMT") (:SERVER . "Apache/1.3.33 (Darwin) mod_lisp/2.43")
(:SIGNATURE-RESULT . "NIL") (:POSTED-CONTENT . "(:SYSTEM :ANAPHORA :SIGNATURE ((\"anaphora.asd\" . 3287421247) (\"packages.lisp\" . 3287421247) (\"early.lisp\" . 3287421247) (\"symbolic.lisp\" . 3287421247) (\"anaphora.lisp\" . 3287421247)) :FEATURES (:KMR-MOP :CLX-ANSI-COMMON-LISP :ARANEIDA-THREADS :CL-FAD :CLOSER-MOP :ASDF-INSTALL :ASDF :GWKING :PRIMARY-CLASSES :CCL :CCL-2 :CCL-3 :CCL-4 :CORAL :COMMON-LISP :MCL :OPENMCL :ANSI-CL :PROCESSES :UNIX :OPENMCL-NATIVE-THREADS :OPENMCL-PARTIAL-MOP :MCL-COMMON-MOP-SUBSET :OPENMCL-MOP-2 :POWERPC :PPC-TARGET :PPC-CLOS :PPC32-TARGET :PPC32-HOST :DARWINPPC-TARGET :DARWINPPC-HOST :DARWIN :POWEROPEN-TARGET :32-BIT-TARGET :32-BIT-HOST :BIG-ENDIAN-TARGET :BIG-ENDIAN-HOST :OPENMCL-PRIVATE-HASH-TABLES) :IMPLEMENTATION \"openmcl-1.0-darwin-powerpc\" :VERSION \"0.1\" :PATHNAME-SEPARATOR \"/\")")
(:CONTENT-LENGTH . "772") (:REMOTE-IP-ADDR . "155.212.227.170") (:REMOTE-IP-PORT . "30528") (:SCRIPT-FILENAME . "/Library/WebServer/Documents/compare.lsp") (:SERVER-IP-ADDR . "10.0.1.2") (:SERVER-IP-PORT . "80") (:SERVER-PROTOCOL . "HTTP/1.0") (:METHOD . "POST")
(:URL . "http://metabang.gotdns.com/compare.lsp") (:SERVER-ID . "metabang") (:SERVER-BASEVERSION . "Apache/1.3.28") (:MODLISP-VERSION . "2.43") (:HOST . "metabang.gotdns.com") (:USER-AGENT . "simple HTTP for Common Lisp")
(:CONNECTION . "close") (:CONTENT-TYPE . "x-www-form-urlencoded"))
:OK
?
System-check and the Enterprise Lisp system checker are both in beta at the moment. I've tested on several lisps but expect that problems and edge cases remain. Please let me know if you try system-check and it fails. I'm also working on some improvements to ASDF-Install so that it can behave more reasonably when installing multiple systems and with improving the back and forth between the client and the server. Enterprise Lisp is all about making Lisp easier for everyone, so please let me know if anything seems awry.
announce: simple-http The world didn't need another HTTP library. Unfortunately, I did. Thus is born Simple-HTTP. It builds on the base of Trivial-HTTP and adds things like HTTP Head, HTTP download and HTTP-resolve. Eventually, it will be pulling in some useful bits and pieces from Lemonodor and Lisp Paste (e.g., here and here). Those that know should tell me what else would be worth adding. Enjoy.
announce: improved CL-Markdown These changes aren't nearly as cool as ABL's but CL-Markdown has taken another few small steps forward toward Markdown compliance. The latest code tweaks the block structure processing and improves paragraph recognition logic. This means that its output looks more like that produced with the real markdown.
announce: improved asdf-binary-locations Thanks to a patch from Peter Seibel and several good ideas and hints from Robert Goldman, ASDF-Binary-Locations has some changes and improvements. The most significant change is that the variable *system-configuration-paths* has been renamed to source-to-target-mappings* because the latter name is, IMHO, much, much better than the former. There are also two new variables to control behavior and the innards have been rewritten with generic functions so that you can have fine control over where exactly things go at both the system, the operation and the component level. The three control variables are:
Note that if you've already set *system-configuration-paths* (e.g., in you lisp startup file), then ASDF-Binary-Locations will warn you about the change and automagically set *source-to-target-mappings* to whatever value you gave *system-configuration-paths*. Also note that the finer control mentioned about has not been extensively tested (i.e., it really hasn't been tested at all) yet but will be once I finish the test suite. Won't that be sweet! Please let me know if your milage varies.
more identity theft on the way...
is it standard practice to provide auditors with names and credit care numbers? Why would they need that information? Why aren't they given anonymized data? If "The security and confidentiality of our client information is of critical importance to Ernst & Young", then why don't they do more to take it seriously?
the Golem's Eye Book two of the Bartimaeus trilogy finds the magician Nicholas and the Djinn Bartimaeus once again embroiled in the scheming power politics of an Europe where magic rules. Hidden forces further devious plots: the resistance returns and is betrayed from within; a Golem is destroying the city; betrayal and duplicity abound. In short, it's a lot of fun and worth the enjoyable read. Yes, it is a book for young adults (hey, I'm only 42 (and that, as I suddenly recall, is the answer to everything)) but you'll probably enjoy it too.
the Amulet of Samarkand One of the pleasures of having kids is that you can read kids book without feeling that you're too old to be reading them. The Amulet of Samarkand is very much a kids book but its a good one and it comes with interstitial themes worthy of adults. The book tells the tale of an earth where magic works thanks to the ability of some humans to summon spirits of all kinds. It is part one of the three part Bartimaeus trilogy (the eponymous name comes from that of the main narrator, a djinn of remarkable ability (including much sarcasm and wit). Amulet follows a complex plot with a young hero/anti-hero, many magicians of dubious intent, political scheming and many levels, a anti-magical resistance and enough twists to confuse an adder. It is all painted with a broad brush full of vim, verve and vitality. Buried under all this enjoyable fluff, however, are the messages that power is often neither to be trusted, nor desired; that the beneficence of a government is often inversely proportional to how loudly it congratulates itself; and that (to quote Bruce Cockburn) "when you get down to the bottom, loves the only thing that matters."
It's probably because I'm tired and it's unlikely you really want to know. Nonetheless, here is my response to the iDon't marketing campaign of sandisk
ok, no more venting until at least tomorrow... (hmm, that gives me 7-minutes).
CLSQL and me: I feel so Microsoft Access ugly I did a bunch of database stuff back when SQL 92 was exciting. I used early PC database systems like dBase IV, Foxpro, Borland's Paradox, and Microsoft's Access. Since auto-increment columns hadn't reached down to those trenches, I ended up doing the old "keep track of the maximum key in a separate key table yourself" trick. Not fun, but effective -- well, it works. Today, I was messing with CLSQL (connecting to SQLite) and felt stymied trying to correctly get my primary keys to work. In the hopes that a wiser soul will feel my pain, here is what I did. (def-view-class primary-key-mixin ()
((id :db-kind :base :type integer
:db-constraints (:primary-key)
:reader id :initarg :id)
(table-name :db-kind :virtual
:reader table-name
:initarg :table-name)))
(defmethod initialize-instance :after ((instance primary-key-mixin) &key)
(unless (and (slot-boundp instance 'id) (id instance))
(setf (slot-value instance 'id) (find-next-id (table-name instance)))))
(def-view-class sample-table (primary-key-mixin)
((name :db-kind :base :type (varchar 40)
:db-constraints (:unique :not-null)
:accessor name :initarg :name))
(:default-initargs
:table-name "managed-system"))
(def-view-class primary-key ()
((table-name :db-kind :base
:db-constraints :primary-key
:type (string 20)
:accessor table-name
:initarg :table-name)
(max-key :db-kind :base :type integer
:accessor max-key
:initarg :max-key
:initform 0)))
(defun recreate-tables (&key really?)
(unless really?
(cerror "Yes, really!" "Do you really want to trash the tables and start fresh?"))
(clsql:drop-table [primary-key] :if-does-not-exist :ignore)
(create-view-from-class 'primary-key)
(clsql:drop-table [sample-table] :if-does-not-exist :ignore)
(create-view-from-class 'sample-table)
)
(defun find-next-id (table-name)
(with-transaction nil
(bind ((exists?
(select [max-key]
:from [primary-key]
:where [= [table-name] table-name]
:flatp t))
(next-key (if exists? (1+ (first exists?)) 0)))
(if exists?
(update-records [primary-key]
:av-pairs `(([max-key] ,next-key))
:where [= [table-name] table-name])
(insert-records
:into [primary-key]
:av-pairs `(([table-name] ,table-name) ([max-key] ,next-key))))
(values next-key))))
This defines two view-classes (and the recreate-tables function makes tables out them). The primary-key table keeps track of the highest key assigned so far; the primary-key-mixin uses it to assign keys as necessary. Since instances can be created and not added to the database, it's quite likely that we'll have gaps but that's not a big deal. This let's me execute code like: ? (setf *s* (make-instance 'sample-table :name "Gary"))
#<MANAGED-SYSTEM #x88C5F8E>
? (update-records-from-instance *s*)
; no value
? (setf *s* (make-instance 'sample-table :name "Wendy"))
#<MANAGED-SYSTEM #x88C5F9A>
? (update-records-from-instance *s*)
; no value
? (select [*] :from [managed-system])
(("cl-containers" 0) ("moptilities" 1))
("NAME" "ID")
Which, while not really exciting, is at least moderately painless. Aside from the fact that doing all of this key management myself strikes me as unbearably last decade (not to mention error prone and probably non-union), I figure that there must be a better way. Any suggestions?
asdf:test-op redux I wrote about asdf:test-op a bit ago and have since modified many of my opinions as to the way to do it. Yesterday, I noticed that Greg Pfeil had come to exactly my new conclusions back in late March (I read planet lisp quasi-religiously but missed this somehow). The chief changes are to move most everything to the system definition (not the test system definition) so that will test my-system (makes sense!). The only bit Greg leaves out is (defmethod operation-done-p
((o test-op)
(c (eql (find-system 'moptilities))))
(values nil))
Which keeps ASDF thinking that your test operation hasn't been done. (As we all know, testing is never done). Thanks Greg.
Comments? I've been meaning (and hoping) to write my own simple comment engine for the last upteen months. Today, I broke down and signed up with Haloscan. I'm not sure how customizable it can be or how I'll like it but I'm willing to see what happens.
Daring fireball I like John Gruber.
I love simplicity. I don't want to fight with my machine or with my inner mustard chooser. Too often, too much is not more than enough, it's too damn much.
Bellwether When I mentioned fads last week, Bill Clementson was kind enough to recommend me two books: bellwether by Connie Willis and Pattern Recognition by William Gibson. I've read quite a bit of Gibson's work over the years -- dark, but interesting -- but had never even heard of Willis. After reading bellwether, I'm very happy to have finally been introduced. Willis is a delightful writer whose characters speak with the sort of ironic detachment of the modern person and yet still remain fully human and approachable (in this, she reminds me of Walker Percy). Her subject in bellwether, appropriately enough, is fads and trends and why it is that they ebb and flow across the human condition subject to a tidal pull all their own. The book offers an answer (though I don't think Willis believes it completely): that genius arises out of chaos, as a sort of self-organized criticality that forms because anything else would cause total system collapse; and that trends are both part of this self-organization and also the result of human bellwethers who are "a little faster, a little more greedy". Bellwethers lead without leading (though not in the Taoist sense <smile>) and move at least partly to their own deep beat... pulling at least some of the rest of us in their wake. Regardless of its sociological value, however, this book is a wonderful read. Highly recommended. Slurping up files is one of Perl's strengths and I've always assumed that Lisp could not do as well. I was wrong. This slurping in Lisp page demonstrates how at least some Lisps can do Perl one better. I found it via Stefan School's blog. Go Lisp, go!
brief note on CL-Graph Someone rightly mentioned to me that CL-Graph doesn't have much in the way of a high level overview. Tinaa documentation isn't bad for seeing the trees but it doesn't make the forest any easier to navigate. Here, then is a very brief snapshot of CL-Graph from on high: Structurally, a graph is a container-uses-nodes-mixin. This means that the things you put in the container are wrapped up in some other structure (a node). Examples of containers that use nodes are graph-container, binary-search-tree, heap-container. Examples of containers without nodes are list-container, array-container, basic-queue and so forth. In practice, this means that when you add a thing to a graph, the thing gets wrapped up in a vertex structure. When you do a find-vertex, you get back the vertex structure (not the element you added). For example: ? (add-vertex *graph* 23) #<23> :new ; tells you that this was a new vertex ? (find-vertex *graph* 23) #<23> ; the vertex ? (describe *) #<23> Class: #<STANDARD-CLASS GRAPH-CONTAINER-VERTEX> Wrapper: #<CCL::CLASS-WRAPPER GRAPH-CONTAINER-VERTEX #x8686356> Instance slots ELEMENT: 23 DEPTH-LEVEL: 0 VERTEX-ID: 0 TAG: NIL GRAPH: #<GRAPH-CONTAINER [1,0] #x8E0EBD6> COLOR: NIL RANK: NIL PREVIOUS-NODE: NIL NEXT-NODE: NIL DISCOVERY-TIME: -1 FINISH-TIME: -1 VERTEX-EDGES: #<VECTOR-CONTAINER 0 #x8E0B3EE> ; No value ? (element (find-vertex *graph* 23)) => 23 23 The main functions for creating and querying graphs are add-vertex / find-vertex / delete-vertex and add-edge-between-vertexes / find-edge-between-vertexes and delete-edge-between-vertexes. Once you have a graph or a vertex, you can map a function over its elements / edges using iterate-elements or iterate-edges. If you want to actually map over the vertexes, you can use iterate-nodes. There are also lots of iteration / collection functions for dealing with the children and parents of vertexes in directed graphs. Quite a lot of CL-Graph is bound up with different ways of querying, collecting and iterating over the graph structures. It also has some simple graph algorithms (mostly of the summary sort like clustering coefficient, vertex degree, etc.) and it has decent export to DOT and integration with GraphViz. It's main claim to fame is that it does a good deal of bookkeeping to keep track of vertexes and edges so that you can concentrate on using the graph to do your stuff.
Notes on a Lisp Library Management I'm continuing to muddle through various ideas for Enterprise Lisp. One thing I think that the Lisp community needs is better library management. This means:
ASDF (and MK-Defsystem and a few other defsystems) serve the system definition role admirably. My own defsystem-compatibiliy aims to make it easier to use multiple defsystems simultaneously (*). ASDF-Install does a great job grabbing new systems. The missing piece is system maintenance: making sure you have the latest of every and that it all works together. What follows is my proposal for improving the situation. Desiderata
The view from above The system maintainer consists of the following processes: 
Questions and Answers This is the section where careful readers get to catch my errors and e-mail me as to how to make things better... So pay attention. Q: What do you mean by known systems? A: Initially, Checker will work with ASDF-Installable systems. The main point of this, however, is to let Checker know where a system is to be found so there is no reason that other systems could not be registered for the service. Q: How does checker know when a system has changed? A: Checker does two periodic checks on known systems. First, it compares the :last-modified date of an HTTP head request. If the last-modified date of the systems tarball is greater than the date Checker saw before, then the system may have changed. Secondly, Checker uses the system definition to build a system signature: a list of system files and file-write dates. Checker can compare the signature it has with the new signature to see if files have been added or removed and to see if file dates have changed. Note that occasional false positives are OK. Q: What about *features*? Won't they mess with this signature you're talking about? A: Yes, features are a problem. For the curious, there are 52 systems on the Cliki that contain features in their system defs (out of about 250). These systems contain 40 different features. Most of them are operating system or Lisp implementation related. A few are more specialized. I propose to get around the features problem by using a custom reader to grab every file in a system definition. Conclusion The system described above is being actively developed and I hope to have something beta-testable sometime this work (oh, oh, that sounds like a promise. I have to stop writing now and get back to coding). Please let me hear from you if I've missed something obvious or subtle. (*) To be fair, defsystem-compatibility (DSC) currently only supports ASDF and EKSL's system definer: Generic Load Utilities. However, only time and lack of clamor prevents DSC from working with others. (**) Yes, this gives a central point of failure but web server and hardware is up to the task.
On the evolution of libraries Lately, I've been pondering on the problems of library evolution. At one extreme of library creation is the put-it-all-in-here. At the other is the carve-nature-at-its-joints and have lots of dependencies. Philosophically, I tend towards the latter because shared code sucks (do it once, do it right) but it leaves one with an icky library management problem. Then there's the issue of what to do with patches and additions? If a maintainer exists, patches and additions can pass through them. If there isn't a maintainer or if they find the changes don't fit with their vision of the library, then what happens? One can create a fork -- but that adds another library that may have issues working with the pre-fork version and it raises the code duplication issue in spades. One can also create another library that depends on the original... now we just have the yet-another-library problem. It's interesting that we're culturally willing to create websites as wikis and are finding ways to deal with the problems of attribution and controversy. What would happen if we had a code wiki where everyone could edit everything. It's a scary thought because it takes so little to make code fail and because a little code can do a lot of damage very quickly (the software fault travels around the world before the patch gets out of bed or something...). Rambling on, this makes me think of the difference between syntax and semantics: word wikis work pretty well because they use language to pass around semantic knowledge and humans are remarkably good at dealing with faults. Code webs deal only in syntax and computers are almost unbearably brittle. The magic of computing, as my PhD advisor liked to say, is that it performs semantic transformations using only syntax. The problem is that the semantics all comes from the programmer. So what sort of infrastructures would make a code wiki possible?
Ora Lassila on the semantic web Ora Lassila, a research fellow at Nokia, gave an interesting sounding talk (pdf) last week to the W3C advisory committee (I wasn't there, I'm just name dropping - smirk). His slides have include these provocative bullets:
I think that this sounds about right and is part of the reason for the general attractiveness of XML and Lisp : code is data is code. When we need to, we can grovel over our source easily and do cool stuff.
Social Phenomenom (er, Phenomenon) I've been looking at Ruby quite recently while working on Montezuma with John Wiseman on a port of Ferret (which is a ruby port of the java Lucene text indexing engine). I don't see anything particularly special about Ruby; overall, it seems like another reinvention of the wheel with more syntax with which to be confused! That said, all these "scripting plus" languages do fit a niche that Lisp has not been able to play in because Lisp has too much baggage (in my holy opinion) and because Lisp qua Lisp is missing the batteries like sockets, web services, etc... What I find most interesting, however, is the social phenomenon: why and how did Ruby and Python make it to the big time? Why did Harry Potter become such a hit? Ruby, Python and Harry are all good but none of them seem, to me, markedly better than their competitors... Who researches this kind of stuff? Are there papers out there that claim to explain what is going on? Tipping Point? Wisdom of Crowds. Where is Malcom Gladwell when you need him?
the Problem with Threads Someone mentioned Edward A. Lee's The Problems with Threads paper over at Lambda the Ultimate and I just finished reading it. The argument is simple: people live in a concurrent world so concurrent programming shouldn't be all that hard. But concurrent programming with threads is very hard so what's the problem? The problem is that threads are strongly non-deterministic and programmers (and languages and frameworks) must make great efforts to recreate determinism. It would be better, thinks Lee, to keep everything as deterministic as possible and only add non-determinism when necessary. I.e., we need better abstractions (perhaps built on top of threads). Lee goes on to mention many of the alternatives and suggests that they have not taken deep roots in part because they are not necessary (yet -- there just aren't enough massively parallel non-trivial systems in use), because sequential programming is at the heart of all mainstream languages (and most non-mainstream ones too), and because it's hard to do multi-language programming (tool support, etc). The message, he says, is clear: "we should not [aim to] replace existing languages." He suggest coordination languages as a different extension mechanism (think Aspects and weavers) but even those have a hard time taking root because "language wars are religious wars and few of these religions are polytheistic" (I love that quote). He cites work with graphical notations and draws the parallel to UML's ability to abstract above specific language syntax and allow multi-languge use. There is, he hopes, hope. This is a very readable paper with great examples and stories about had hard (thread based) parallelism can be and is. I don't know if his answers are correct (but who am I to know?! <smile>) but I strongly agree that we do need better models of computation and better languages to support those models. This should, perhaps, give Lispers hope. After all, what language to you know that is better positioned to support malleable syntax and coordination language experimentation?
Apologies: it was my fault, not CL-Markdown's I meant <pre>. Depressing.
CL-Markdown update I've removed CL-Markdown's dependency on LML2 (though CL-Markdown-Test still uses it to generate the comparison reports). I've also fixed several small tickets. The most important one probably being the correct handling of line breaks with <pre>s sections. I also changed the signature of the markdown form. The new one looks like:
I hope that's clear. It makes it easy to go from strings or files to strings or files in any supported format (i.e., in HTML There is still some distance to go before the basics are complete but things are starting to look pretty good.
Proving once again that I can do math, but not arithmatic... Michael Price wins the prize for quickly informing me that the version number of ASDF-Install on the website was 0.5.1, rather than the 0.5.2 that I had claimed. This should now be fixed. I should also mention that ASDF-Install has a new mailing list all to itself. See asdf-install-devel for details. Finally, I'm in the process of bringing the tutorial back up to date.
ASDF-Install update The version now on Common-Lisp.net has been radically restructured and also has several patches and improvements. The restructuring just pulls various related forms out of installer.lisp and into their own homes. Mostly, this was to help me organize and maintain it. The improvements include:
Please let me know if anything goes astray.
Software status update
Doing it like we've always done A nice essay on the dangers of resting on our mental laurels
Enterprise Lisp Wiki I just created a simple wiki for Enterprise Lisp on Infogami (yes, they are the same people that did Reddit. So. <smile>). You'll need to join in order to edit. Let me know if that's a problem for you. I'm happy to post changes and ideas from other people if they send me an e-mail.
Drew McDermott and anti-literacy Drew McDermott has a nice essay on the benefits and difficulties of literate programming. The bit that resonates with me the most is this paragraph:
except that I find this happening to me about three-quarters of the time that I try test-first development. I don't know exactly what I'm doing (cf. Paul Graham's wonderful introduction to On Lisp or ANSI Standard Lisp (I can't remember which one)) and writing tests often turns out to be silly. Perhaps I'm just not getting out or can't rid myself of other bad habits or perhaps test first isn't something that makes sense all of the time.
On Selfish Memes: Culture as complex adaptive system This is a paper (pdf) that hits all the buzzwords hard and then hits them some more. We've got evolution, complex adaptive systems, power laws, dynamical systems, fitness landscapes, memes, phemes, and all the rest. The goal is to use memetics (which sounds way too muchlike dianetics to make me comfortable!) to explain culture and society. The chief problem is that (in the papers own words)
(The writing style -- loosely speaking -- doesn't help) The level of abstraction hides some much philosophical sleight of hand and wooly thinking that no amount of formalism, charts and graphs can save us.
This is one of the oldest self-inflicted tricks in the book of bad simulation: insert that which is to be proved into the foundations of the simulation and be astonished when the simulation acts that way (note that I'm not suggesting bad faith on the practitioner of the paper -- it's just that natural stupidity is something that must be fought against constantly. In closing, I should mention that this paper is one of the references for a United States Army SBIR Request for Proposal. The RFP is reasonable enough but if this paper is supposed to be guidence... I'm more frightened than ever. | |