Zach's Journal

Like lemonodor of old, I downloaded all asdf-installable systems on cliki. Instead of loading them, though, I just loaded their system files to get some metadata.

(Back then, there were only 202 asdf-installable systems, and 50% of them failed to load. There are over 450 now, but I'm not sure if the success percentage is better or worse.)

The resulting file has one sexp in it, and each element in the sexp is a list. The car of the list is a system name, and the cdr is a list of other systems it depends on.

I used that data to make a graphviz dot file, but I wasn't entirely happy with the results. Can you make something that looks awesome from all that dependency data?

Giant Pumpkin Pyramid, originally uploaded by shannabeane.

We went to the pumpkin festival over in Cumberland this afternoon. They're trying to break a world record for jack-o-lanterns, and while most of the pumpkins were just sitting in grids on the ground (with little tags from an accounting firm certifying the counts), the most impressive display was on this multilevel scaffolding structure.

Rich Hickey left a comment on my last post with a link to the videos of his talk:

Part 1 and Part 2.

These are essentially screencasts of his slides, with audio added. I haven't watched them (I was there in person, after all) so I'm not sure if the Q&A audio was faithfully captured; check it out and let me know.

Update Michael Livshin reports the questions are not always audible, but the answers usually make up for it.

Rich Hickey, originally uploaded by Zach Beane.

Rich Hickey's Clojure talk last night was a big hit. The room was packed, the 90 minute talk ran to a dense 180 minutes, and he got two ovations from appreciative Lisp and Scheme hackers. The back and forth of questions and answers was exciting, with the questions exhibiting a good knowledge of the problems in the design space, and Rich's answers demonstrating he has thought about all the same problems in detail, made key decisions, and followed them through. It was also awesome that he could make subtle humorous points about esoteric languages and everyone knew exactly what he was talking about.

Highly recommended.

Tim Kerchmar has written an excellent article about working with a vendor to debug a problem. His specific context is working with Roger Corman, but the advice applies in many other situations. For example, I think Lisp library authors are in the same boat as one-man vendors, except you aren't paying them.

Here's a bit of it:

1. If the software vendor is a one man shop, support is a distraction from other work. He is not sitting around hitting refresh on his email waiting for the next question to answer. Your good attitude about this frustrating bug in his product will go a long way towards his feelings of good will.
2. You don't want to piss him off. He's probably the only guy in the world who knows this software product deeply, and if you couldn't trivially solve the bug from header files or documentation, you're going to need his help.
3. When it comes to support, you get what you pay for, although the author's pride in his product can go a small ways.
4. He is one person. The official term for you is "user". There are lots of you guys clamoring for his time.
5. If you do your homework, he will do his.
6. Don't distract him with your bad spelling or grammar.

In SBCL, ASDF hooks into the REQUIRE mechanism so you can type (require 'mysystem). On other systems you can also figure out a way to hook REQUIRE, so you can avoid typing (asdf:oos 'asdf:load-op 'mysystem).

I got tired of trying to figure out how to do this in different ways on different Lisp systems, so below is a file I've been loading from my init files to get consistent, concise ASDF loading across implementations. It also adds a crude registry maintenance system and a way to reload a system without using :FORCE. :FORCE has the downside of recompiling and reloading all other dependencies, which is never what I want.

It's a work in progress, but so far I've been pretty happy using (asdf:load* 'mysystem) and (asdf:reload 'mysystem). YMMV.

(require 'asdf)

(in-package #:asdf)

(export '(load* reload register register-permanently))

(defvar *registry-file*
  (merge-pathnames (make-pathname :directory '(:relative "asdf")
                                  :name "registry"
                                  :type "sexp")
                   (user-homedir-pathname)))

(eval-when (:compile-toplevel :load-toplevel :execute)
  (let ((registry (probe-file *registry-file*)))
    (when registry
      (with-open-file (stream registry)
        (loop for form = (read stream nil)
              while form do (push form *central-registry*))
        (setf *central-registry*
              (remove-duplicates *central-registry* :test #'equalp))))))

(defun load* (system &key verbose)
  (oos 'load-op system :verbose verbose)
  t)

(defun reload (system)
  (let ((name (coerce-name system)))
    (remhash name *defined-systems*)
    (load* name)))

(defun register (form)
  (pushnew form *central-registry* :test #'equalp))

(defun register-permanently (form)
  (register form)
  (ensure-directories-exist *registry-file*)
  (with-open-file (stream *registry-file*
                   :direction :output
                   :if-exists :append
                   :if-does-not-exist :create)
    (prin1 form stream)
    (terpri stream)))

;;; Automatically recompile stale FASLs

(handler-bind ((style-warning #'muffle-warning))
  (defmethod perform :around ((o load-op) (c cl-source-file))
    (handler-case (call-next-method o c)
      (#+sbcl sb-ext:invalid-fasl
       #-sbcl error ()
       (perform (make-instance 'compile-op) c)
       (call-next-method)))))

Do you have any ugly local hacks that you think aren't polished enough to share, but that make your life much easier? Share them anyway, and let's clean them up and use them and improve life everywhere.

Sometimes asdf-install drives me crazy. Here's what I've been doing in my ~/.sbclrc so it doesn't ask me where I want to install all the time:

(require 'asdf)
(require 'asdf-install)

(setf asdf-install::*locations*
      (list (list "~/asdf/source/" "~/asdf/systems/" "ASDF")))

(push #p"~/asdf/systems/" asdf:*central-registry*)

(fmakunbound 'asdf-install::where)

(defun asdf-install::where ()
  (first asdf-install::*locations*))

On a related note, I can't live without tilde expansion in sbcl any more.

The Boston Lisp meeting is tomorrow night. Rich Hickey will talk about Clojure. I'll be there, and you should be too. For more info, check Fare's post.

Bill Clementson wrote a summary of the September Vancouver Lisp meeting, including slides and audio of Hans Hübner's presentation.

Chun Tian released a new version of cl-net-snmp. The comp.lang.lisp announcement has some interesting details. He registered an enterprise number for Lisp with the IANA and has been working on a Lisp MIB. I think that is pretty cool.

Dan Weinreb added his take on the failure of Lisp, and it got a zillion comments.

I've just published ZS3, a Common Lisp library for working with Amazon S3. I wrote it to support the Alzheimer's Art Quilt Initiative virtual patch generator, which stores the patches on S3.

I first tried using CL-S3, but I couldn't get it working on SBCL for binary file uploads. There were some workarounds possible, but it seemed like starting from scratch might work better for me.

ZS3 has been tested and works fine on SBCL, Clozure CL, Allegro CL, and LispWorks CL. It doesn't work on CLISP due to a Gray stream issue; I haven't been able to figure out the problem. CLISP debuggers welcome! I haven't tested it on other Common Lisps, so I'd love to hear if it works or doesn't work elsewhere. Update The patch linked here does the trick on CLISP.

It should install fine with asdf-install. It's also available from git.xach.com.

To give you a general idea of what it can do, here's the defpackage form:

(defpackage #:zs3
  (:use #:cl)
  ;; Credentials
  (:export #:*credentials*
           #:access-key
           #:secret-key
           #:file-credentials)
  ;; Buckets
  (:export #:all-buckets
           #:creation-date
           #:name
           #:all-keys
           #:bucket-exists-p
           #:create-bucket
           #:delete-bucket
           #:bucket-location)
  ;; Bucket queries
  (:export #:query-bucket
           #:continue-bucket-query
           #:bucket-name
           #:keys
           #:common-prefixes
           #:prefix
           #:marker
           #:delimiter
           #:truncatedp
           #:last-modified
           #:etag
           #:size
           #:owner)
  ;; Objects
  (:export #:get-object
           #:get-vector
           #:get-string
           #:get-file
           #:put-object
           #:put-vector
           #:put-string
           #:put-file
           #:copy-object
           #:delete-object
           #:delete-objects
           #:delete-all-objects
           #:object-metadata)
  ;; Access Control
  (:export #:get-acl
           #:put-acl
           #:grant
           #:acl-eqv
           #:*all-users*
           #:*aws-users*
           #:*log-delivery*
           #:acl-email
           #:acl-person
           #:me
           #:make-public
           #:make-private)
  ;; Logging
  (:export #:enable-logging-to
           #:disable-logging-to
           #:enable-logging
           #:disable-logging
           #:logging-setup)
  ;; Misc.
  (:export #:*use-ssl*
           #:make-post-policy
           #:head
           #:authorized-url
           #:resource-url)
  ;; Util
  (:export #:octet-vector
           #:now+
           #:now-
           #:file-etag
           #:parameters-alist
           #:clear-redirects)
  (:shadow #:method))

For the full scoop, check out the full documentation.

This would have been much more difficult to write, portably, without the great (and portable) libraries Drakma, Closure XML, and Ironclad.

I knew a gentleman, who was so good a manager of his time, that he would not even lose that small portion of it, which the calls of nature obliged him to pass in the necessary-house; but gradually went through all the Latin poets, in those moments. He bought, for example, a common edition of Horace, of which he tore off gradually a couple of pages, carried them with him to that necessary place, read them first, and then sent them down as a sacrifice to Cloacina: this was so much time fairly gained; and I recommend you to follow his example.

— Letter XXI, Lord Chesterfield's Letters to his Son

These embarrassments that the Quakers suffer'd from having establish'd and published it as one of their principles that no kind of war was lawful, and which, being once published, they could not afterwards, however they might change their minds, easily get rid of, reminds me of what I think a more prudent conduct in another sect among us, that of the Dunkers. I was acquainted with one of its founders, Michael Welfare, soon after it appear'd. He complain'd to me that they were grievously calumniated by the zealots of other persuasions, and charg'd with abominable principles and practices, to which they were utter strangers. I told him this had always been the case with new sects, and that, to put a stop to such abuse, I imagin'd it might be well to publish the articles of their belief, and the rules of their discipline. He said that it had been propos'd among them, but not agreed to, for this reason:

"When we were first drawn together as a society," says he, "it had pleased God to enlighten our minds so far as to see that some doctrines, which we once esteemed truths, were errors; and that others, which we had esteemed errors, were real truths. From time to time He has been pleased to afford us farther light, and our principles have been improving, and our errors diminishing. Now we are not sure that we are arrived at the end of this progression, and at the perfection of spiritual or theological knowledge; and we fear that, if we should once print our confession of faith, we should feel ourselves as if bound and confin'd by it, and perhaps be unwilling to receive farther improvement, and our successors still more so, as conceiving what we their elders and founders had done, to be something sacred, never to be departed from."

Synchronized vigilance., originally uploaded by panacealater.

My dad saw these out the back window of the barn this morning. Momma and two "babies".

NFL commissioner Roger Goodell said Thursday he expected the league’s competition committee would review the rule that possession could not change because the whistle blew during the offseason, as it has in the past.

Thank you, Associated Press.

Ami Simms of the Alzheimer's Art Quilt Initiative saw roflbot last month and contacted me about a project for World Alzheimer's Day: help people create their own "virtual quilt patch" in memory of a loved one with Alzheimer's. I host the virtual patches so anyone can make one and put it on their website, blog, etc.

The patch creator is open to the public now. You can find out more about the project here.

(Powered, of course, by Common Lisp and various Wigflip tools.)

I'm glad Jasko made it easy for Cusp users to make executables, but I found his characterization of newbie interactions pretty insulting. I like to help people with Lisp stuff, and the conversation usually goes like this:

• Q: "Does Lisp have a compiler?"
• A: "Yes, you can use COMPILE or COMPILE-FILE..."
• Q: "Awesome!" time passes "Wait, what I really want is something else..."
• A: "Ok, you can do that with save-lisp-and-die, or cl-launch, and commercial Lisps make it pretty easy too..."
• Q: "Cool, I'll check it out" or "If there isn't something exactly like gcc for Lisp, Lisp sucks!"
• A: "Glad to help!" or "You can still get stuff done even without a single-file executable" or "Whatever, dude."

Here's a peek at a library I'm working on for Amazon S3 access from Common Lisp. It uses Drakma, Closure XML, and Ironclad.

(Why make a new library at all, when there's CL-S3? Unfortunately, I had trouble working with binary data with CL-S3 on SBCL. It has been easier to make a new library than work around the trouble.)

;;; Load credentials from a file when needed

;;; Let's make a bucket to work in.

;;; Saved for later use

;;; Put some boring octets...

;;; Now let's fetch them back as strings...

;;; The default for GET-STRING and PUT-STRING is UTF-8. What about
;;; other encodings?

;;; Conditional fetching

;;; Partial fetching

;;; Upload a file, key name defaults to FILE-NAMESTRING of pathname

;;; Oops, it's private by default, make it publicly accessible at
;;; http://zs3-demo.s3.amazonaws.com/bork.jpg

;;; Let's put up a non-public version of that file

;;; Oops, non-public

;;; secret.jpg is not accessible. But let's make 
;;; an url that will let people access it for the next 7 days

;;; Let's copy the secret.jpg to a third name

;;; Compare the objects that result

;;; What do the first four octets of bork.jpg look like?

;;; What's in place now?

;;; If you want a copy to try out, email me.

I think a better headline would be Genius Feature Coming to Older iPods Classic.

Github hackers created "impact" charts that are similar to the movie charts, but they used the HTML canvas instead of generating PNGs. Looks cool, and is more interactive than mine.

I use Firefox 2. Did you know it's ancient and not standards-compliant? Then clearly you haven't visited thn.gs with it. If you do, you get this lovely message:

Your browser is not supported (probably it won't be)
and we are not sorry about it.

Why?
Normally people use modern, standarts-compliant browsers and they're happy with them. If we would allow you to see our site in your browser, it will look like a mess, but we don't care because it's not our mistake. Basically, your browser simply can't handle such an application like Things, but there's plenty of recommended browsers: Safari 3, Firefox 3, Opera 9, Chrome, etc.

I'm working on a simple S3 client. I just finished grabbing all the deps and sub-deps I need:

• alexandria.asd
• babel-streams.asd
• babel.asd
• cffi-grovel.asd
• cffi.asd
• chunga.asd
• cl+ssl.asd
• cl-base64.asd
• closure-common.asd
• cxml.asd
• drakma.asd
• flexi-streams.asd
• ironclad.asd
• puri.asd
• split-sequence.asd
• trivial-features.asd
• trivial-gray-streams.asd
• usocket.asd

It was tiring. I am looking forward to the glorious future of Lisp distribution systems.