Monday, 12 January 2009

Metadata in Clojure

Each Clojure function can have a documentation string associated with it. This must be declared immediately before the parameters, for example:

(defn foo-baz
"Foo-bazes bar"
(+ bar 77))

user> (doc foo-baz)
Foo-bazes bar

This is actually short hand for:

(defn #^{:doc "Foo-bazes bar"} foo-baz
(+ bar 77))

#^ indicates to the reader that the following map is metadata to be associated with the next element read. You can use this to document def'd values too (e.g. (def #^{:doc "This is the value x"} x).

Metadata is associated with symbols or collections, and meta can be used to view the associated values. Initially I was doing (meta count) and expecting it to come up with the details about agent. This is wrong because count is an instance of the function, whereas what I need to pass as an argument to meta is the symbol associated with the function e.g.

user> (meta count)
user> (meta #'count) ; ^#'count also works (^ is a reader character to get the metadat)
{:ns #, :name count, :doc "Returns the number of items in the collection.
(count nil) returns\n 0. Also works on strings, arrays, and Java Collections and Maps",
:arglists ([coll]), :line 842, :file "core.clj"}

Note how extra metadata has arrived when it was compiled, including the line number of the original file, name space etc.

There are many cool potential applications of metadata in Clojure, though I haven't seen any implemented yet!:

  • Aspect oriented programming (side note inspiration for this came from Lisp Advice (see Emacsspeak mentioned in the book Beautiful Code.)
  • Software tools - knowing the line number and file helps in a number of ways, maybe could be used to do semantic diff (for example, re-ordering my functions wouldn't make any difference because they'd semantically still be the same).


  1. Hello Jeff,
    I enjoyed your post. One of the potential applications is the usage of metadata to produce editing environments more aware of macros. One of the advantages of the syntax-case macro system for scheme is that syntax-objects are passed to the lambda instead of raw s-expressions. This is neat because it allows the syntax object to be annotated with stuff like line numbers that the s-exp can't hold itself. PLT scheme uses this in their Dr. Scheme environment, and I understand that it allows the editor and debugger to better understand the macro-instead of the debugger breaking and showing the place in the macro expansion where code went wrong, it points to the location within the macro itself that caused it! I think this is a killer feature, and it would be really neat if Clojure aware editors and debuggers implemented something similar through metadata.
    I think Clojure macros hit a sweet spot between CL macros and Scheme macros - I think the mandatory namespace qualification is a win,as you get certain aspects of hygiene, and metadata might allow for more contextual info to be attached to symbols a la syntax case.
    I will continue to follow your blog :) have fun with lisp.

  2. Hi Pat,

    Thanks for the comments and pointing out some of the possible applications of metadata in the future.

    One of the ideas I had for a project along these lines was implementing type inference by annotating functions with metadata. I think if the base functions in core.clj were annotated with the data they required, then everything else could follow from Hindley–Milner inference.

    Of course, this wouldn't force you into static typing, it's just be like Clojure Lint.

    I wonder if this would allow you to layer something like Typed Scheme over the top whilst minimizing the syntactic overhead of types.

    I think I'll remain pondering this idea for some time :)

  3. Hello Jeff,

    That sounds like a killer application-the type-checker could then be turned on and off at compile time, and you could supply hints as to whether you want your functions to be statically checked or dynamic. Sadly, I am not too versed in the type-inference literature-are you thinking of a particular paper/implementation?

  4. Hi Patrick,

    I'm not too versed in the literature either I'm afraid.

    I looked at chapter 30 of Programming Languages: Application and Interpretation for a basic description.

    TAPL is the bible of types in programming languages, but I confess I haven't got a copy (yet!).

    In terms of a particular implementation, Hindley–Milner inference is the only algorithm I've looked at.

    Unfortunately I never got very far with this idea. I think there's definitely some scope to do some cool stuff there, but I never have the time to do anything other than toy projects :(

    If anyone wants to start some kind of project looking at this, then I'd be happy to contribute :)