Monday, 6 April 2009

Proebstring's Law

An interesting law from Todd Proebsting which I hadn't heard before:

advances in compiler optimizations double computing power every 18 years


Compare this to Moore's Law and it suggests that research into compiler optimization isn't going to get as much say programmer productivity (better tools for writing, analysing and understanding code). This is similar to the arguments made in favour of functional programming languages. It doesn't matter if functional languages are any slower at run-time; the development time improvement means they win overall.

Saturday, 4 April 2009

Twitter using Scala

Twitter was previously using Ruby for large parts of the application, but has recently introduced some parts of Scala.

An interview with some of the developers over at Artima discusses some of the reasoning there.

It's an interesting decision, seemingly motivated by the need for performance and static typing. Let's look at the static typing reasoning:

As our system has grown, a lot of the logic in our Ruby system sort of replicates a type system, either in our unit tests or as validations on models. I think it may just be a property of large systems in dynamic languages, that eventually you end up rewriting your own type system, and you sort of do it badly. You’re checking for null values all over the place. There’s lots of calls to Ruby’s kind_of? method, which asks, “Is this a kind of User object? Because that’s what we’re expecting. If we don’t get that, this is going to explode.”


I'm not sure I agree with this reasoning of the Twitter guys description. Firstly, Ruby code with a bunch of kind_of? littered everywhere is almost certainly bad. Perhaps this is because as the system has evolved the initial design decisions are not quite right, and the system hasn't evolved and accrued too much technical debt in the process. The benefits of hindsight mean that you could probably rewrite bits in any language and see improvements.

Secondly, what does replicating a type-system mean? Before we can replicate a type system, we need to know what one is:

[A type system is a] tractable syntactic method for proving the absence of certain program behaviors by classifying phrases according to the kinds of values they compute. (Pierce 2002).


This sounds an awful lot like how I define a test; I'm looking to test that certain behaviours hold true under whatever conditions I write my test for.

Has using Scala static-typing meant that you can write any less tests? The flavour of errors that static typing can detect is pretty large but it's not exhaustive, you've still got to write tests to show that some behaviours work as expected. I can't find any information on this, but this would be really good information to get!

The other argument seems to be performance. Scala (and Clojure) leverage the JVM so have the advantage of 10 years+ of development on that. There's a version of Ruby for the JVM (JRuby), but that was rejected because some third-party Ruby libraries depended on native C libraries that aren't compatible.

I'm not sure where I sit yet on the dynamic / static typing debate, and to be honest I'm not sure it's that interesting given the tests as types argument (unless the amount of tests decreases significantly if you use types). I don't think the Artima article tells us any of the real answers of whether one is better than the other for Twitter. As I mentioned before, rewriting it in any language is probably going to result in cleaner, clearer code just because of the accumulated knowledge of the past one.

Implementing Minilight in Clojure (2)

Any 3D rendering rubbish seems to require a 3d vector class of some description. A quick Google Code Search brings up thousands of results.

The Minilight code is no exception.

For now, my Clojure version will just use the built in vec based sequence. I'm sure that I'll find I'll need raw Java array access, but it's inconvenient to start with the ugly stuff.

ANSI Common Lisp mentions this explicitly,
"Lisp is really two programming languages, a language for writing fast programs and a language for writing programs fast."
I think the same applies to Clojure, I know I can always get the raw speed of Java when I need it, but I can stay nice and high level for most of the time (80/20 rule and all that).

So here's the most basic set of vector like functions I could write for Clojure. The cross product definition is the ugliest out there. Probably because it's only valid in 3 (or 7 according to Wikipedia!) dimensions!



(defn dot
"Dot product of two vectors"
[v1 v2] (reduce + (map * v1 v2)))

(defn cross
"Cross product of two 3 vectors"
[v1 v2]
(vec
(- (* (get v1 1) (get v2 2)) (get v1 2) (get v2 1))
(- (* (get v1 2) (get v2 0)) (get v1 0) (get v2 2))
(- (* (get v1 0) (get v2 1)) (get v1 1) (get v2 0))))

(defn unary-minus
"Negation of a vector"
[v] (vec (map - v)))

(defn unitize
[v]
(let [len (Math/sqrt (dot v v))
overlen (if (not (zero? len)) (/ 1 len) 0)]
(vec (map (fn [x] (* overlen x)) v))))

(defn between
[mini maxi x]
(min (max x mini) maxi))

(defn get-clamped
"Clamp v within the bounds minimum and maximum"
[v mini maxi]
(vec (map between mini maxi v)))