Programming Epiphanies

What is a programming epiphany? It's that moment that you have when you realize that the way you've coded is wrong wrong wrong, and there's a better way to do things.  Here's a few of my programming turning points.
When I was studying Computer Science at the University of Southampton object-oriented programming was simply:

• Inheritance
• Encapsulation
• Polymorphism

Inheritance was first, and that meant it was the highest priority for me.  If I could find a way of inheriting A from B, I probably would.  Encapsulation?  That's just wrapping all those lovely members with getters and setters.  Polymorphism wasn't something I really ever thought about.  It came last so it was probably something I could get away without.

Around this time, I was programming using C++ and Microsoft Foundation Class.  My understanding of MFC at the time gelled quite well with how I understood object-oriented code.  Plenty of inheritance!  I even felt I saw a use for polymorphism with overriding some of those virtual function things.

Things got a bit better towards the end of my degree.  I found a copy of Effective C++ and read about const correctness.  I remember having particularly knotty issues in some of my code (probably due to my understanding of encapsulation) and not being able to find the bug.  By liberally sprinkling const over the code base (it's like a virus!) I eventually found my unwanted mutation.  My first epiphany; design code correctly and make the bugs impossible.

I bumbled my way through a research degree, and then became a research scientist for a bit.  I never really wrote code that anyone else had to read, so my code was just good enough.  My next big leap in learning came with my next job. For posterity here's the original advert (no idea how I got in!).

Day 1.  Someone mentions this thing called the visitor pattern, then ubiquitous language and then a few more things.  WTF?  Visitor pattern?  Names matter? Oh dear, there's a huge amount I don't know.  I managed to get through the day without getting found at, visited Amazon and ordered a few dozen books.  My second epiphany, smarter people than me have likely solved your problems before; get reading.  I went through my pattern craze, no doubt needlessly applying them sometimes, but I worked it out of my system.

At some point came another one, singletons are bad.  I think everyone realizes this at some point and has an instant recoil of all design patterns.  I love functional programming, and so I remember finding Design Patterns in Functional Language presentation and thinking to myself that maybe patterns are just missing language features?

All good things must come to an end, and next I turned to the dark side of enterprise programming.  If you don't know what this is, it's very simple.  A sales person promises impossible to a clueless manager, and then a team of software engineers will work at solving the impossible problem with an equally impossible deadline. I identified a huge amount with Death March, but lacked the gumption to quit.  Not all was bad though, by seeing every possible variant of wrong I learnt something incredibly important.  You can't tolerate complexity.

Quality isn't just something you can get back another day; quality matters.  Once you've lost quality, once you've lost clean code, you're fucked.  You might get away with it for a bit (the human brain can deal with remarkable amounts of complexity), but in the end that ball of mud will crush you.

So how'd you build in quality from the ground up?  As part of my job, I visited an extremely Californian company that practiced XP.  All code pair-programmed, all code with a failing test first.  This made a big (though not very immediate!) impression on me.  It wasn't until I read Growing Object Oriented Software Guided By Tests that it clicked in a way that felt right and TDD seemed a bit more natural.

So what are your programming epiphanies? What are the moments in your development so far that changed the way you think about writing code?

Legacy Code Retreat

I had the pleasure of attending a Legacy Code Retreat at the weekend organized by the Cambridge Software Craftsmanship group. Legacy code is a subject close to my heart. I've spent most of my working life immersed in it, and I'm been at least fairly successful is altering my point of view from "OMG THE CODE IS SHIT" to "what can I learn from this?".

The event was excellently co-facilitated by Alastair Smith and Erik Talboom and followed a familiar format of short sessions with tight constraints and retrospectives to discuss.

This was my first encounter of TDD as if you meant it which was certainly a challenging experience and one that I'll definitely try again.

I enjoyed the discipline of baby steps. We set a timer for a minute and either had to write a test, or perform a refactoring (but not both) within that minute. Failing to do that would reset the code and you'd start again. The object of this exercise was to judge the smallest steps you can work in. This is really interesting experiment to do; it forces you to think of your work as a series of smaller and smaller steps.

I also learnt the term "Golden Master", refering to a record of the working system. Working with Andrea, we were tasked with writing system level tests to capture the behaviour. We quickly hit upon the idea of simply capturing the entire output of the program as the simplest end-to-end test. This proved to be amazingly powerful; armed just with an end-to-end test we felt very confident about changing the code. We understood that the seeded randomness didn't cover every case, but we figured we could just capture more runs of the program to increase the confidence. Our simple measure of the confidence we had was that commenting out or changing a line of code in the program always seemed to result in that integration test failing.

The main take away for me was just that the discipline of focusing on a single thing is incredibly powerful. Instead of "fixing all the easy things", I'm going to try and take a step back and focus on something specific (be it renaming, simplifying conditionals, or just increasing test coverage).

Technical Debt and Legacy Code

It's time to ban the phrase "technical debt". It's a metaphor that's been stretched to mean too many things, from hand waving usages to make teams "go faster" to being a catch-all excuse for developers to write terrible code. I'd like to challenge you to avoid saying "technical debt" and instead try to define the actual problem. By stating the problem concretely, you're already closer to the solution.

Let's go right back to the beginning. Ward Cunningham introduced the metaphor of technical debt at OOPSLA 92 in "The Wycash Portfolio Management System" experience report. Let's look at what he says:

Another, more serious pitfall is the failure to consolidate. Although immature code may work fine and be completely acceptable to the customer, excess quantities will make a program unmasterable, leading to extreme specialization of programmers and finally an inflexible product. Shipping first time code is like going into debt. A little debt speeds development so long as it is paid back promptly with a rewrite. Objects make the cost of this transaction tolerable. The danger occurs when the debt is not repaid. Every minute spent on not-quite-right code counts as interest on that debt. Entire engineering organizations can be brought to a stand-still under the debt load of an unconsolidated implementation, object- oriented or otherwise.

What Ward is saying here is that the build up of technical debt starts when you ship something with an incomplete understanding of the problem. It's the acknowledgement that we never ship the right thing first time. This isn't news to anyone! No matter what the customer says, the first time they get the new feature they want something slightly different. Technical debt starts to occur when you don't reflect that difference back to the underlying code.

The original definition of technical debt is nothing to do with quality of the code alone, it's simply an acknowledgement that once your code no longer models the problem, you're in trouble. Things aren't going to get better until you acknowledge that and fix things. I'm willing to bet that this definition of technical debt isn't the one you use.

Unfortunately, when I hear the term technical debt it's usually in the context of moaning about legacy code.

Most of us know what legacy code is. It's really hard to work with. It's difficult to like (though some might say there is Joy in Legacy Code), and progress is slow. Saying it's full of technical debt is almost certainly true, but it's not very helpful. You need to define that debt in meaningful ways so you can do something about it (or explain your woes concretely to the rest of the team so you can make time to tackle it as a team). In this context, technical debt is nothing but a lazy phrase to stop you concretely defining what the problems really are. As Bob Martin says "A Mess is Not Technical Debt" article.

Ever wonder why it's so hard to convince your boss to have time to tackle technical debt? You're fighting the wrong battle. You're describing the problem in terms of your domain, not the problem domain. What happens if you're not allowed to say "technical debt"? Now you're forced to describe the problem in concrete terms. How is the code mismatched against the customer requirements? Which one is the more compelling case to make to your boss?

So stop being lazy; define problems in concrete terms and don't use the dreaded phrase "technical debt".