tag:blogger.com,1999:blog-57439830442248336682024-03-13T08:02:46.530-07:00FatvatExploring functional programmingJeffhttp://www.blogger.com/profile/08195722595923882332noreply@blogger.comBlogger213125tag:blogger.com,1999:blog-5743983044224833668.post-2027942251514944172017-01-11T22:56:00.004-08:002017-01-11T23:01:38.812-08:00The Dark Path<div style="background-color: #f8fff8; box-sizing: border-box; color: #29323c; font-family: FrescoPlusNormal, Georgia, serif; font-size: 17.92px; margin-bottom: 0.8em;">
Over the last few months I’ve dabbled in building in doing my accounts using a <i>spreadsheet </i>(Google Sheets and Excel). T<span style="font-size: 17.92px;">he similarities are so stark that I wonder if this isn’t a new trend in managing accounts. If so, <i>it is a dark path.</i></span></div>
<div style="background-color: #f8fff8; box-sizing: border-box; color: #29323c; font-family: FrescoPlusNormal, Georgia, serif; font-size: 17.92px; margin-bottom: 0.8em;">
Both tools have integrated some functional characteristics. For example, they both update automatically to reflect changes in values. This is a good thing, in general. </div>
<div style="background-color: #f8fff8; box-sizing: border-box; color: #29323c; font-family: FrescoPlusNormal, Georgia, serif; font-size: 17.92px; margin-bottom: 0.8em;">
My problem is that both tools have doubled down on automation. Both seem to be intent on forcing me to write references to <em style="box-sizing: border-box; font-family: FrescoSansPlusPro-Normal-Italic; font-size: 1.1em;">every needed cell</em> in the spreadsheet!</div>
<div style="background-color: #f8fff8; box-sizing: border-box; color: #29323c; font-family: FrescoPlusNormal, Georgia, serif; font-size: 17.92px; margin-bottom: 0.8em;">
Now I don’t want you to think that I’m opposed to automation. I’m not. I use pen and paper, I use tools. I have a slight preference for pen and paper, but I'm using a spreadsheet too.</div>
<div style="background-color: #f8fff8; box-sizing: border-box; color: #29323c; margin-bottom: 0.8em;">
<span style="font-family: "frescoplusnormal" , "georgia" , serif; font-size: 17.92px;">It’s not the fact that spreadsheets automate that has me concerned. Rather, it is the </span><span style="font-family: FrescoSansPlusPro-Normal-Italic;"><span style="font-size: 19.712px;"><i>depth.</i></span></span></div>
<div style="background-color: #f8fff8; box-sizing: border-box; color: #29323c; margin-bottom: 0.8em;">
<span style="font-family: FrescoSansPlusPro-Normal-Italic;"><span style="font-size: 19.712px;">Previously, I'd created tables in Word. I can structure it so that it's correct; but I can also violate many of the "rules" whenever I need or want to. Word underlines some bits in green/red and throws up a few roadblocks; but not so many as to be obstructionist.</span></span></div>
<div style="background-color: #f8fff8; box-sizing: border-box; color: #29323c; font-family: FrescoPlusNormal, Georgia, serif; font-size: 17.92px; margin-bottom: 0.8em;">
Google Sheets and Excel, on the other hand, are completely inflexible when it comes to their rules. For example, in Google Sheets if I sum up a column then <i>by God</i> every thing in that column and <i>all the dependent references</i> have to be adorned by being a "number". There is no way, in this tool, to silently ignore me mistaking a string for a number!</div>
<div style="background-color: #f8fff8; box-sizing: border-box; color: #29323c; font-family: FrescoPlusNormal, Georgia, serif; font-size: 17.92px; margin-bottom: 0.8em;">
Now, perhaps you think this is a good thing. Perhaps you think that there have been a lot of bugs in systems that have resulted from un-coerced numbers. Perhaps you think that that if you aren’t escorted, step by step, through the dependent cells it would be risky and error prone. And, of course, you would be right about that. </div>
<div style="background-color: #f8fff8; box-sizing: border-box; color: #29323c; font-family: FrescoPlusNormal, Georgia, serif; font-size: 17.92px; margin-bottom: 0.8em;">
<span style="font-size: 17.92px;">The question is: Whose job is it to manage the</span><span style="font-size: 17.92px;"> </span><span style="font-family: monospace , serif;"><span style="font-size: 16.128px;">"numbers"</span></span><span style="font-size: 17.92px;">. The tool? Or the pen and paper?</span></div>
<div style="background-color: #f8fff8; box-sizing: border-box; color: #29323c; font-family: FrescoPlusNormal, Georgia, serif; font-size: 17.92px; margin-bottom: 0.8em;">
These so called "spreadsheets" <span style="font-size: 17.92px;">are like the little Dutch boy sticking his fingers in the dike. Every time there’s a new kind of bug, we add a feature to prevent that kind of bug. And so these tools accumulate more and more fingers in holes in dikes. The problem is, eventually you run out of fingers and toes!</span></div>
<div style="background-color: #f8fff8; box-sizing: border-box; color: #29323c; font-family: FrescoPlusNormal, Georgia, serif; font-size: 17.92px; margin-bottom: 0.8em;">
<em style="box-sizing: border-box; font-family: FrescoSansPlusPro-Normal-Italic; font-size: 1.1em;">This is the wrong path!</em></div>
<div style="background-color: #f8fff8; box-sizing: border-box; color: #29323c; font-family: FrescoPlusNormal, Georgia, serif; font-size: 17.92px; margin-bottom: 0.8em;">
Ask yourself why we are trying to plug defects with tools. The answer ought to be obvious. We are trying to plug these defects because these defects happen too often.</div>
<div style="background-color: #f8fff8; box-sizing: border-box; color: #29323c; margin-bottom: 0.8em;">
<span style="font-family: "frescoplusnormal" , "georgia" , serif; font-size: 17.92px;">Now, ask yourself why these defects happen too often. If your answer is that our tools don’t prevent them, then I strongly suggest that you quit your job and never think about use a spreadsheet again; because errors are <em style="box-sizing: border-box; font-family: FrescoSansPlusPro-Normal-Italic; font-size: 1.1em;">never</em> the fault of our tools. Defects are the fault of <em style="box-sizing: border-box; font-family: FrescoSansPlusPro-Normal-Italic; font-size: 1.1em;">users</em>. It is </span><span style="font-family: FrescoSansPlusPro-Normal-Italic;"><span style="font-size: 19.712px;"><i>users </i></span></span><span style="font-family: "frescoplusnormal" , "georgia" , serif;"><span style="font-size: 17.92px;">who create defects – not spreadsheets.</span></span></div>
<div style="background-color: #f8fff8; box-sizing: border-box; color: #29323c; font-family: FrescoPlusNormal, Georgia, serif; font-size: 17.92px; margin-bottom: 0.8em;">
And what is it that programmers are supposed to do to prevent defects? I’ll give you one guess. Here are some hints. It’s a verb. It starts with a “T”. Yeah. You got it.<em style="box-sizing: border-box; font-family: FrescoSansPlusPro-Normal-Italic; font-size: 1.1em;">TEST!</em></div>
<div style="background-color: #f8fff8; box-sizing: border-box; color: #29323c; font-family: FrescoPlusNormal, Georgia, serif; font-size: 17.92px; margin-bottom: 0.8em;">
You <em style="box-sizing: border-box; font-family: FrescoSansPlusPro-Normal-Italic; font-size: 1.1em;">test</em> every number is indeed a number. You <em style="box-sizing: border-box; font-family: FrescoSansPlusPro-Normal-Italic; font-size: 1.1em;">test</em> that your formulas refer to actual elements; not empty cells. You <em style="box-sizing: border-box; font-family: FrescoSansPlusPro-Normal-Italic; font-size: 1.1em;">test</em> that you've recalculated everything!</div>
<div style="background-color: #f8fff8; box-sizing: border-box; color: #29323c; font-family: FrescoPlusNormal, Georgia, serif; font-size: 17.92px; margin-bottom: 0.8em;">
Why are these spreadsheets adopting all these features?</div>
<div style="background-color: #f8fff8; box-sizing: border-box; color: #29323c; font-family: FrescoPlusNormal, Georgia, serif; font-size: 17.92px; margin-bottom: 0.8em;">
<span style="font-size: 17.92px;">We now have spreadsheets that</span><span style="font-size: 17.92px;"> </span><em style="box-sizing: border-box; font-family: FrescoSansPlusPro-Normal-Italic; font-size: 1.1em;">force</em><span style="font-size: 17.92px;"> </span><span style="font-size: 17.92px;">us to adorn every function, all the way up the dependent cells, with</span><span style="font-size: 17.92px;"> </span><code class="highlighter-rouge" style="box-sizing: border-box; font-family: monospace, serif; font-size: 0.9em; line-height: normal;">number</code><span style="font-size: 17.92px;">. We now have spreadsheets that are so constraining, and so over-specified, that you have to specify all the elements that they refer to!</span></div>
<div style="background-color: #f8fff8; box-sizing: border-box; color: #29323c; font-family: FrescoPlusNormal, Georgia, serif; font-size: 17.92px; margin-bottom: 0.8em;">
<span style="font-size: 17.92px;">All these constraints, that these spreadsheets are imposing, presume that the user has perfect knowledge of the system;</span><span style="font-size: 17.92px;"> </span><em style="box-sizing: border-box; font-family: FrescoSansPlusPro-Normal-Italic; font-size: 1.1em;">before the system is written</em><span style="font-size: 17.92px;">. They presume that you</span><span style="font-size: 17.92px;"> </span><em style="box-sizing: border-box; font-family: FrescoSansPlusPro-Normal-Italic; font-size: 1.1em;">know</em><span style="font-size: 17.92px;"> </span><span style="font-size: 17.92px;">which number is a number. They presume you <i>know </i>to not to mix different units. They presume you <i>know </i>which input should link to which output. They presume you <i>know </i>what units a result will come back in.</span></div>
<div style="background-color: #f8fff8; box-sizing: border-box; color: #29323c; font-family: FrescoPlusNormal, Georgia, serif; font-size: 17.92px; margin-bottom: 0.8em;">
And because of all this presumption, they <em style="box-sizing: border-box; font-family: FrescoSansPlusPro-Normal-Italic; font-size: 1.1em;">punish</em> you when you are wrong. </div>
<div style="background-color: #f8fff8; box-sizing: border-box; color: #29323c; font-family: FrescoPlusNormal, Georgia, serif; font-size: 17.92px; margin-bottom: 0.8em;">
And how do you avoid being punished? There are two ways. One that works; and one that doesn’t. The one that doesn’t work is to design everything up front before starting. The one that <em style="box-sizing: border-box; font-family: FrescoSansPlusPro-Normal-Italic; font-size: 1.1em;">does</em> avoid the punishment is to <em style="box-sizing: border-box; font-family: FrescoSansPlusPro-Normal-Italic; font-size: 1.1em;">override all the safeties</em>.</div>
<div style="background-color: #f8fff8; box-sizing: border-box; color: #29323c; font-family: FrescoPlusNormal, Georgia, serif; font-size: 17.92px; margin-bottom: 0.8em;">
</div>
<div style="background-color: #f8fff8; box-sizing: border-box; color: #29323c; font-family: FrescoPlusNormal, Georgia, serif; font-size: 17.92px; margin-bottom: 0.8em;">
And so, you write everything on paper and you leave these so called "spreadsheets" alone.</div>
<div style="background-color: #f8fff8; box-sizing: border-box; color: #29323c; font-family: FrescoPlusNormal, Georgia, serif; font-size: 17.92px; margin-bottom: 0.8em;">
<span style="font-size: 17.92px;">Why did the nuclear plant at Chernobyl catch fire, melt down, destroy a small city, and leave a large area uninhabitable? </span><em style="box-sizing: border-box; font-family: FrescoSansPlusPro-Normal-Italic; font-size: 1.1em;">They overrode all the safeties.</em><span style="font-size: 17.92px;"> So don’t depend on safeties to prevent catastrophes. Instead, you’d better get used to writing lots and lots of tests, no matter what spreadsheet you are using!</span></div>
<div style="background-color: #f8fff8; box-sizing: border-box; color: #29323c; font-family: FrescoPlusNormal, Georgia, serif; font-size: 17.92px; margin-bottom: 0.8em;">
--</div>
<div style="background-color: #f8fff8; box-sizing: border-box; color: #29323c; font-family: FrescoPlusNormal, Georgia, serif; font-size: 17.92px; margin-bottom: 0.8em;">
Of course, this is just early morning unfunny parody of an article by Bob Martin, <a href="http://blog.cleancoder.com/uncle-bob/2017/01/11/TheDarkPath.html">The Dark Path</a>.</div>
<div style="background-color: #f8fff8; box-sizing: border-box; color: #29323c; font-family: FrescoPlusNormal, Georgia, serif; font-size: 17.92px; margin-bottom: 0.8em;">
I strong disagree with the sentiment expressed in the article. Types are a tool that help you write code safely. Tests are a tool that help you write code safely. <span style="font-size: 17.92px;">Neither replaces the other. </span></div>
<div style="background-color: #f8fff8; box-sizing: border-box; color: #29323c; font-family: FrescoPlusNormal, Georgia, serif; font-size: 17.92px; margin-bottom: 0.8em;">
<span style="font-size: 17.92px;">To suggest that we should abandon static-typing is wrong. If I change something returning type A to returning type B I can see how that might mean I have to change a lot of my code base. That doesn't mean static typing is bad, it could mean anything! </span></div>
<div style="background-color: #f8fff8; box-sizing: border-box; margin-bottom: 0.8em;">
</div>
<ul>
<li><span style="color: #29323c; font-family: "frescoplusnormal" , "georgia" , serif;"><span style="font-size: 17.92px;">Maybe the design sucks, why does so much of the code know about type A?</span></span></li>
<li><span style="color: #29323c; font-family: "frescoplusnormal" , "georgia" , serif;"><span style="font-size: 17.92px;">Maybe it's a good thing - you didn't know everything upfront, an assumption has changed, so you should change the code?</span></span></li>
<li><span style="color: #29323c; font-family: "frescoplusnormal" , "georgia" , serif;"><span style="font-size: 17.92px;">Maybe you want to experiment? We should find a way to <a href="https://downloads.haskell.org/~ghc/7.8.2/docs/html/users_guide/defer-type-errors.html">defer type errors</a> to runtime? </span></span></li>
<li><span style="color: #29323c; font-family: "frescoplusnormal" , "georgia" , serif;"><span style="font-size: 17.92px;">Maybe we should invest in tooling to make this problem more tractable? (<a href="https://bitbucket.org/jlahoda/jackpot30/wiki/Home">Jackpot</a>!?)</span></span></li>
</ul>
<div>
<span style="color: #29323c; font-family: "frescoplusnormal" , "georgia" , serif;"><span style="font-size: 17.92px;"><br /></span></span></div>
<div>
<span style="color: #29323c; font-family: "frescoplusnormal" , "georgia" , serif;"><span style="font-size: 17.92px;">We, as software engineers, should be actively looking to advance the state of the art. We should be building tools to support our ways of working, not rallying against those that do.</span></span></div>
<br />
<div style="background-color: #f8fff8; box-sizing: border-box; color: #29323c; font-family: FrescoPlusNormal, Georgia, serif; font-size: 17.92px; margin-bottom: 0.8em;">
<br /></div>
<div style="background-color: #f8fff8; box-sizing: border-box; color: #29323c; font-family: FrescoPlusNormal, Georgia, serif; font-size: 17.92px; margin-bottom: 0.8em;">
<span style="font-size: 17.92px;"><br /></span></div>
Jeffhttp://www.blogger.com/profile/08195722595923882332noreply@blogger.comtag:blogger.com,1999:blog-5743983044224833668.post-59728557868915743702017-01-11T01:10:00.000-08:002017-01-11T01:10:25.514-08:00Radical Focus - OKRs<a href="https://www.amazon.co.uk/Radical-Focus-Achieving-Important-Objectives-ebook/dp/B01BFKJA0Y">Radical Focus</a> explains the OKR process with a business narrative (a startup building the StarBucks of Coffee). The story shows how focusing on OKRs supports the team taking the tough decisions (e.g. stopping promising work if it doesn't support an objective) and spending their limited runway of activity on the right tasks.<br />
<br />
<br />
It's a short read, with some good points but having read the book and watched "The Executioners Tale" (<a href="https://vimeo.com/86392023">https://vimeo.com/86392023</a>) I'd pick the video next time!<br />
<br />
<h2>
Key takeaways</h2>
<ul>
<li>OKRs are great for setting goals, BUT without a system to achieve them you are as likely to fail as with anything else.</li>
<li>A mission keeps you on the rails - the OKRs provide focus and milestones.</li>
<li>Set only one OKR for the company - it's about <i>focus</i></li>
<li>Timescales should be about 3 months - too long and it's too far away to have impact, and too short and it's not bold enough</li>
<li>Objectives are <i>inspirational</i> not metrics</li>
<li>Repeat the message. The goal needs to be in front of everyones mind and tied to all activities. - "When you are tired of saying it, people are starting to hear it" (Jeff Weiner, CEO of LinkedIn")</li>
<li>Heuristic for KRs - one usage metric, a revenue metric and a satisfaction metric.</li>
<li>A good key result should be a bit scary - a 50/50 confidence you can make it is about right.</li>
<li>Use health metrics to identify areas to protect as you meet the goals (what can't you screw up?)</li>
<li>Use the four-square template (<a href="http://www.tightship.io/assets/weekly-meeting.jpg">http://www.tightship.io/assets/weekly-meeting.jpg</a>) to keep focus</li>
<li>Reinforce the message at the beginning and end of the week (Monday discuss/challenge, Friday demonstrate/celebrate)</li>
</ul>
Jeffhttp://www.blogger.com/profile/08195722595923882332noreply@blogger.comtag:blogger.com,1999:blog-5743983044224833668.post-77704239930247161502017-01-03T08:58:00.003-08:002017-01-03T08:58:23.396-08:00The 3X ApproachI watched a <a href="https://webinar.andela.com/thank-you-kent-beck-broadcast/">webinar</a> recently about Kent Beck's characterization of product development as a triathlon and thought I'd summarize the notes here!<br />
<br />
Kent presents product development as a three-phased approach (eXplore, eXpand, eXtract). This really closely models the product life cycle (<a href="https://hbr.org/1965/11/exploit-the-product-life-cycle">from HBR</a>).<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
</div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg1lZlNMPFNUAGFscfAcUoNfD_FHpOhW7a0m4KQpGxYQN5pMB9KnEF0i4I2S_6S72ufL9eItz_2FJ6ubA8FiKZimU5-nM_d3fzRnu8oTRpYV4SnUWNaOC2eqZt4M0vP3sRtl-G70Aw6mDQ/s1600/65608_A.gif" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg1lZlNMPFNUAGFscfAcUoNfD_FHpOhW7a0m4KQpGxYQN5pMB9KnEF0i4I2S_6S72ufL9eItz_2FJ6ubA8FiKZimU5-nM_d3fzRnu8oTRpYV4SnUWNaOC2eqZt4M0vP3sRtl-G70Aw6mDQ/s1600/65608_A.gif" /></a></div>
<br />
Kent's insight is that you should act differently depending on the phase you are in.<br />
<br />
In the Explore phase (stage #1 above) you can't predict the future. You've no idea how long finding market fit is going to take. It's a high risk activity. The main driver of success is the rate at which you can run experiments and learn quickly. At this stage software quality is irrelevant - the half-life of the code is short. You want a cross-functional, loosely co-ordinated team to deliver this phase.<br />
<br />
The next stage is Expand - this is rapid growth, equivalent to the B round of venture capital. You have validated the market, you know what to do and how to do it. Time to scale, develop features and get it in the hands of users and build those feedback cycles.<br />
<br />
Finally, you are at Extract. This is where economies of scale shine. Work here is predictable - adding a feature will result in a known about of revenue. You can estimate things well because you've done it plenty of times before. Quality is really important - cutting corners now will cost you because you'll be stuck with it for ever.<br />
<br />
Organizations can get tuned to a particular way of thinking and that can constrain. For example, it's easiest to get tuned into the last phase Extract (see <a href="https://en.wikipedia.org/wiki/The_Innovator's_Dilemma">Innovator's Dilemma</a>)<br />
<br />
Beck states that this is one of the things XP got wrong- there is no one-size fits all methodology. The 3X model says it's all about where you are on the <a href="http://www.galsinsights.com/the-innovation-s-curve/">s-curve</a>.Jeffhttp://www.blogger.com/profile/08195722595923882332noreply@blogger.comtag:blogger.com,1999:blog-5743983044224833668.post-54353945329993077782016-08-31T06:57:00.000-07:002016-08-31T06:57:04.413-07:00Design Pattern Haikus<h2>
<a href="https://en.wikipedia.org/wiki/Singleton_pattern">Singletons</a></h2>
<div style="text-align: center;">
<i>a singleton is</i></div>
<div style="text-align: center;">
<i>a global variable</i></div>
<div style="text-align: center;">
<i>sounds a bit better </i></div>
<div>
<i><br /></i></div>
<div>
A singleton is just a fancy name for a global variable. I like to think of a singleton as a way of warping to another point in space/time, changing the space time continuum and then heading back. You've seen the sci-fi films where this happens, right? It has similar effects on your code, making it difficult to understand what the hell is happening.</div>
<div>
<br /></div>
<h2>
<a href="https://en.wikipedia.org/wiki/Interpreter_pattern">Interpreter</a></h2>
<div style="text-align: center;">
<i>interpreter is</i></div>
<div style="text-align: center;">
<i>domain specific language</i></div>
<div style="text-align: center;">
<i>that is all it is</i></div>
<div>
<i><br /></i></div>
<div>
Well, there's not much more to say is there? You want your code to be readable in the language of your domain. One way to do this is to make the code look more like the language of your domain. A <a href="https://en.wikipedia.org/wiki/Domain-specific_language">domain-specific language</a> is one way to accomplish this.</div>
<div>
<i><br /></i></div>
<h2>
<a href="https://en.wikipedia.org/wiki/Visitor_pattern">Visitor</a></h2>
<div style="text-align: center;">
<i>multiple dispatch</i></div>
<div style="text-align: center;">
<i>that is what you really wanted</i></div>
<div style="text-align: center;">
<i>visitor will do</i></div>
<div style="text-align: center;">
<i><br /></i></div>
<div style="text-align: left;">
You've got a function that wants to do different things depending on the types of the arguments. You've got a language that only allows you to do one different thing via polymorphism. You don't want type-switching cos someone told you that was bad.</div>
<div style="text-align: left;">
<br /></div>
<div style="text-align: left;">
Assuming you've got those prerequisites then go for the visitor pattern. Alternatively, look at <a href="https://en.wikipedia.org/wiki/Multiple_dispatch">multiple dispatch</a> and see there's no problem there at all really.</div>
<div style="text-align: left;">
<br /></div>
<h2 style="text-align: left;">
<a href="https://en.wikipedia.org/wiki/Strategy_pattern">Strategy </a></h2>
<div style="text-align: center;">
<i>functions compose well</i></div>
<div style="text-align: center;">
<i>classes do not compose at all</i></div>
<div style="text-align: center;">
<i>love strategy pattern</i></div>
<div style="text-align: center;">
<i><br /></i></div>
<div style="text-align: left;">
You want to break apart a set of work into small discrete components (let's call them "objects"). The strategy pattern allows you to plug these together to solve a problem.</div>
<div style="text-align: left;">
<br /></div>
<div style="text-align: left;">
Or... You want to break apart an algorithm into discrete functions (let's call them functions). Functions glue together.</div>
<div>
<br /></div>
Jeffhttp://www.blogger.com/profile/08195722595923882332noreply@blogger.comtag:blogger.com,1999:blog-5743983044224833668.post-9733740268481032982016-05-11T22:54:00.004-07:002016-05-11T22:54:56.940-07:00Professionalism in Software EngineeringYesterday, I attended a talk by <a href="https://sites.google.com/site/unclebobconsultingllc/">Bob Martin</a> on Professionalism in Code hosted by <a href="https://twitter.com/sweast">Software East at </a><a href="http://www.red-gate.com/">Redgate</a>. I also spent considerable time beforehand working out whether it'd be unfair to introduce Bob as the "Donald Trump of Computer Programming". Sanity prevailed and I didn't. But I just wrote it up there, didn't I? D'oh.<br />
<br />
Is the software engineering industry professional? As software engineering professionals we should have skill and good judgement. Does the software engineering industry have that?<br />
<br />
Bob gave the <a href="http://www.sec.gov/litigation/admin/2013/34-70694.pdf">famous example [PDF]</a> of Knight Capital and <a href="https://en.wikipedia.org/wiki/Volkswagen_emissions_scandal">Volkswagen</a>, but there's many more (<a href="http://courses.cs.vt.edu/professionalism/Therac_25/Therac_1.html">Therac-25</a>, <a href="http://www.defensenews.com/story/defense/air-space/2016/01/22/dod-weapons-tester-concerned-f-35-software-development/79191890/">the F35</a> and <a href="http://www.csus.edu/indiv/v/velianitis/161/ChaosReport.pdf">The Chaos Report [PDF]</a>). You could argue that the software industry is in meltdown and developer incompetence / poor judgement has cost the industry billions. I think you'd have a pretty convincing case! Or would you? There was no mention the other side - the tremendous advantages our haphazard industry has made to almost the whole planet (the Internet, mobile phones, communication).<br />
<br />
Bob painted the nightmare scenario - regulation. Imagine some time from now, some bug somewhere (a missing ; even) results in a number of deaths. The nuclear reactor blows, the self-driving cars go made on a leap year and start running people over, the planes turn upside down when crossing the equator (etc). The natural result of this is the Government blames us (software developers) and starts to put some regulations in place. Again, this feels believable-ish.<br />
<br />
But why hasn't it happened yet? Well, safety-critical systems are pretty regulated. See this <a href="https://www.faa.gov/regulations_policies/handbooks_manuals/aviation/risk_management/ss_handbook/media/Chap10_1200.pdf">lump</a> [PDF trigger warning] from the FAA about how they do things. I'm not going to argue it's perfect, but it's demonstrably good enough to stop null pointers making things fall out the sky regularly.<br />
<br />
So what should we do to prevent this threat of regulation? Well, we should:<br />
<ul>
<li>Not ship shit!</li>
<li>Give reasonable estimates!</li>
<li>QA should find no bugs</li>
<li>Software should get better, not worse</li>
<li>Invest 20 hours per week in personal development</li>
</ul>
Most of this stuff is easy to agree. Of course we shouldn't ship shit - are you insane? Of course we should strive to write bug free code. If you want to explore these ideas more, Bob's book (<a href="http://www.amazon.com/Clean-Coder-Conduct-Professional-Programmers/dp/0137081073">The Clean Coder</a>) covers the topics in much greater detail. It was a great talk and got the audience thinking.<br />
<br />
Are these the right things to make the industry professional? Maybe. Maybe as an industry we should look at other areas:<br />
<ul>
<li>Should we write code in unsafe languages like C for safety critical systems?</li>
<li>Should we put JavaScript on a plane?</li>
<li>Is that shared mutable state acceptable in my car?</li>
<li>Languages with null are a <a href="http://www.infoq.com/presentations/Null-References-The-Billion-Dollar-Mistake-Tony-Hoare">billion dollar mistake</a> - outlaw them!</li>
<li>Should we use <a href="http://blog.cleancoder.com/uncle-bob/2016/05/01/TypeWars.html">test-driven development as justification for using weak languages</a>?</li>
</ul>
<div>
There's a huge space for software engineers to explore about building professional quality software. We're not explored much yet. I'm certainly no historian, but I'd imagine professions like Medicine, Law and Engineering took more than 60 or so years to establish what good looked like.</div>
<div>
<br /></div>
<div>
The deliciously chaotic world of software engineering is going to continue for a while yet!</div>
<div>
<br /></div>
<div>
<br /></div>
Jeffhttp://www.blogger.com/profile/08195722595923882332noreply@blogger.comtag:blogger.com,1999:blog-5743983044224833668.post-47112837135757134282016-04-04T09:23:00.000-07:002016-04-04T09:23:00.185-07:00Azure Platform Services<a href="http://www.fatvat.co.uk/2016/04/whats-what-in-azure.html">Last time</a> around we compiled a large compendium of links detailing Azure Infrastructure services. In this post, we'll compile an even larger set of links detailing Azure's Platform Services (<a href="https://en.wikipedia.org/wiki/Platform_as_a_service">PaaS</a> rather than IaaS).<br />
<br />
The breadth of services that Azure offers is pretty overwhelming, so take a deep breath :)<br />
<h2>
Cloud Compute</h2>
<div>
<a href="https://azure.microsoft.com/en-us/services/cloud-services/">Azure Cloud Services</a> allows you to create a compute service (ASP.NET, Python, node and PHP are all supported). This might be a worker role (e.g. a background tasks) or a web role (e.g. simple requests to display data). Cloud Services gives you the ability to scale these horizontally as needed.</div>
<div>
<br /></div>
<div>
<a href="https://azure.microsoft.com/en-us/services/batch/">Azure Batch</a> runs your large scale parallel tasks and big batch processing jobs. Azure scales these as needed.<br />
<br />
<a href="https://azure.microsoft.com/en-us/services/service-fabric/">Azure Service Fabric</a> is an orchestration layer for micro-service based deployments. It was borne out of internal use at Microsoft used to develop Azure itself.<br />
<br />
<a href="https://www.remoteapp.windowsazure.com/en/">Azure RemoteApp</a> is a bridging technology (similar to Citrix type things) allowing you to access your application anywhere.</div>
<div>
<br /></div>
<div>
<h2>
Web and Mobile</h2>
</div>
<div>
<a href="https://azure.microsoft.com/en-us/services/app-service/web/">Web App Service</a> allows you to deploy web applications in languages like C#, node.js and Python. It's now part of the more general Azure App Service.</div>
<div>
<br /></div>
<div>
<a href="https://azure.microsoft.com/en-us/services/app-service/api/">API App Service</a> allows you to deploy secured API services and generate appropriate clients to access them.</div>
<div>
<br /></div>
<div>
<a href="https://azure.microsoft.com/en-us/services/api-management/">API Management Service</a> lets you "take any API and publish a service in minutes". More specifically you get monitoring, RESTful and JSON-ful support and the ability to combine multiple back-ends into a single API endpoint.</div>
<div>
<br /></div>
<div>
<a href="https://azure.microsoft.com/en-us/services/app-service/mobile/">Mobile App Service</a> gives you an API specifically for mobiles, including support for off-line sync.</div>
<div>
<br /></div>
<div>
<a href="https://azure.microsoft.com/en-us/services/app-service/logic/">Logic App Service</a> lets you integrate business processes and workflows visually. It's goal is to make it easy for you to join your data from on-premise to cloud-based workflows.</div>
<div>
<br /></div>
<div>
<a href="https://azure.microsoft.com/en-us/services/app-service/logic/">Notification Hubs</a> provide scalable push-notifications to all major platforms (including iOS and Android).</div>
<div>
<br /></div>
<h2>
Data</h2>
<div>
<a href="https://azure.microsoft.com/en-us/services/sql-database/">SQL Database</a> provides a fully managed PaaS version of SQL Server with advanced features such as an index advisor (monitoring your workload to see access patterns that would benefit from an index).<br />
<br />
<a href="https://azure.microsoft.com/en-us/services/sql-data-warehouse/">SQL Data Warehouse</a> is a <a href="https://en.wikipedia.org/wiki/Data_warehouse">data warehouse</a> that can scale to huge volumes of data (pricing is based separately on Compute and Storage use).<br />
<br />
<a href="https://azure.microsoft.com/en-us/services/cache/">Redis Cache</a> is the PaaS version of <a href="http://redis.io/">Redis</a>, an in-memory data structure store.<br />
<br />
<a href="https://azure.microsoft.com/en-us/services/documentdb/">DocumentDB</a> is a store for JSON documents. As of Build 2016, it was announced that there is a <a href="https://www.mongodb.org/">MongoDB</a> compatibility layer (see <a href="https://azure.microsoft.com/en-us/documentation/articles/documentdb-protocol-mongodb/">here</a>).<br />
<br />
<a href="https://azure.microsoft.com/en-us/services/search/">Azure Search</a> provides a fully managed search service (with <a href="https://azure.microsoft.com/en-us/updates/general-availability-lucene-query-syntax-in-azure-search/">Lucene query compatibility</a>).</div>
<div>
<br />
<a href="https://azure.microsoft.com/en-us/documentation/articles/storage-dotnet-how-to-use-tables/">Azure Table Storage</a> provides you with a key-value store aimed at large schema-less documents.<br />
<br />
<h2>
Analytics and IoT</h2>
</div>
<div>
<a href="https://azure.microsoft.com/en-us/services/hdinsight/">HDInsight</a> is a managed Apache Hadoop (map/reduce), Spark, R, HBase and Storm service made "easy".</div>
<div>
<br /></div>
<div>
<a href="https://azure.microsoft.com/en-us/services/machine-learning/">Azure Machine Learning</a> is a set of machine-learning API's, allowing you to applying advanced analytics to a wide source of data (pictures, people, text etc.). As of Build 2016, this seems to be in the process of being rebadged "cognitive services".</div>
<div>
<br /></div>
<div>
<a href="https://azure.microsoft.com/en-us/services/stream-analytics/">Azure Stream Analytics</a> gives you the ability to do real time processing of streaming data from huge numbers of sources.</div>
<div>
<br /></div>
<div>
<a href="https://azure.microsoft.com/en-us/services/data-factory/">Azure Data Factory</a> is a set of data orchestration API's, allowing you to mangle data from different sources together (with tools for data lineage, ETL and so on).</div>
<div>
<br /></div>
<div>
<a href="https://azure.microsoft.com/en-us/services/event-hubs/">Azure Event Hubs</a> is a scalable pub-sub service for aggregating events from many sources.</div>
<div>
<br /></div>
<div>
<a href="https://azure.microsoft.com/en-us/services/mobile-engagement/">Mobile Engagement</a> is a set of API's for monitoring and understanding app usage on mobile devices.</div>
Jeffhttp://www.blogger.com/profile/08195722595923882332noreply@blogger.comtag:blogger.com,1999:blog-5743983044224833668.post-28139533212945156042016-04-01T02:13:00.003-07:002016-04-01T02:22:58.853-07:00Azure Infrastructure Services<span id="docs-internal-guid-102e9e4f-d103-81be-4423-0f53416b71ab"></span><br />
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<a href="https://azure.microsoft.com/en-us/overview/what-is-azure/" style="text-decoration: none;"><span style="background-color: transparent; color: #1155cc; font-family: "arial"; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: underline; vertical-align: baseline; white-space: pre-wrap;">Azure</span></a><span style="background-color: transparent; color: black; font-family: "arial"; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> “is a growing collection of integrated cloud services for moving faster, achieving more, and saving money”. Well, that’s the marketing lingo, but what are the actual services available?</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: "arial"; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><br /></span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: "arial"; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><img height="356" src="https://msdnshared.blob.core.windows.net/media/MSDNBlogsFS/prod.evol.blogs.msdn.com/CommunityServer.Blogs.Components.WeblogFiles/00/00/01/62/62/4846.azure.jpg" width="640" /></span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: "arial"; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><br /></span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: "arial"; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">The diagram above, from <a href="https://blogs.msdn.microsoft.com/niallsblog/2015/05/18/running-a-high-volume-website-on-azure-infrastructure-services/">here</a>, is the best example I've seen of capturing everything that Azure is and the services that it offers.</span><br />
<span style="background-color: transparent; color: black; font-family: "arial"; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><br /></span>
<span style="background-color: transparent; color: black; font-family: "arial"; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">Let's start with the infrastructure services.</span></div>
<div>
<span style="font-family: "arial"; font-size: 14.6667px; line-height: 20.24px; white-space: pre-wrap;"><br /></span></div>
<div>
<span style="font-family: "arial"; font-size: 14.6667px; line-height: 20.24px; white-space: pre-wrap;">Infrastructure services (<a href="https://en.wikipedia.org/wiki/Cloud_computing#Infrastructure_as_a_service_.28IaaS.29">IaaS</a>) abstract away physical machines to services that can be molded via code rather than plugging in cables.</span></div>
<div>
<span style="font-family: "arial"; font-size: 14.6667px; line-height: 20.24px; white-space: pre-wrap;"><br /></span></div>
<div>
<span style="font-family: "arial";"><span style="font-size: 14.6667px; line-height: 20.24px; white-space: pre-wrap;">Under the banner of Compute, there are a couple of services. </span></span><span style="font-family: "arial"; font-size: 14.6667px; line-height: 20.24px; white-space: pre-wrap;">Azure </span><a href="https://azure.microsoft.com/en-us/services/virtual-machines/" style="font-family: Arial; font-size: 14.6667px; line-height: 20.24px; white-space: pre-wrap;">Virtual machines</a><span style="font-family: "arial"; font-size: 14.6667px; line-height: 20.24px; white-space: pre-wrap;"> let you deploy images in any way. They aren't just limited to Windows, support includes Linux, Oracle IBM and SAP. </span><a href="https://azure.microsoft.com/en-us/services/container-service/" style="font-family: Arial; font-size: 14.6667px; line-height: 20.24px; white-space: pre-wrap;">Azure Container Service</a><span style="font-family: "arial"; font-size: 14.6667px; line-height: 20.24px; white-space: pre-wrap;"> allows you to deploy containers to Azure. This is heavily open-source friendly and allows you to use </span><a href="http://mesos.apache.org/" style="font-family: Arial; font-size: 14.6667px; line-height: 20.24px; white-space: pre-wrap;">Apache Mesos</a><span style="font-family: "arial"; font-size: 14.6667px; line-height: 20.24px; white-space: pre-wrap;"> or </span><a href="https://docs.docker.com/swarm/" style="font-family: Arial; font-size: 14.6667px; line-height: 20.24px; white-space: pre-wrap;">Docker Swarm</a><span style="font-family: "arial"; font-size: 14.6667px; line-height: 20.24px; white-space: pre-wrap;"> to orchestrate.</span></div>
<div>
<span style="font-family: "arial"; font-size: 14.6667px; line-height: 20.24px; white-space: pre-wrap;"><br /></span></div>
<div>
<span style="font-family: "arial"; font-size: 14.6667px; line-height: 20.24px; white-space: pre-wrap;">There's many options for file storage as a service. </span><a href="https://azure.microsoft.com/en-us/documentation/articles/storage-dotnet-how-to-use-blobs/#what-is-blob-storage" style="font-family: Arial; font-size: 14.6667px; line-height: 20.24px; white-space: pre-wrap;">Azure Blob Storage</a><span style="font-family: "arial"; font-size: 14.6667px; line-height: 20.24px; white-space: pre-wrap;"> provides a service for storing large amounts of unstructured data that can be accessed via http(s). A blob account has multiple containers (think of these are folders or organizational units) and each container can </span><a href="https://msdn.microsoft.com/library/azure/ee691964.aspx" style="font-family: Arial; font-size: 14.6667px; line-height: 20.24px; white-space: pre-wrap;">lumps of data</a><span style="font-family: "arial"; font-size: 14.6667px; line-height: 20.24px; white-space: pre-wrap;"> (blocks, append-only, page blobs). </span><a href="https://azure.microsoft.com/en-us/services/storage/files/" style="font-family: Arial; font-size: 14.6667px; line-height: 20.24px; white-space: pre-wrap;">Azure Files</a><span style="font-family: "arial"; font-size: 14.6667px; line-height: 20.24px; white-space: pre-wrap;"> provides fully managed file shares using the standard SMB protocol. This allows you to migrate file shared-based applications to the cloud with no changes. </span><span style="font-family: "arial"; font-size: 14.6667px; line-height: 20.24px; white-space: pre-wrap;">Finally in the storage offerings there are a variety of low-latency and high throughput storages referred to as </span><a href="https://azure.microsoft.com/en-us/services/storage/premium-storage/" style="font-family: Arial; font-size: 14.6667px; line-height: 20.24px; white-space: pre-wrap;">premium storage</a><span style="font-family: "arial"; font-size: 14.6667px; line-height: 20.24px; white-space: pre-wrap;">. These are essentially pre-configured virtual machines with optimized technology (e.g. SSD) for storage. Options include hosting SQL Server, Oracle, MySQL, Redis and MongoDB.</span></div>
<div>
<span style="font-family: "arial"; font-size: 14.6667px; line-height: 20.24px; white-space: pre-wrap;"><br /></span></div>
<div>
<span style="font-family: "arial"; font-size: 14.6667px; line-height: 20.24px; white-space: pre-wrap;">There's a whole raft of networking services. <a href="https://azure.microsoft.com/en-us/services/virtual-network/">Azure Virtual Network</a> provides an environment to run your machines and applications where you can control the subnets, access control policies and more. <a href="https://azure.microsoft.com/en-us/documentation/articles/load-balancer-overview/">Azure Load Balancer</a> does exactly what it says on the tin - it's a Layer 4 load balancer that allows you to distribute incoming traffic. <a href="https://azure.microsoft.com/en-us/services/dns/">Azure DNS</a> is another <a href="https://en.wikipedia.org/wiki/Does_exactly_what_it_says_on_the_tin">Ronseal service</a>! <a href="https://azure.microsoft.com/en-us/services/expressroute/">ExpressRoute</a> lets you create private connections between your data center and Azure data centers (giving you up to 10 Gbps). <a href="https://azure.microsoft.com/en-us/services/traffic-manager/">Traffic Manager </a> is similar to load balancing, but with more flexibility around failover, A/B testing and combining Azure / on-prem systems. <a href="https://azure.microsoft.com/en-us/documentation/articles/vpn-gateway-about-vpngateways/">Azure VPN Gateways</a> is another virtual network manager, and <a href="https://azure.microsoft.com/en-us/services/application-gateway/">Application Gateway</a> is an application level load balancer. </span><br />
<span style="font-family: "arial"; font-size: 14.6667px; line-height: 20.24px; white-space: pre-wrap;"><br /></span>
<span style="font-family: "arial"; font-size: 14.6667px; line-height: 20.24px; white-space: pre-wrap;">Confused yet?</span></div>
<div>
<br /></div>
<div>
<span style="font-family: "arial"; font-size: 14.6667px; line-height: 20.24px; white-space: pre-wrap;"><br /></span></div>
<div>
<span style="font-family: "arial"; font-size: 14.6667px; line-height: 20.24px; white-space: pre-wrap;"><br /></span></div>
Jeffhttp://www.blogger.com/profile/08195722595923882332noreply@blogger.comtag:blogger.com,1999:blog-5743983044224833668.post-45186815036703702812016-01-18T03:26:00.003-08:002016-01-18T03:26:39.363-08:00Evolutionary Design Reading List<h1 dir="ltr" style="line-height: 1.38; margin-bottom: 6pt; margin-top: 20pt;">
<span style="font-family: Arial; font-size: 14.6667px; font-weight: 400; line-height: 1.38; white-space: pre-wrap;">Evolving a shared library with an API in flux is a tough problem, but there’s plenty of principles, practices and patterns around this.</span></h1>
<b id="docs-internal-guid-83391871-547b-566f-9ccc-2e266d6377b4" style="font-weight: normal;"><br /></b>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<a href="http://martinfowler.com/bliki/ParallelChange.html" style="text-decoration: none;"><span style="background-color: transparent; color: #1155cc; font-family: Arial; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: underline; vertical-align: baseline; white-space: pre-wrap;">Parallel Change</span></a><span style="background-color: transparent; color: black; font-family: Arial; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> - Parallel change, also known as expand and contract is a pattern to implement backward-incompatible changes to an interface in a safe manner by breaking the change into three distinct phases: expand, migrate, and contract.</span></div>
<b style="font-weight: normal;"><br /></b>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<a href="http://programmers.stackexchange.com/questions/12401/be-liberal-in-what-you-accept-or-not" style="text-decoration: none;"><span style="background-color: transparent; color: #1155cc; font-family: Arial; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: underline; vertical-align: baseline; white-space: pre-wrap;">Postel’s Principle</span></a><span style="background-color: transparent; color: black; font-family: Arial; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> - Be conservative in what you send; be liberal in what you accept.</span></div>
<b style="font-weight: normal;"><br /></b>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<a href="http://martinfowler.com/articles/refactoring-dependencies.html" style="text-decoration: none;"><span style="background-color: transparent; color: #1155cc; font-family: Arial; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: underline; vertical-align: baseline; white-space: pre-wrap;">Refactoring Module Dependencies</span></a><span style="background-color: transparent; color: black; font-family: Arial; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> - Some patterns for refactoring module dependencies</span></div>
<b style="font-weight: normal;"><br /></b>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<a href="http://butunclebob.com/ArticleS.UncleBob.PrinciplesOfOod" style="text-decoration: none;"><span style="background-color: transparent; color: #1155cc; font-family: Arial; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: underline; vertical-align: baseline; white-space: pre-wrap;">Package Management Principles</span></a><span style="background-color: transparent; color: black; font-family: Arial; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> - Principles of packages (REP, CCP, CRP, ADP, SDP, SAP). Think of these as a higher-level version of SOLID.</span></div>
<b style="font-weight: normal;"><br /></b>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<a href="http://www.martinfowler.com/bliki/StranglerApplication.html" style="text-decoration: none;"><span style="background-color: transparent; color: #1155cc; font-family: Arial; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: underline; vertical-align: baseline; white-space: pre-wrap;">Strangler Application</span></a><span style="background-color: transparent; color: black; font-family: Arial; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> - A metaphor describing growing a new system around the edges of old.</span></div>
<b style="font-weight: normal;"><br /></b>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<a href="http://www.martinfowler.com/bliki/AssetCapture.html" style="text-decoration: none;"><span style="background-color: transparent; color: #1155cc; font-family: Arial; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: underline; vertical-align: baseline; white-space: pre-wrap;">Asset Capture</span></a><span style="background-color: transparent; color: black; font-family: Arial; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> - A strategy for migrating between a strangler application and back again.</span></div>
<b style="font-weight: normal;"><br /></b>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<a href="https://www.cs.umd.edu/class/spring2003/cmsc838p/Design/criteria.pdf" style="text-decoration: none;"><span style="background-color: transparent; color: #1155cc; font-family: Arial; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: underline; vertical-align: baseline; white-space: pre-wrap;">On the Criteria to be used in Decomposing Systems into Modules</span></a><span style="background-color: transparent; color: black; font-family: Arial; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> - Parnas’ classic paper on modular systems (referenced by Tim in his recent talk).</span></div>
<b style="font-weight: normal;"><br /></b>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<a href="https://www.youtube.com/watch?v=NAF7HWW_eJs" style="text-decoration: none;"><span style="background-color: transparent; color: #1155cc; font-family: Arial; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: underline; vertical-align: baseline; white-space: pre-wrap;">Escape Integration Test Syrup</span></a><span style="background-color: transparent; color: black; font-family: Arial; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> - Talk from Agile on the Beach about testing and rapidly changing dependencies.</span></div>
<br />
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<a href="http://semver.org/" style="text-decoration: none;"><span style="background-color: transparent; color: #1155cc; font-family: Arial; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: underline; vertical-align: baseline; white-space: pre-wrap;">Semantic Versioning</span></a><span style="background-color: transparent; color: black; font-family: Arial; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> - For completeness!</span></div>
Jeffhttp://www.blogger.com/profile/08195722595923882332noreply@blogger.comtag:blogger.com,1999:blog-5743983044224833668.post-89640089760404392822015-11-09T22:56:00.002-08:002015-11-09T22:56:58.695-08:00The (very) Basics of R with the Game of Life<a href="https://www.blogger.com/is%20programming%20lang">R</a> is a programming language for statistical computing and graphics. It's also the language of choice amongst pirates. Arrr! R is increasingly important for big data analysis, and both <a href="https://www.oracle.com/engineered-systems/big-data-appliance/index.html">Oracle</a> and <a href="http://blogs.msdn.com/b/lukassteindl/archive/2015/11/06/sql-server-2016-ctp3-r-integration.aspx">Microsoft</a> have recently announced support for database analytics using R.<br />
<br />
So, how do you get started with R? Well, for the rest of this I'm going to assume that you already know how to program in a { } language like Java / C# and I'm going to cover the minimum amount possible to do something vaguely useful. The first step is to download the environment. You can get this from <a href="https://www.r-project.org/">here</a>. Once you've got something downloaded and installed you should be able to bring up a terminal and start R. I really like the built in demos. Bring up a list of them with <code>demo()</code> and type <code>demo(graphics)</code> to get an idea of the capabilities of R.<br />
<br />
These are the boring syntax bits:<br />
<ul><li>R is a case sensitive language</li>
<li>Comments start with <code>#</code> and run to the end of the line</li>
<li>Functions are called with parentheses e.g. f(x,y,z)</li>
</ul>The "standard library" of R is called the <a href="https://stat.ethz.ch/R-manual/R-devel/library/base/html/00Index.html">R Base Package</a>. When you bring up R, you bring up a <i>workspace</i>. A workspace is just what is in scope as any one time. You can examine the workspace by calling the <code>ls</code> function.<br />
<br />
<span style="font-family: "courier new" , "courier" , monospace;"> # Initially my workspace is empty</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> > ls()</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> character(0)</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"><br />
</span> <span style="font-family: "courier new" , "courier" , monospace;"> # Now I set a value and lo-and-behold, it's in my workspace</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> > x = "banana"</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> > ls()</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> [1] "x"</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"><br />
</span> <span style="font-family: "courier new" , "courier" , monospace;"> # I can save my workspace with</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> > save(file="~/foo.RData");</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"><br />
</span> <span style="font-family: "courier new" , "courier" , monospace;"> # I can load my workspace with</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> > load("~/foo.RData");</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"><br />
</span> <span style="font-family: "courier new" , "courier" , monospace;"> # I can remove elements from the workspace with rm</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> > rm(x)</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> > ls()</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> character(0)</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"><br />
</span> <span style="font-family: "courier new" , "courier" , monospace;"> # I can nuke my workspace with rm(list=ls())</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> > x = 'banana'</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> > rm(list=ls())</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> > ls()</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> character(0) </span><br />
<div><br />
</div>We've seen above that R supports string data, but it also supports vectors, lists, arrays, matrices, tables and data frames. To define a vector you use the <code>c</code> function. For example:<br />
<br />
<pre></pre><br />
<span style="font-family: "courier new" , "courier" , monospace;"> > x = c(1,2,3,4,5)</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> [1] 1 2 3 4 5</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"><br />
</span> <span style="font-family: "courier new" , "courier" , monospace;"> > length(x)</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> [1] 5</span><br />
<br />
Remember everything in a vector must be of the same type. Elements are co-erced to the same type, so <code>c(1,'1',TRUE)</code> results in a vector of string types. Indexing into vectors starts at 1 (<a href="http://www.cs.utexas.edu/users/EWD/transcriptions/EWD08xx/EWD831.html">not zero</a>). You can use Python style list selection:<br />
<br />
<pre></pre><span style="font-family: "courier new" , "courier" , monospace;"> > x = c(1,2,3,4,5,6,7,8,9,10)</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> > x[7:10] # select 7 thru 10</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> [1] 7 8 9 10</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> > x[-(1:3)] # - does exclusion</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> [1] 4 5 6 7 8 9 10</span><br />
<br />
To define a list, you use, ahem, <code>list</code>. Items in list are named components (see the rules of <a href="https://cran.r-project.org/doc/FAQ/R-FAQ.html#What-are-valid-names_003f">variable naming</a>).<br />
<br />
<span style="font-family: "courier new" , "courier" , monospace;"> > y = list(name="Fred", lastname="Bloggs", age=21)</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> > y</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> $name</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> [1] "Fred"</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> $lastname</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> [1] "Bloggs"</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> $age</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> [1] 21</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> > y$name # access the name property</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> [1] "Fred"</span><br />
<br />
Finally, let's look at matrices. You construct them with <code>matrix</code> and pass in a vector to construct from, together with the size.<br />
<pre></pre><br />
<span style="font-family: "courier new" , "courier" , monospace;"> > m = matrix( c(1,2,3,4,5,6,7,8,9), nrow=3, ncol=3)</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> > m</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> [,1] [,2] [,3]</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> [1,] 1 4 7</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> [2,] 2 5 8</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> [3,] 3 6 9</span><br />
<br />
OK, that should be enough boring information out the way to let me write a function for the <a href="https://en.wikipedia.org/wiki/Conway%27s_Game_of_Life">Game of Life</a>. All I want to do is take a matrix in, apply the update rules, and return a new one. How hard can that be? You write R files with the extension <code>".R"</code> and bring them into your workspace with the <code>source</code> function. Here's an embarrassingly poor go at the Game of Life (note I've only spent 5 minutes with the language, so if you've got any improvements to suggest or more idiomatic ways of doing the same thing, they are greatly received!).<br />
<br />
<script src="https://gist.github.com/fffej/720b2da0462067b7ee1e.js"></script><br />
<br />
Testing this at the REPL with a simple pattern.<br />
<br />
<pre> > blinker = matrix(0,5,5)
> blinker[3,2:4]=1
> blinker
[,1] [,2] [,3] [,4] [,5]
[1,] 0 0 0 0 0
[2,] 0 0 0 0 0
[3,] 0 1 1 1 0
[4,] 0 0 0 0 0
[5,] 0 0 0 0 0
> nextStep(blinker)
[,1] [,2] [,3] [,4] [,5]
[1,] 0 0 0 0 0
[2,] 0 0 1 0 0
[3,] 0 0 1 0 0
[4,] 0 0 1 0 0
[5,] 0 0 0 0 0
</pre><br />
Huzzah! Next steps are probably to write some <a href="https://cran.r-project.org/web/packages/RUnit/vignettes/RUnit.pdf">unit tests [PDF]</a> around it, but learning how to install packages can wait till another day!Jeffhttp://www.blogger.com/profile/08195722595923882332noreply@blogger.comtag:blogger.com,1999:blog-5743983044224833668.post-76120349898976711102015-10-17T10:17:00.000-07:002015-10-17T10:17:48.416-07:00Mindless TDDIn this post, I look at a common misinterpretation of TDD I often see at coding dojos/katas.<br />
<br />
A quick refresher - TDD is simple, but not easy.<br />
<br />
<ol>
<li>Write a failing test</li>
<li>Write the minimum code to make test pass</li>
<li>Refactor</li>
</ol>
<br />
You have to think at each step. This is often overlooked, and TDD portrayed as series of mindless steps (if that were true, developers probably wouldn't get paid so well!).<br />
<br />
Let's take the <a href="http://www.butunclebob.com/ArticleS.UncleBob.TheBowlingGameKata">Bowling Kata</a> as an example of how easy it is to fall into mindless steps. What's the right test to do first?<br />
<br />
<span style="font-family: 'Courier New', Courier, monospace;"> [Test]</span><br />
<span style="font-family: Courier New, Courier, monospace;"> public void PinExists() {</span><br />
<span style="font-family: Courier New, Courier, monospace;"> Assert.That(new Pin(), IsStanding, Is.True);</span><br />
<span style="font-family: Courier New, Courier, monospace;"> }</span><br />
<br />
We're bound to need a Pin class right? And we should definitely check whether it's standing up or falling down. We continue in this vein, and create a Pin that can be knocked down and stood up. Everything proceeds swimmingly. 15 - 20 minutes have elapsed and we have a unit tested equivalent of bool that's no use to anyone. <br />
<br />
I've seen similar anti-patterns in the <a href="https://en.wikipedia.org/wiki/Conway%27s_Game_of_Life">Game of Life</a> kata. We write some tests for a cell, and the rules (three live neighbours means alive and so on). We use this to drive a <span style="font-family: Courier New, Courier, monospace;">Cell</span> class and we add methods to change the state depending on the number of neighbours. Some time passes, and then we realize we actually want a grid of these objects, and they change state based on their neighbours state and we're in a bit of a pickle. Our first guess at an implementation has given us a big problem.<br />
<br />
If we somehow manage to solve the problem from this state, we end up with a load of tests that are coupled to the implementation. Worse, because we've ended up creating lots of classes, we start prematurely applying SOLID, breaking down things into even more waffly collections of useless objects with no coherent basis. Unsurprisingly, it's difficult to see the value in test-driven development when practiced like this.<br />
<br />
So what's the common problem in both these cases? <br />
<br />
Uncle Bob has described this behaviour <a href="http://butunclebob.com/files/downloads/Bowling%20Game%20Kata.ppt">Slide 9</a> of the Bowling Kata PPT describes a similar problem, but attributes it to over-design and suggests TDD as the solution. I agree, but I think some people pervert TDD to mean test-driven development of my supposed solution, rather than TDD of the problem itself.<br />
<br />
The common problem is simple. Not starting with the end in mind!<br />
<br />
If we'd have started the Bowling Kata from the outside-in, our first test might have simply bowled 10 gutter balls and verified we return a zero. We could already ship this to (really) terrible bowlers and it'd work! <br />
<br />
Maybe next we could ensure that if we didn't bowl any spares/strikes it'd sum the scores up. Again, now we can ship this to a wider audience. Next up, let's solve spares, then strikes and at each stage we can ship!<br />
<br />
Each time around the TDD loop we should have solved more of the problem and be closer to fully solving it. <b>TDD should be continuous delivery</b>, if the first test isn't solving the problem for a simple case it's probably not the right test.<br />
<br />
Similarly for the Game of Life, instead of starting from a supposed solution of a cell class, what happens if your first test is just evolving a grid full of dead cells? What happens if we add the rules one at a time? You can ship every test once you've added the boilerplate of the "null" case.<br />
<br />
TDD isn't about testing your possible implementation on the way to solving the problem, it's about writing relevant tests first and driving the implementation from that. <b>Start from the problem!</b><br />
<br />
TDD done right is <b>vicious</b> - it's a series of surgical strikes (tests) aimed at getting you to solve the problem with the minimum amount of code possible.Jeffhttp://www.blogger.com/profile/08195722595923882332noreply@blogger.comtag:blogger.com,1999:blog-5743983044224833668.post-49427185717616054172015-03-31T08:31:00.000-07:002015-03-31T08:31:11.711-07:00Anatomy of a class.Do you ever view a class and get filled with a sense of dread? I did today, so I thought a good old-fashioned rant was in order.<br />
<br />
I opened up a class today and was greeted with this. First off, don't worry, I made the Wibble up. Secondly, if wibble was the first thing you noticed we're probably in trouble.<br />
<br />
<code></code><br />
public sealed class MultiWibbledEntitiesDataPresenterFactory<TDetectionContext, TWibbledEntity, TProvider> : BasicFactory<Unit, IDataPresenter<MultiWibbledEntitiesContext<TDetectionContext, TWibbledEntity>>><br />
where TWibbledEntity : WibbledEntity<br />
where TProvider : IProvider<TWibbledEntity><br />
{<br />
private readonly IUtcDateTimeProvider m_UtcDateTimeProvider;<br />
private readonly ILocalDateTimeProvider m_LocalDateTimeProvider;<br />
private readonly IWibbledEntityDetector<TDetectionContext> m_WibbledEntityDetector;<br />
private readonly IFactory<TWibbledEntity, TProvider> m_ProviderFactory;<br />
private readonly IFactory<TWibbledEntity, IDataPresenter<SingleWibbledEntityContext<TWibbledEntity, TProvider>>> m_RawWibbledEntityDataPresenterFactory;<br />
private readonly Func<IUtcDateTimeProvider, string, IStatusLogger> m_StatusLoggerBuilder;<br />
<br />
public MultiWibbledEntitiesDataPresenterFactory(<br />
IUtcDateTimeProvider utcDateTimeProvider,<br />
ILocalDateTimeProvider localDateTimeProvider,<br />
IWibbledEntityDetector<TDetectionContext> wibbledEntityDetector,<br />
IFactory<TWibbledEntity, TProvider> providerFactory,<br />
IFactory<TWibbledEntity, IDataPresenter<SingleWibbledEntityContext<TWibbledEntity, TProvider>>> rawWibbledEntityDataPresenterFactory,<br />
Func<IUtcDateTimeProvider, string, IStatusLogger> statusLoggerBuilder<br />
)<br />
{<br />
m_UtcDateTimeProvider = utcDateTimeProvider;<br />
m_LocalDateTimeProvider = localDateTimeProvider;<br />
m_WibbledEntityDetector = wibbledEntityDetector;<br />
m_ProviderFactory = providerFactory;<br />
m_RawWibbledEntityDataPresenterFactory = rawWibbledEntityDataPresenterFactory;<br />
m_StatusLoggerBuilder = statusLoggerBuilder;<br />
}<br />
<br />
protected override IDataPresenter<MultiWibbledEntitiesContext<TDetectionContext, TWibbledEntity>> ConstructItem(Unit key)<br />
{<br />
return new MultiWibbledEntitiesDataPresenter<TDetectionContext, TWibbledEntity, TProvider>(<br />
m_UtcDateTimeProvider,<br />
m_LocalDateTimeProvider,<br />
m_WibbledEntityDetector,<br />
m_ProviderFactory,<br />
m_RawWibbledEntityDataPresenterFactory,<br />
m_StatusLoggerBuilder<br />
);<br />
}<br />
}<br />
<br />
<br />
OK, you've read through that. You've probably died a little inside. What did you learn? Well, this is a <code>MultiWibbledEntitiesDataPresentorFactory</code>. <br />
<br />
What the actual fuck? <br />
<br />
A multi wibbled entities data presenter factory. <br />
<br />
Spacing it out doesn't help much either. It's a factory that makes data presenters for multi wibbled things. OK, that starts to make some sense. I guess I'd use this class if ever I needed to make a multi-wibbled-entities-data-presenter-factory.<br />
<br />
Let's say that's the case. How do I construct one of these factory things? I need a couple of time providers (UTC and local time, just in case), a detector (no idea what that is), two more factories and a function called "<code>statusLoggerBuilder</code>". And this is just to create an object (albeit a rather complicated multi-wibbled data presenter object).<br />
<br />
What can you do with the class? Not a lot, there aren't any public methods other than the constructor, and that's pretty boring. So, in order to make any progress, you'll have to explore a few more classes. You'll need to visit the "<code>BasicFactory</code>". You'll need to look at <code>Unit</code>, <code>IDataPresenter</code> and a few more parameterized classes. In order to work out what this does, I've got to read all these files.<br />
<br />
What's with all the generics? Does this tell me the original developer was a template meta-programming C++ person? Why all the complexity?<br />
<br />
How many files do I need to open in order to understand this class?<br />
<br />
What problem is this class solving? The code doesn't tell me this, there aren't any comments and there aren't any tests. The only way for me to understand this code is to navigate all the code's friends and work out what each of them do.<br />
<br />
But on the plus side, I can create one and test it, so it must be good right?Jeffhttp://www.blogger.com/profile/08195722595923882332noreply@blogger.comtag:blogger.com,1999:blog-5743983044224833668.post-41465396617919688092015-01-19T11:00:00.000-08:002015-01-20T09:26:05.735-08:00The Diamond Square Algorithm<div dir="ltr" style="line-height: 1.15; margin-bottom: 0pt; margin-top: 0pt; text-align: justify;">
<div dir="ltr" style="line-height: 1.15; margin-bottom: 0pt; margin-top: 0pt; text-align: justify;">
<span style="color: black; font-family: Arial; font-size: 15px; vertical-align: baseline; white-space: pre-wrap;">Ever wondered how to generate a landscape? I've been fascinated by these since the days of </span><a href="http://en.wikipedia.org/wiki/VistaPro" style="line-height: 1.15; text-decoration: none;"><span style="color: #1155cc; font-family: Arial; font-size: 15px; text-decoration: underline; vertical-align: baseline; white-space: pre-wrap;">Vista Pro</span></a><span style="color: black; font-family: Arial; font-size: 15px; vertical-align: baseline; white-space: pre-wrap;"> on my trusty Amiga.</span></div>
<div dir="ltr" style="line-height: 1.15; margin-bottom: 0pt; margin-top: 0pt; text-align: justify;">
<b style="font-weight: normal;"><br /></b></div>
<div dir="ltr" style="line-height: 1.15; margin-bottom: 0pt; margin-top: 0pt; text-align: justify;">
<span style="background-color: transparent; color: black; font-family: Arial; font-size: 15px; font-style: normal; font-variant: normal; font-weight: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">The </span><a href="http://en.wikipedia.org/wiki/Diamond-square_algorithm" style="text-decoration: none;"><span style="background-color: transparent; color: #1155cc; font-family: Arial; font-size: 15px; font-style: normal; font-variant: normal; font-weight: normal; text-decoration: underline; vertical-align: baseline; white-space: pre-wrap;">diamond-square algorithm</span></a><span style="background-color: transparent; color: black; font-family: Arial; font-size: 15px; font-style: normal; font-variant: normal; font-weight: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> is a method for generating heightmaps. It's a great algorithm because it's amazingly simple and produces something very visual (similar to the emergent behaviour exhibited by the </span><a href="http://www.fatvat.co.uk/2009/07/flocking-about-with-clojure.html" style="text-decoration: none;"><span style="background-color: transparent; color: #1155cc; font-family: Arial; font-size: 15px; font-style: normal; font-variant: normal; font-weight: normal; text-decoration: underline; vertical-align: baseline; white-space: pre-wrap;">flocking algorithm</span></a><span style="background-color: transparent; color: black; font-family: Arial; font-size: 15px; font-style: normal; font-variant: normal; font-weight: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">. My kind of algorithm! In this post, I'll try to explain the implementation using </span><a href="https://www.haskell.org/" style="text-decoration: none;"><span style="background-color: transparent; color: #1155cc; font-family: Arial; font-size: 15px; font-style: normal; font-variant: normal; font-weight: normal; text-decoration: underline; vertical-align: baseline; white-space: pre-wrap;">Haskell</span></a><span style="background-color: transparent; color: black; font-family: Arial; font-size: 15px; font-style: normal; font-variant: normal; font-weight: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> and generate some pretty pictures.</span></div>
<div dir="ltr" style="line-height: 1.15; margin-bottom: 0pt; margin-top: 0pt; text-align: justify;">
<b style="font-weight: normal;"><br /></b></div>
<div dir="ltr" style="line-height: 1.15; margin-bottom: 0pt; margin-top: 0pt; text-align: justify;">
<span style="background-color: transparent; color: black; font-family: Arial; font-size: 15px; font-style: normal; font-variant: normal; font-weight: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">As the </span><a href="http://design.osu.edu/carlson/history/PDFs/p371-fournier.pdf" style="text-decoration: none;"><span style="background-color: transparent; color: #1155cc; font-family: Arial; font-size: 15px; font-style: normal; font-variant: normal; font-weight: normal; text-decoration: underline; vertical-align: baseline; white-space: pre-wrap;">paper</span></a><span style="background-color: transparent; color: black; font-family: Arial; font-size: 15px; font-style: normal; font-variant: normal; font-weight: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> [PDF] states, previous modelling techniques for graphics were based on the idea that you can simply describe a landscape as some set of deterministic functions. Bezier and B-spline patches used higher-order polynomials to describe objects and this approach was good for rendering artificial objects. Natural objects, such as terrain, don't have regular patterns so an approach likes splines doesn't work.</span></div>
<div dir="ltr" style="line-height: 1.15; margin-bottom: 0pt; margin-top: 0pt; text-align: justify;">
<b style="font-weight: normal;"><br /></b></div>
<div dir="ltr" style="line-height: 1.15; margin-bottom: 0pt; margin-top: 0pt; text-align: justify;">
<span style="background-color: transparent; color: black; font-family: Arial; font-size: 15px; font-style: normal; font-variant: normal; font-weight: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">This algorithm was innovative because it used a stochastic approach. Given some simple rules (and some randomness!) the algorithm generates a "natural" looking landscape. The paper describes models for 1D, 2D and 3D surfaces. We'll just use the simplest possible example, rendering a height map.</span></div>
<div dir="ltr" style="line-height: 1.15; margin-bottom: 0pt; margin-top: 0pt; text-align: justify;">
<b style="font-weight: normal;"><br /></b></div>
<div dir="ltr" style="line-height: 1.15; margin-bottom: 0pt; margin-top: 0pt; text-align: justify;">
<span style="background-color: transparent; color: black; font-family: Arial; font-size: 15px; font-style: normal; font-variant: normal; font-weight: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">We start with a square with each corner given a randomly assigned height. If the area of this square is 1, then we’re done. Easy. We’ll call the corners, TL, TR, BL and BR (representing top left, top right, bottom left and bottom right).</span></div>
<div dir="ltr" style="line-height: 1.15; margin-bottom: 0pt; margin-top: 0pt; text-align: justify;">
<b style="font-weight: normal;"><br /></b></div>
<div dir="ltr" style="line-height: 1.15; margin-bottom: 0pt; margin-top: 0pt; text-align: center;">
<span style="background-color: transparent; color: black; font-family: Arial; font-size: 15px; font-style: normal; font-variant: normal; font-weight: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><img height="190px;" src="https://docs.google.com/drawings/d/sMQxrvDdIy2CGe4e0_u9SqQ/image?w=205&h=190&rev=42&ac=1" style="-webkit-transform: rotate(0rad); border: none; transform: rotate(0rad);" width="205px;" /></span></div>
<div dir="ltr" style="line-height: 1.15; margin-bottom: 0pt; margin-top: 0pt; text-align: justify;">
<b style="font-weight: normal;"><br /></b></div>
<div dir="ltr" style="line-height: 1.15; margin-bottom: 0pt; margin-top: 0pt; text-align: justify;">
<span style="background-color: transparent; color: black; font-family: Arial; font-size: 15px; font-style: normal; font-variant: normal; font-weight: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">If the square is too big, then we recursively divide it into smaller squares.</span></div>
<div dir="ltr" style="line-height: 1.15; margin-bottom: 0pt; margin-top: 0pt; text-align: center;">
<span style="background-color: transparent; color: black; font-family: Arial; font-size: 15px; font-style: normal; font-variant: normal; font-weight: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><img height="217px;" src="https://docs.google.com/drawings/d/smyplwofRGxboDtPad4GlXA/image?w=339&h=217&rev=148&ac=1" style="-webkit-transform: rotate(0rad); border: none; transform: rotate(0rad);" width="339px;" /></span></div>
<div dir="ltr" style="line-height: 1.15; margin-bottom: 0pt; margin-top: 0pt; text-align: justify;">
<span style="background-color: transparent; color: black; font-family: Arial; font-size: 15px; font-style: normal; font-variant: normal; font-weight: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">We assign each new square a height based on the average of the points surrounding it. Note there’s nothing stochastic about this approach yet, it’s purely deterministic.</span></div>
<div dir="ltr" style="line-height: 1.15; margin-bottom: 0pt; margin-top: 0pt; text-align: justify;">
<b style="font-weight: normal;"><br /></b></div>
<div dir="ltr" style="line-height: 1.15; margin-bottom: 0pt; margin-top: 0pt; text-align: justify;">
<span style="background-color: transparent; color: black; font-family: Arial; font-size: 15px; font-style: normal; font-variant: normal; font-weight: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">We can model this with Haskell pretty clearly. We start off by defining a simple type to represent a Square.</span></div>
<div dir="ltr" style="line-height: 1.15; margin-bottom: 0pt; margin-top: 0pt; text-align: justify;">
<b style="font-weight: normal;"><br /></b></div>
<div dir="ltr" style="line-height: 1.15; margin-bottom: 0pt; margin-top: 0pt; text-align: justify;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">type Point = (Int,Int)</span></div>
<div dir="ltr" style="line-height: 1.15; margin-bottom: 0pt; margin-top: 0pt; text-align: justify;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">data Square = Square</span></div>
<div dir="ltr" style="line-height: 1.15; margin-bottom: 0pt; margin-top: 0pt; text-align: justify;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> {</span></div>
<div dir="ltr" style="line-height: 1.15; margin-bottom: 0pt; margin-top: 0pt; text-align: justify;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> position :: Point </span></div>
<div dir="ltr" style="line-height: 1.15; margin-bottom: 0pt; margin-top: 0pt; text-align: justify;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> , size :: Int </span></div>
<div dir="ltr" style="line-height: 1.15; margin-bottom: 0pt; margin-top: 0pt; text-align: justify;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> , tl :: Double -- Height of top left</span></div>
<div dir="ltr" style="line-height: 1.15; margin-bottom: 0pt; margin-top: 0pt; text-align: justify;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> , tr :: Double -- Height of top right</span></div>
<div dir="ltr" style="line-height: 1.15; margin-bottom: 0pt; margin-top: 0pt; text-align: justify;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> , bl :: Double -- Height of bottom left</span></div>
<div dir="ltr" style="line-height: 1.15; margin-bottom: 0pt; margin-top: 0pt; text-align: justify;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> , br :: Double -- Height of bottom right</span></div>
<div dir="ltr" style="line-height: 1.15; margin-bottom: 0pt; margin-top: 0pt; text-align: justify;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> } deriving (Show,Eq)</span></div>
<div dir="ltr" style="line-height: 1.15; margin-bottom: 0pt; margin-top: 0pt; text-align: justify;">
<b style="font-weight: normal;"><br /></b></div>
<div dir="ltr" style="line-height: 1.15; margin-bottom: 0pt; margin-top: 0pt; text-align: justify;">
<span style="background-color: transparent; color: black; font-family: Arial; font-size: 15px; font-style: normal; font-variant: normal; font-weight: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">Now all we have to do write a little function to divide things into four. Firstly let’s capture the pattern that dividing stops when the size of the square is one.</span></div>
<div dir="ltr" style="line-height: 1.15; margin-bottom: 0pt; margin-top: 0pt; text-align: justify;">
<b style="font-weight: normal;"><br /></b></div>
<div dir="ltr" style="line-height: 1.15; margin-bottom: 0pt; margin-top: 0pt; text-align: justify;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">isUnit :: Square -> Bool</span></div>
<div dir="ltr" style="line-height: 1.15; margin-bottom: 0pt; margin-top: 0pt; text-align: justify;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">isUnit sq = size sq == 1</span></div>
<div dir="ltr" style="line-height: 1.15; margin-bottom: 0pt; margin-top: 0pt; text-align: justify;">
<b style="font-weight: normal;"><br /></b></div>
<div dir="ltr" style="line-height: 1.15; margin-bottom: 0pt; margin-top: 0pt; text-align: justify;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">allSubSquares :: (Square -> [Square]) -> Square -> [Square]</span></div>
<div dir="ltr" style="line-height: 1.15; margin-bottom: 0pt; margin-top: 0pt; text-align: justify;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">allSubSquares f sq </span></div>
<div dir="ltr" style="line-height: 1.15; margin-bottom: 0pt; margin-top: 0pt; text-align: justify;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> | isUnit sq = [sq]</span></div>
<div dir="ltr" style="line-height: 1.15; margin-bottom: 0pt; margin-top: 0pt; text-align: justify;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> | otherwise = concatMap (allSubSquares f) (f sq)</span></div>
<div dir="ltr" style="line-height: 1.15; margin-bottom: 0pt; margin-top: 0pt; text-align: justify;">
<b style="font-weight: normal;"><br /></b></div>
<div dir="ltr" style="line-height: 1.15; margin-bottom: 0pt; margin-top: 0pt; text-align: justify;">
<span style="background-color: transparent; color: black; font-family: Arial; font-size: 15px; font-style: normal; font-variant: normal; font-weight: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">The </span><span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 15px; font-style: normal; font-variant: normal; font-weight: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">allSubSquares</span><span style="background-color: transparent; color: black; font-family: Arial; font-size: 15px; font-style: normal; font-variant: normal; font-weight: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> function now simply repeatedly called our splitting function until things are reduced to the tiniest possible size.</span></div>
<div dir="ltr" style="line-height: 1.15; margin-bottom: 0pt; margin-top: 0pt; text-align: justify;">
<b style="font-weight: normal;"><br /></b></div>
<div dir="ltr" style="line-height: 1.15; margin-bottom: 0pt; margin-top: 0pt; text-align: justify;">
<span style="background-color: transparent; color: black; font-family: Arial; font-size: 15px; font-style: normal; font-variant: normal; font-weight: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">What does our split function look like? Well, all it has to do is calculate the new squares as the picture defines above. It looks a little like this:</span></div>
<div dir="ltr" style="line-height: 1.15; margin-bottom: 0pt; margin-top: 0pt; text-align: justify;">
<b style="font-weight: normal;"><br /></b></div>
<div dir="ltr" style="line-height: 1.15; margin-bottom: 0pt; margin-top: 0pt; text-align: justify;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">divide :: Double -> Square -> [Square]</span></div>
<div dir="ltr" style="line-height: 1.15; margin-bottom: 0pt; margin-top: 0pt; text-align: justify;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">divide eps parent = [</span></div>
<div dir="ltr" style="line-height: 1.15; margin-bottom: 0pt; margin-top: 0pt; text-align: justify;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> sq { tr = at, br = ah, bl = al } -- top left unchanged</span></div>
<div dir="ltr" style="line-height: 1.15; margin-bottom: 0pt; margin-top: 0pt; text-align: justify;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> , (move sq (half,0)) { tl = at, bl = ah, br = ar } -- top right unchanged</span></div>
<div dir="ltr" style="line-height: 1.15; margin-bottom: 0pt; margin-top: 0pt; text-align: justify;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> , (move sq (0,half)) { tr = ah, br = ab, tl = al } -- bottom left unchanged</span></div>
<div dir="ltr" style="line-height: 1.15; margin-bottom: 0pt; margin-top: 0pt; text-align: justify;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> , (move sq (half,half)) { tl = ah, bl = ab, tr = ar } -- bottom right unchanged</span></div>
<div dir="ltr" style="line-height: 1.15; margin-bottom: 0pt; margin-top: 0pt; text-align: justify;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> ]</span></div>
<div dir="ltr" style="line-height: 1.15; margin-bottom: 0pt; margin-top: 0pt; text-align: justify;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> where </span></div>
<div dir="ltr" style="line-height: 1.15; margin-bottom: 0pt; margin-top: 0pt; text-align: justify;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> half = size parent `div` 2</span></div>
<div dir="ltr" style="line-height: 1.15; margin-bottom: 0pt; margin-top: 0pt; text-align: justify;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> sq = parent { size = half }</span></div>
<div dir="ltr" style="line-height: 1.15; margin-bottom: 0pt; margin-top: 0pt; text-align: justify;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> at = averageTopHeight parent</span></div>
<div dir="ltr" style="line-height: 1.15; margin-bottom: 0pt; margin-top: 0pt; text-align: justify;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> ah = averageHeight eps parent -- height of middle</span></div>
<div dir="ltr" style="line-height: 1.15; margin-bottom: 0pt; margin-top: 0pt; text-align: justify;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> ab = averageBottomHeight parent</span></div>
<div dir="ltr" style="line-height: 1.15; margin-bottom: 0pt; margin-top: 0pt; text-align: justify;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> ar = averageRightHeight parent</span></div>
<div dir="ltr" style="line-height: 1.15; margin-bottom: 0pt; margin-top: 0pt; text-align: justify;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> al = averageLeftHeight parent</span></div>
<div dir="ltr" style="line-height: 1.15; margin-bottom: 0pt; margin-top: 0pt; text-align: justify;">
<b style="font-weight: normal;"><br /></b></div>
<div dir="ltr" style="line-height: 1.15; margin-bottom: 0pt; margin-top: 0pt; text-align: justify;">
<span style="background-color: transparent; color: black; font-family: Arial; font-size: 15px; font-style: normal; font-variant: normal; font-weight: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">OK, this isn’t very exciting (and I’ve left out the boilerplate). But we have something now, it’s deterministic, but it creates cool results.</span></div>
<div dir="ltr" style="line-height: 1.15; margin-bottom: 0pt; margin-top: 0pt; text-align: justify;">
<b style="font-weight: normal;"><br /></b></div>
<div dir="ltr" style="line-height: 1.15; margin-bottom: 0pt; margin-top: 0pt; text-align: center;">
<span style="background-color: transparent; color: black; font-family: Arial; font-size: 15px; font-style: normal; font-variant: normal; font-weight: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><img height="182px;" src="https://lh4.googleusercontent.com/BoZ6locK27aNJf78LzKUlW1xhLO7lqfG1D39WZeOzbr1bHjQrxA0jil5dJ1ZTXWpGQXhUqmjOyK3dBFvXFCYGefgoDZNOV2Em_SFy95-fSTEyOMWPed9cWfCUqdcqnk38A" style="-webkit-transform: rotate(0rad); border: none; transform: rotate(0rad);" width="182px;" /></span></div>
<div dir="ltr" style="line-height: 1.15; margin-bottom: 0pt; margin-top: 0pt; text-align: justify;">
<b style="font-weight: normal;"><br /></b></div>
<div dir="ltr" style="line-height: 1.15; margin-bottom: 0pt; margin-top: 0pt; text-align: justify;">
<span style="background-color: transparent; color: black; font-family: Arial; font-size: 15px; font-style: normal; font-variant: normal; font-weight: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">Woo. I used </span><a href="https://hackage.haskell.org/package/JuicyPixels" style="text-decoration: none;"><span style="background-color: transparent; color: #1155cc; font-family: Arial; font-size: 15px; font-style: normal; font-variant: normal; font-weight: normal; text-decoration: underline; vertical-align: baseline; white-space: pre-wrap;">JuicyPixels</span></a><span style="background-color: transparent; color: black; font-family: Arial; font-size: 15px; font-style: normal; font-variant: normal; font-weight: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> to render the image. I really wish I’d found this library a long time ago, it’s fabulously simple to use and all I needed to do was use the sexy </span><span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 15px; font-style: normal; font-variant: normal; font-weight: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">generateImage</span><span style="background-color: transparent; color: black; font-family: Arial; font-size: 15px; font-style: normal; font-variant: normal; font-weight: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> function.</span></div>
<div dir="ltr" style="line-height: 1.15; margin-bottom: 0pt; margin-top: 0pt; text-align: justify;">
<b style="font-weight: normal;"><br /></b></div>
<div dir="ltr" style="line-height: 1.15; margin-bottom: 0pt; margin-top: 0pt; text-align: justify;">
<span style="background-color: transparent; color: black; font-family: Arial; font-size: 15px; font-style: normal; font-variant: normal; font-weight: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">So how do we actually generate something that looks vaguely natural?</span></div>
<div dir="ltr" style="line-height: 1.15; margin-bottom: 0pt; margin-top: 0pt; text-align: justify;">
<b style="font-weight: normal;"><br /></b></div>
<div dir="ltr" style="line-height: 1.15; margin-bottom: 0pt; margin-top: 0pt; text-align: justify;">
<span style="background-color: transparent; color: black; font-family: Arial; font-size: 15px; font-style: normal; font-variant: normal; font-weight: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">The answer is randomness. Tightly controlled. Let’s look at our original square divider and make one really small change.</span></div>
<div dir="ltr" style="line-height: 1.15; margin-bottom: 0pt; margin-top: 0pt; text-align: justify;">
<b style="font-weight: normal;"><br /></b></div>
<div dir="ltr" style="line-height: 1.15; margin-bottom: 0pt; margin-top: 0pt; text-align: center;">
<span style="background-color: transparent; color: black; font-family: Arial; font-size: 15px; font-style: normal; font-variant: normal; font-weight: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><img height="217px;" src="https://docs.google.com/drawings/d/sTAt5XJxnBAfX-cQa2Dz-Og/image?w=339&h=217&rev=24&ac=1" style="-webkit-transform: rotate(0rad); border: none; transform: rotate(0rad);" width="339px;" /></span></div>
<div dir="ltr" style="line-height: 1.15; margin-bottom: 0pt; margin-top: 0pt; text-align: justify;">
<b style="font-weight: normal;"><br /></b></div>
<div dir="ltr" style="line-height: 1.15; margin-bottom: 0pt; margin-top: 0pt; text-align: justify;">
<span style="background-color: transparent; color: black; font-family: Arial; font-size: 15px; font-style: normal; font-variant: normal; font-weight: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">I’ll save you the trouble of finding it, it’s that pesky “e” we’ve added to the middle. What is e?</span></div>
<div dir="ltr" style="line-height: 1.15; margin-bottom: 0pt; margin-top: 0pt; text-align: justify;">
<b style="font-weight: normal;"><br /></b></div>
<div dir="ltr" style="line-height: 1.15; margin-bottom: 0pt; margin-top: 0pt; text-align: justify;">
<span style="background-color: transparent; color: black; font-family: Arial; font-size: 15px; font-style: normal; font-variant: normal; font-weight: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">Well, it’s the stochastic approach. It’s a random number that’s assigned to displace the midpoint. When the square is big, the displacement is big. When the square is small, the displacement is small. In fact we simply define e as a random number [-0.5, 0.5] scaled by the size of the square.</span></div>
<div dir="ltr" style="line-height: 1.15; margin-bottom: 0pt; margin-top: 0pt; text-align: justify;">
<b style="font-weight: normal;"><br /></b></div>
<div dir="ltr" style="line-height: 1.15; margin-bottom: 0pt; margin-top: 0pt; text-align: justify;">
<span style="background-color: transparent; color: black; font-family: Arial; font-size: 15px; font-style: normal; font-variant: normal; font-weight: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">What happens when we add this displacement is kind of cool. We now get a random surface that smooths itself out and almost looks natural.</span></div>
<div dir="ltr" style="line-height: 1.15; margin-bottom: 0pt; margin-top: 0pt; text-align: justify;">
<b style="font-weight: normal;"><br /></b></div>
<div dir="ltr" style="line-height: 1.15; margin-bottom: 0pt; margin-top: 0pt; text-align: center;">
<span style="background-color: transparent; color: black; font-family: Arial; font-size: 15px; font-style: normal; font-variant: normal; font-weight: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><img height="249px;" src="https://lh6.googleusercontent.com/WgHwgcOtHB_5CIYpmdKLthz6WBOdwp7E_3EPngXmHjRN9NFr-aonlLe1EBkCjFQOklvtN6Py4OeJkqlufr0RtpJZxUQmtMYJ4cwuSF286cbXA6c7Tm5Fia8TC6upE_xXlw" style="-webkit-transform: rotate(0rad); border: none; transform: rotate(0rad);" width="249px;" /></span></div>
<div dir="ltr" style="line-height: 1.15; margin-bottom: 0pt; margin-top: 0pt; text-align: justify;">
<b style="font-weight: normal;"><br /></b></div>
<div dir="ltr" style="line-height: 1.15; margin-bottom: 0pt; margin-top: 0pt; text-align: justify;">
<span style="background-color: transparent; color: black; font-family: Arial; font-size: 15px; font-style: normal; font-variant: normal; font-weight: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">I think this is pretty neat. It’s a smooth landscape that could easily look natural. We can do even better by giving a bit of color. I’ve done this using a simple color map as described on </span><a href="http://stackoverflow.com/q/7706339/128645" style="text-decoration: none;"><span style="background-color: transparent; color: #1155cc; font-family: Arial; font-size: 15px; font-style: normal; font-variant: normal; font-weight: normal; text-decoration: underline; vertical-align: baseline; white-space: pre-wrap;">Stackoverflow</span></a><span style="background-color: transparent; color: black; font-family: Arial; font-size: 15px; font-style: normal; font-variant: normal; font-weight: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">.</span></div>
<div dir="ltr" style="line-height: 1.15; margin-bottom: 0pt; margin-top: 0pt; text-align: justify;">
<b style="font-weight: normal;"><br /></b></div>
<div dir="ltr" style="line-height: 1.15; margin-bottom: 0pt; margin-top: 0pt; text-align: justify;">
<span style="background-color: transparent; color: black; font-family: Arial; font-size: 15px; font-style: normal; font-variant: normal; font-weight: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">Using a map generated from similar parameters, we get a much prettier colour. If you squint a bit, imagine something it could be a natural scene right?</span></div>
<div dir="ltr" style="line-height: 1.15; margin-bottom: 0pt; margin-top: 0pt; text-align: justify;">
<b style="font-weight: normal;"><br /></b></div>
<div dir="ltr" style="line-height: 1.15; margin-bottom: 0pt; margin-top: 0pt; text-align: center;">
<span style="background-color: transparent; color: black; font-family: Arial; font-size: 15px; font-style: normal; font-variant: normal; font-weight: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><img height="320" src="https://lh3.googleusercontent.com/RP02AsOJwqjI7sIoxBsCwk3sDfOcJZJH574HjsH0UnAszx8hjCfUA8DQU3Qax-Oh1wFli-mKUe2XfgzuXW2J_pWktUN3D14xS0n55Mly_EZsC5naMPwi8_XKrKBv5aP9Vg" style="-webkit-transform: rotate(0rad); border: none; transform: rotate(0rad);" width="320" /></span></div>
<div dir="ltr" style="line-height: 1.15; margin-bottom: 0pt; margin-top: 0pt; text-align: justify;">
<br /></div>
<div dir="ltr" style="line-height: 1.15; margin-bottom: 0pt; margin-top: 0pt; text-align: justify;">
<span style="background-color: transparent; color: black; font-family: Arial; font-size: 15px; font-style: normal; font-variant: normal; font-weight: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">All the code for this is available on my GitHub profile, the </span><a href="https://github.com/fffej/haskellprojects/tree/master/diamond-square" style="text-decoration: none;"><span style="background-color: transparent; color: #1155cc; font-family: Arial; font-size: 15px; font-style: normal; font-variant: normal; font-weight: normal; text-decoration: underline; vertical-align: baseline; white-space: pre-wrap;">diamond-square</span></a><span style="background-color: transparent; color: black; font-family: Arial; font-size: 15px; font-style: normal; font-variant: normal; font-weight: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> project. Some fun extensions to this would be to create some animations, or actually render it in 3D with OpenGL.</span></div>
</div>
Jeffhttp://www.blogger.com/profile/08195722595923882332noreply@blogger.com0tag:blogger.com,1999:blog-5743983044224833668.post-73229408075806511782015-01-13T14:00:00.000-08:002015-01-13T14:00:00.267-08:00Book review: Lead with Respect - A Novel of Lean PracticeBook review: Lead with Respect - A Novel of Lean Practice<br />
<br />
As I said in a <a href="http://www.fatvat.co.uk/2015/01/book-review-lean-manager-novel-of-lean.html">previous post</a>, I'm a sucker for a business novel. <a href="http://www.amazon.com/gp/product/B00M4VQG60/ref=as_li_tl?ie=UTF8&camp=1789&creative=390957&creativeASIN=B00M4VQG60&linkCode=as2&tag=fatvat-20&linkId=SXCRD3756AX3W24Y">Lead with Respect</a> is another business novel by father and son team Michael and Freddy Balle.<br />
<br />
My goal in reading this was to get an idea of how lean management might apply to software development.<br />
<br />
The story starts with Jane the CEO of Southcape software who is working on some software for a famed Lean company called Nexplas. As you might expect, things aren't going well. The software doesn't do what's required and milestones aren't being met. This sets the stage for the sensei/student relationship between Jane (CEO of a software company) and Andrew (VP of a manufacturing company). Throughout the book, Andrew imparts knowledge to Jane (and hopefully the reader too).<br />
<br />
The core theme of the book is, as the title suggests, respect. Respect, in the lean sense, is much wider than the <a href="http://www.thefreedictionary.com/respect">dictionary definition</a> of respect. In Lean, respect means:<br />
<br />
<ul> <li>Engage everybody all the time in problem solving, together, by making every effort to understand each other's point of view.</li>
<li>Guarantee the quality, productivity and flexibility as we try to cut nonsatisfaction and nonvalue-added work.</li>
<li>Share success and reward involvement and initiative which makes our respect promise credible and sustains our long-term growth. Customer satisfaction simply can't happen without employee satisfaction.</li>
</ul><br />
"Lead with Respect" rallies against a preconceived notion of Lean as grinding people into the ground, cutting costs and working them till they drop. I've never had this view of lean, but I can see how it would make excellent <a href="http://en.wikipedia.org/wiki/Fear,_uncertainty_and_doubt">FUD</a> for competiting philosophies.<br />
<br />
So what does a <i>Lean</i> manager actually do? Put simply:<br />
<br />
<blockquote>Our job as managers is to create conditions where people can be successful at their job. And what that comes down to is working together to solve the problems we face.</blockquote><br />
This all sounds easy, right?<br />
<br />
What are problems? The book ties problems down to continuous improvement through the familiar equation that a job is the sum of work and the continuous improvement that must be a part of employment. We should accept that continuous improvement is a part of the job. I don't think this is a difficult case to argue. In software engineering (and other <a href="http://en.wikipedia.org/wiki/Knowledge_worker">knowledge work</a>) if you aren't constantly learning, then you are falling behind. This differs<br />
<br />
Lead with Respect argues that our job is to support people in this journey of continuous improvement. Continuous improvement is about change; change is scary! We should work with people to break larger challenges into smaller, every day steps. The link is made (again) with kaizen and standards, namely that you can't have continuous improvement without some standards.<br />
<br />
<blockquote>To improve performance we have to improve processes. To improve processes we have to improve individual's competence and their ability to work with others.</blockquote><br />
In the book, Jane improves her performance as a team using things that are familiar to most software engineers who've got any experience of agile. Pair programming, test-driven development and listening to customers. None of this is surprising. Towards the end of the book, another tool for conversations is introduced in the form of <a href="https://en.wikipedia.org/wiki/A3_Problem_Solving">A3 Problem Solving</a>. This is something that sounds like a process nonsense, but the book does a good job of explaining that it's about the scientific method. By following a structured approach it provides a way to have structured conversations which in turn makes it easier for others to understand the problem and potentially coach people to a solution. This is something that Mike Rother explores in his book, <a href="http://www.amazon.com/gp/product/0071635238/ref=as_li_tl?ie=UTF8&camp=1789&creative=390957&creativeASIN=0071635238&linkCode=as2&tag=fatvat-20&linkId=CVAPYMRCV6W2PRU3">Toyota Kata</a> which is yet another item on my ever-growing reading list.<br />
<br />
Was this book a good read? Well, it was enjoyable enough, the characters were believable at least. I'm not sure I got as much out of it as the earlier book and some of the discussions about software felt a bit unrealistic. The key themes definitely seem transferable to any discipline.Jeffhttp://www.blogger.com/profile/08195722595923882332noreply@blogger.com0tag:blogger.com,1999:blog-5743983044224833668.post-5130570826811668272015-01-05T13:00:00.000-08:002015-01-05T13:00:00.398-08:00Book Review: The Lean Manager - A novel of lean transformationI've recently taken on a different role at work and as part of that I've tried to force myself to read as many books on management topics as possible.<br />
<br />
After reading <a href="http://en.wikipedia.org/wiki/The_Goal_(novel)">The Goal</a> and <a href="http://en.wikipedia.org/wiki/The_Phoenix_Project:_A_Novel_About_IT,_DevOps,_and_Helping_Your_Business_Win">The Phoenix Project</a>, I realized that I'm a sucker for a business novel. After a bit of searching, I settled on the book by Freddy Balle and Michael Balle entitled <a href="http://www.lean.org/Bookstore/ProductDetails.cfm?SelectedProductID=261">The Lean Manager: A novel of lean transformation</a>.<br />
<br />
The book's setting is in an automotive part plant in France that is under threat from closure. Andy, the books protaganist, agrees a deal with the CEO, Phil (also his mentor), that if the plant becomes competitive it will not close. It's a familiar setting from other books, I suppose the concept is that the best catalyst for change is adversity.<br />
<br />
What did this book teach me?<br />
<br />
<blockquote>Standardized work and kaizen are two sides of the same coin.</blockquote><br />
For a long time, I've resisted the idea that standardizing any work to do with programming is a good thing. The best teams I've worked in have always had implicit coding standards created by being a closely knit team unafraid to voice concerns when standards (even if they are only in the heads of a few) weren't met. The idea of explicitly setting coding standards (and I'm not talking about tabs vs. spaces, more in the style of <a href="http://www.gotw.ca/publications/c++cs.htm">101 Coding Guidelines for C++</a>) has always been an anathema for me. I think the reasons for this are simple; I used to think standards implied something external to the team influencing how they work.<br />
<br />
<blockquote>Standardized work is about agreeing how the work should be done best, to better see the problems. Kaizen is about encouraging operators and frontline supervisors to solve all the problems that appear as gaps to the standard.</blockquote><br />
I realize this is talking about manufacturing, not software engineering, but the idea of defining a standard and viewing a gap to the standard as a problem is a powerful one. As a stupid example, let's say you define automated acceptance testing as standard for all new features, but fail to meet it. Why x 5? What can you learn from this that changes the way you develop software? By stating a standard and holding yourself to account you see the problems and force a conversation about it. Standardized work encourages problem solving (kaizen) by acting as a tool that allows you to have the right conversations.<br />
<br />
Another key theme from the book is the idea of "go and see". The best way to learn is to go and see. This applies everywhere. Go and see (<a href="http://en.wikipedia.org/wiki/Genchi_Genbutsu">Genchi Benbutsu</a>) teams, Go and see customers. Go to the place where the work happens and magic will happen. Again, this sounds like a very simple thing (<a href="http://en.wikipedia.org/wiki/Management_by_wandering_around">management by wandering around</a>) but it's deceptively powerful when adopted as a deliberate technique (or at least, it is in the book!). <br />
<br />
Visual Work Management is another tool in the Lean toolbox. Part of Go and See is being able to immediately recognize problems. We already have something like this in the software engineering industry with build status monitoring (<a href="http://sirenofshame.com/">Siren of Shame</a>!). What else could we visualize? The advantage of the automative industry is the <a href="http://en.wikipedia.org/wiki/Takt_time">takt time</a> is often short (if customers are demanding 10K units a week, the takt time is in minutes). In Software Engineering, our sprints are often weeks. It's difficult to know if things are going off the rails. Perhaps some <a href="http://alistair.cockburn.us/Elephant+Carpaccio+exercise">elephant carpaccio</a> is in order?<br />
<br />
Does go and see translate to software engineering? Definitely for some parts, namely visiting customers and understanding their requirements (<quote>customers want holes, not drills</quote>). Does this apply at other times, such as when teams are writing code? I suspect it does; the only way to understand why teams are flying or struggling is to actually see them in action.<br />
<br />
The last big theme from the book was that developing people is just as important as developing the product. The idea is simply that once *everyone* is contributing to product improvement and innovation then you've built yourself a significant advantage that is almost impossible to copy.<br />
<br />
All in all, The Lean Manager was an enjoyable read. I'm not sure how many of themes adapt perfectly to software engineering, but definitely food for thought!Jeffhttp://www.blogger.com/profile/08195722595923882332noreply@blogger.com0tag:blogger.com,1999:blog-5743983044224833668.post-82191050359102175572014-12-01T00:36:00.001-08:002014-12-01T00:36:31.286-08:00How much time should you spend fixing bugs in legacy code?How much time should you spend fixing bugs in legacy code?<br />
<br />
There's a huge amount written about dealing with greenfield code. You start with practices such as test-driven development, walking skeletons and thin vertical stripes of functionality. Legacy code is much harder. Given hundreds of thousands of lines of poorly structured code, where'd you start? <a href="http://www.amazon.com/Working-Effectively-Legacy-Michael-Feathers/dp/0131177052">Working Effectively with Legacy Code</a> gives some great pointers; put seams in, get the tests in place and TDD the new feature work. I'm interested in the next level up, how do you balance feature work against bug fixing?<br />
<br />
I'm got an interesting problem. We've got a clump of legacy software that product management tell me needs new features, but we also know from support that the number of bugs is a worry. From my point of view as a development manager I want data that allows me to make the right decision and that requires evidence and understanding of the scale and scope of the problem.<br />
<br />
<blockquote>It is impossible to find any domain in which humans outperformed crude extrapolation algorithms, less still sophisticated ones (Expert Political Judgement: How good is it? How can we know? [via How to Measure Anything: Finding the Value of Intangibles in Business)</blockquote><br />
I'd like to move from a faith-based to a science-based approach to balancing new features over bug count.<br />
<br />
One field that provides some inspiration is population estimation. Given a small sample size, how do you estimate the total population?<br />
<br />
<a href="http://en.wikipedia.org/wiki/Mark_and_recapture">Mark and recapture</a> is a common method for population estimation. Capture 100 animals, tag them and release them. Repeat the process. The number of tagged animals is proportional to the number of tagged animals in the population. If we had no morals whatsoever, we could release an update to 1000 users and sample the number of bugs. We could then release the same update to another 1000 users and see how many bugs we see again.<br />
<br />
This isn't a great way to do things, but it does give us some simple formula. If we use the same notation as Wikipedia, then<br />
<br />
<ul> <li>N is the total number of bugs<br />
<li>K is the number of bugs found by the first group<br />
<li>n is the number of bugs found by the second group<br />
<li>k is the number of bugs seen for a second time<br />
</ul>
This gives us a simple formula that we could use (N = Kn / k). For bugs for released products, it's even simpler. Since we can tally bugs against each other automatically, we can estimate N without doing anything too amoral. We can use the data from the latest release to arbitrarily divide the users in half, calculate how many bugs each side finds and count the number of duplicates. <br /><br />
After a bit of searching around, I found that this isn't a very novel application of the idea. "<a href="http://www.johndcook.com/blog/2010/07/13/lincoln-index/">How many errors are left to find?</a>" talks about this, but from the perspective of software testing (this seems to have generated some controversy in the response, "<a href="http://www.developsense.com/blog/2010/07/another-silly-quantitative-model/">Another silly quantitative model</a>").<br />
<br />
There's a lot of caveats with model-based approaches like this (what exactly is a bug anyway?), but it's better than nothing.<br />
<br />
Jeffhttp://www.blogger.com/profile/08195722595923882332noreply@blogger.com0tag:blogger.com,1999:blog-5743983044224833668.post-62811867947153342982014-09-23T23:38:00.003-07:002014-09-23T23:41:58.729-07:00The Goal - The Match Bowl ExperimentRecently I've been re-reading <a href="http://en.wikipedia.org/wiki/The_Goal_(novel)">The Goal</a> by <a href="http://en.wikipedia.org/wiki/Eliyahu_M._Goldratt">Eliyahu Goldratt</a>. It's a great little book about manufacturing plants and how to manage them. It introduces the <a href="http://en.wikipedia.org/wiki/Theory_of_constraints">Theory of Constraints</a> and that's relevant for all software developers for an understanding of why our development processes are structured the way they are.<br />
<br />
Throughout the book it uses games and metaphors to illustrate faulty thinking about interconnected processes. In this post, I'd like to introduce Goldratt's dice game. <br />
<br />
In the dice game there are a number of stations (representing part of a business process). The stations are arranged in a line with the output from one station becoming the input of the next. This arrangement represents a production line. In order to move items through the production line players take it in turn to roll a dice. The number rolled is the maximum you can move to the next station. For example, if you roll a six, but only have three items in your station, then you can only move three to the next station.<br />
<br />
Let's imagine a really simple system with 8 stations that starts with 100 units in the left hand bowl with the aim of producing 100 units in the right hand bowl. Each bowl has the same capacity; it'll produce between 1-6 units each work step.<br />
<br />
<div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgCFatlyjhYCVu7D1aIw96AY_3Zj44BWEnQcBylFkMsqs6Jpu3zbWqPPSMcKWT_bJ7rRCaNQmy877B0jFEorkCJWy7LQkcXPoKX8pLmSmhGPUxzqGYkCS9BR1uQnz1Phv77AhZjtX3wQg8/s1600/circle.gif" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgCFatlyjhYCVu7D1aIw96AY_3Zj44BWEnQcBylFkMsqs6Jpu3zbWqPPSMcKWT_bJ7rRCaNQmy877B0jFEorkCJWy7LQkcXPoKX8pLmSmhGPUxzqGYkCS9BR1uQnz1Phv77AhZjtX3wQg8/s1600/circle.gif" /></a></div><br />
Based on the rules above, what's the flow of work going to look like through the system?<br />
<br />
You might expect the flow to be smooth; each workstation has about 0 items at any one time because as soon as they are produced they move onto the next state. The reality is somewhat different.<br />
<br />
The movie below shows the system processing a set of 100 units. The bottleneck (that is the work centre with the most items in it) is highlighted in yellow.<br />
<br />
<div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEirVjPv2MqSTh89BBCDoY2Cg0PH7Dn7RHyT8Qud3OR_CatI_Y1nsDi1ikM3JBaFMm_BAGsXTB0Vc0LzjuIonCjOGOYqoFuI54t8xkygU8kC-GS34i4aIJ1pC6Hxc3zYtN_Vg60eTqNV6Ek/s1600/circle.gif" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEirVjPv2MqSTh89BBCDoY2Cg0PH7Dn7RHyT8Qud3OR_CatI_Y1nsDi1ikM3JBaFMm_BAGsXTB0Vc0LzjuIonCjOGOYqoFuI54t8xkygU8kC-GS34i4aIJ1pC6Hxc3zYtN_Vg60eTqNV6Ek/s1600/circle.gif" /></a></div><br />
What's really interesting is the chaotic nature of the bottleneck. Random fluctuations mean that the bottleneck can appear anywhere. Balancing capacity across each item is clearly not the right answer.<br />
<br />
All the diagrams in this page were built using the excellent <a href="http://projects.haskell.org/diagrams/">Diagrams library</a> for <a href="http://www.haskell.org/">Haskell</a>.Jeffhttp://www.blogger.com/profile/08195722595923882332noreply@blogger.com0tag:blogger.com,1999:blog-5743983044224833668.post-82106023832036175982014-07-26T04:56:00.001-07:002014-07-26T04:56:35.340-07:00Dynamic Time Warping<a href="http://en.wikipedia.org/wiki/Dynamic_time_warping">Dynamic Time Warping</a> is nothing to do with the <a href="https://www.youtube.com/watch?v=vBHONx9vTtI">Rocky Horror show</a>. It's a <a href="http://en.wikipedia.org/wiki/Dynamic_programming">dynamic programming</a> algorithm for aligning sequences of data that vary in terms of speed or time. Some typical applications of dynamic time warping are aligning fragments of speech for the purposes of performing speaker recognition.<br />
<br />
In this post, we'll look at how simple the algorithm is and visualize some of the output you can get from aligning sequences. The complete code is on <a href="https://github.com/fffej/haskellprojects/blob/master/dynamicTimeWarping/DynamicTimeWarping.hs">github</a> and any flames, comments and critiques are most appreciated. You'll need the <a href="">Haskell platform</a> installed and a cabal install of the <a href="https://hackage.haskell.org/package/bmp-1.2.5.2/docs/Codec-BMP.html">Codec.BMP</a> package if you want to generate some images.<br />
<br />
Given two vectors of some symbol <code>a</code> representing a time series (e.g. they both represent the output f(x) = y where x is some time, and y is an output signal) produce as output an array describing the cost. The output array gives the "alignment factor" of the two sequences at difference points in time.<br />
<br />
<script src="https://gist.github.com/fffej/11b921a329939e8e520a.js"></script><br />
<br />
We can find the best alignment path by simply walking back through the matrix from the top right, to the bottom left and taking the minimum choice at each turn. Using this we can visualize the best matching path for two exactly matching sequences. That's dead simple to code up:<br />
<br />
<script src="https://gist.github.com/fffej/ca5b93da956f8bb6842e.js"></script><br />
<br />
Let's look at what happens if we try match the signal against itself and highlight the matching path in white.<br />
<br />
<div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhfUqVXF2ncj9eo4jfVJvFcrOT1eQ_9o4d3FDk9uhXwx7xY7BnYnWOITkOsJ2b0nkhKG7ElrEyWEDP9pwi0lNTPJ70ByipAKalfhJA-jWPsZqCuUfZoj0yKkvVGsqomPYs1XbABCAnWZf8/s1600/perfectlyaligned.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhfUqVXF2ncj9eo4jfVJvFcrOT1eQ_9o4d3FDk9uhXwx7xY7BnYnWOITkOsJ2b0nkhKG7ElrEyWEDP9pwi0lNTPJ70ByipAKalfhJA-jWPsZqCuUfZoj0yKkvVGsqomPYs1XbABCAnWZf8/s320/perfectlyaligned.png" /></a></div><br />
The colour demonstrates how well the signals match. Blue highlights the best match (e.g. least cost) and hotter colours (such as red) highlight the worst cost. This pattern matches simple intuition. Since the sequences are exactly aligned, we'd expect a path from the top right to the bottom left, and that's what we get.<br />
<br />
What happens if we try to match two completely random signals of integers? First off, let's try with the measure of the cost function being the absolute difference between the values (e.g. the cost function passed in is simply <code>cost x y = abs (x - y)</code>).<br />
<br />
<div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhwJfg3i0mFpGwJqRdNl6aKW3gcOLGOGWB7cUJEFDvA7JS3_h1l7ZrjMHY8idpEkFG_s8AJ1O2iVmeEHUDnginWe3CKMR7qh-lq6JW9guZFVA7Dm6rAkp7lDQkjg4sfhWFHtOYCiMELj-o/s1600/random.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhwJfg3i0mFpGwJqRdNl6aKW3gcOLGOGWB7cUJEFDvA7JS3_h1l7ZrjMHY8idpEkFG_s8AJ1O2iVmeEHUDnginWe3CKMR7qh-lq6JW9guZFVA7Dm6rAkp7lDQkjg4sfhWFHtOYCiMELj-o/s320/random.png" /></a></div><br />
Cool patterns. Does this make sense? I think it does. The best match is at the beginning, before the sequences have diverged. As time goes on the match always gets worse because the cumulative absolute difference between the sequences is continuously increasing (albeit randomly).<br />
<br />
What if we try to match a sequence against its opposite? Let's visualize that:<br />
<br />
<div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgqpT1k49xlqXETLZoceD3fXpyMTIgUhMfFnskblnHDZXGYv-Oi9kTu5x1tR_tdTGvb85khzWBz1NMCUb2a-L-fePakhKFZ4uuQoadv0dkYOew1RRwYKk6cww_OHn8cm-oFhOLCbcICU0E/s1600/opposite.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgqpT1k49xlqXETLZoceD3fXpyMTIgUhMfFnskblnHDZXGYv-Oi9kTu5x1tR_tdTGvb85khzWBz1NMCUb2a-L-fePakhKFZ4uuQoadv0dkYOew1RRwYKk6cww_OHn8cm-oFhOLCbcICU0E/s320/opposite.png" /></a></div><br />
That looks odd. What's the intuition describing the image here? The best match of these two signals occurs in the middle (since they are opposite), this feels like this explains the center structure. By the time we reach the end of the signal (the top right) we've got the worst possible match and hence the brightest colour.<br />
<br />
This implementation of the algorithm isn't all that practical. It's an O(N^2) algorithm and thus isn't suitable for signals with a high number of samples. However, it's fun to play with!<br />
<br />
If you want to find out more about an efficient implementation of dynamic time warping then <a href="https://code.google.com/p/fastdtw/">Fast DTW</a> is a great place to start. As someone who enjoys reading papers, it's fantastic to see the code behind it, quoting from the link:<br />
<br />
<blockquote>FastDTW is implemented in Java. If the JVM heap size is not large enough for the cost matrix to fit into memory, the implementation will automatically switch to an on-disk cost matrix. Alternate approaches evaluated in the papers listed below are also implemented: Sakoe-Chiba Band, Abstraction, Piecewise Dynamic Time Warping (PDTW). This is the original/official implementation used in the experiments described in the papers below.</blockquote>Jeffhttp://www.blogger.com/profile/08195722595923882332noreply@blogger.com0tag:blogger.com,1999:blog-5743983044224833668.post-35870640193055269362014-07-14T00:27:00.000-07:002014-07-14T00:27:25.513-07:00The Stable Marriage ProblemIt's been far too long since I wrote posts with any real code in, so in an attempt to get back into good habits I'm going to try to write a few more posts and read up a bit more about some algorithms and the history behind them.<br />
<br />
The <a href="http://en.wikipedia.org/wiki/Stable_marriage_problem">Stable Marriage Problem</a> was originally described by David Gale and Lloyd Shapley in their 1962 paper, "<a href="http://www.econ.ucsb.edu/~tedb/Courses/Ec100C/galeshapley.pdf">College Admissions and the Stability of Marriage</a>". They describe the problem as follows:<br />
<br />
<blockquote>A certain community consists of n men and n women. Each person ranks those of the opposite sex in accordance with his or her preferences for a marriage partner. We seek a satisfactory way of marrying off all member of the community. We call a set of marriage <i>unstable</i> if under it there are a man and a woman who are not married to each other, but prefer each other to their actual mates.<br />
</blockquote><br />
Gale and Shapley shows that for any pattern of preferences it's possible to find a stable set of marriages.<br />
<br />
On its own, this doesn't sound very interesting. However, bringing together resources is an important economic principle and this work formed part of the puzzle of <a href="http://en.wikipedia.org/wiki/Cooperative_game">Cooperative Game Theory</a> and Shapley was jointly awarded the <a href="http://www.nobelprize.org/nobel_prizes/economic-sciences/laureates/2012/">Nobel Prize for economics in 2012</a>.<br />
<br />
So how does the algorithm for Stable Marriages work? <br />
<br />
Let's start by defining the problem. Given two lists of preferences, find the match such that there is no unstable match (that is two pairs that would cooperatively trade partners to make each other better off). The only constraint the types have is that they have is that they are equatable. This isn't the ideal representation (to put it mildly) in a strongly typed language (it doesn't enforce any invariants about the structure of the lists), but it's probably the simplest representation for explaining the algorithm.<br />
<br />
<pre>stableMatch :: (Eq m, Eq w) => [(m,[w])] -> [(w,[m])] -> [(m,w)]
</pre><br />
The algorithm continues whilst there are any unmarried men. If there are no unmarried men, then the algorithm terminates.<br />
<br />
<pre> stableMatch :: (Eq m, Eq w) => [(m,[w])] -> [(w,[m])] -> [(m,w)]
stableMatch ms ws = stableMatch' []
where
stableMatch' ps = case unmarried ms ps of
Just unmarriedMan -> stableMatch' (findMatch unmarriedMan ws ps)
Nothing -> ps
unmarried :: Eq m => [(m,[w])] -> [(m,w)] -> Maybe (m,[w])
unmarried ms ps = find (\(m,_) -> m `notElem` engagedMen) ms
where
engagedMen = map fst ps
</pre><br />
If there is at least one unmarried man, then we need to find a match. We do this by proposing to each of his preferences in turn. If his first preference is not engaged, then we propose. Otherwise, if his potential partner is already engaged and would prefer him then this violates the stable marriage principle and we breakup the engagement and re-engage with our first choice.<br />
<br />
<pre>findMatch :: (Eq m,Eq w) => (m,[w]) -> [(w,[m])] -> [(m,w)] -> [(m,w)]
findMatch (m,w:rest) ws ps = case isEngaged w ps of
-- w is already engaged to m' - is there a better match?
Just m' -> if prefers (getPrefs ws w) m m'
then engage (breakup m' ps) m w
else findMatch (m,rest) ws ps
-- can match with first choice
Nothing -> engage ps m w
</pre><br />
You can see the full code at <a href="https://gist.github.com/fffej/1e90e37b68dced82f5e8">Stable Marriage Problem</a>. As always flames, comments and critiques gratefully received.<br />
Jeffhttp://www.blogger.com/profile/08195722595923882332noreply@blogger.com0tag:blogger.com,1999:blog-5743983044224833668.post-11139471086144541402014-06-12T01:54:00.001-07:002014-06-12T01:54:54.393-07:00Getting the most out of Extract Class<a href="http://www.jetbrains.com/resharper/">Resharper</a> is a wonderful tool. I can't imagine working in the horribleness of legacy code without it.<br />
<br />
Every so often you come across a little workflow that makes slicing and dicing code either. For example, before you could "Move Instance Method" you could "Make Static", "Move Method" and "Make instance method". Knowing you could do this made tearing code apart easier.<br />
<br />
Recently I've been using "Extract class" a million and one times to deal with one of those 10K line long classes that no-one ever admits to having. The classes in question are in this:<br />
<br />
<pre>class DoesEverythingAndThenSome {
private ThingToDoWithA1 m_ThingToDoWithA1;
private ThingToDoWithA2 m_ThingToDoWithA2;
private ThingToDoWithA3 m_ThingToDoWithA3;
private ThingToDoWithB1 m_ThingToDoWithB1;
private ThingToDoWithB2 m_ThingToDoWithB2;
private ThingToDoWithB3 m_ThingToDoWithB3;
// repeat for thousands of other "things"
public void DO_ALL_THE_THINGS_WITH_A () {
}
public void DO_ALL_THE_THINGS_WITH_B () {
}
// thousands of lines of random shit
public void example_of_random_shit () {
if (incrediblyComplicatedCondition()) {
for (var apples in bananas) {
m_ThingsToDoWithA1.SomethingImportant();
}
} else if (auberginesAreLumpy()) {
m_ThingsToDoWithB.SomethingElse();
}
}
}
</pre><br />
Obviously I want to extract out the responsibilities to do with <code>A</code> and <code>B</code> into separate class. Using "Extract Class" directly doesn't work because I can't pull out the references in <code>example_of_random_shit</code> without making properties public and introducing back references from the extracted class to the parent class.<br />
<br />
The simple refactoring is to just extract out each line to do with each field into a single method.<br />
<br />
<pre>class DoesEverythingAndThenSome {
/* the rest is the same as above */
public void SomethingToDoWithA() {
m_ThingsToDoWithA1.SomethingImportant();
}
public void SomethingToDoWithB() {
m_ThingsToDoWithB.SomethingElse();
}
// thousands of lines of random shit
public void example_of_random_shit () {
if (incrediblyComplicatedCondition()) {
for (var apples in bananas) {
SomethingToDoWithA();
}
} else if (auberginesAreLumpy()) {
SomethingToDoWithB();
}
}
}
</pre><br />
Once I've completed this simple refactoring, "Extract Class" can now do the heavy lifting and I can move all the fields, and all the functions across in a single refactoring. What was previously hard (unpicking the back references) is now incredibly simple and I end up with a mechanical transformation to get data and functions in the right place. Extract class will now give me:<br />
<br />
<pre>class ThingsToDoWithA {
private ThingToDoWithA1 m_ThingToDoWithA1;
private ThingToDoWithA2 m_ThingToDoWithA2;
private ThingToDoWithA3 m_ThingToDoWithA3;
// snip constructor
public void DO_ALL_THE_THINGS_WITH_A () {
}
public void SomethingToDoWithA() {
m_ThingsToDoWithA1.SomethingImportant();
}
}
class DoesEverythingAndThenSome {
private ThingToDoWithA m_ThingToDoWithA;
private ThingToDoWithB1 m_ThingToDoWithB1;
private ThingToDoWithB2 m_ThingToDoWithB2;
private ThingToDoWithB3 m_ThingToDoWithB3;
// snip constructor
// repeat for thousands of other "things"
public void DO_ALL_THE_THINGS_WITH_B () {
}
// thousands of lines of random shit
public void example_of_random_shit () {
if (incrediblyComplicatedCondition()) {
for (var apples in bananas) {
m_ThingsToDoWithA.SomethingImportant();
}
} else if (auberginesAreLumpy()) {
m_ThingsToDoWithB.SomethingElse();
}
}
}
</pre><br />
Repeat this mechanically for all the other responsibilities Then the hard work begins of actually working out sensible names...Jeffhttp://www.blogger.com/profile/08195722595923882332noreply@blogger.comtag:blogger.com,1999:blog-5743983044224833668.post-72709995949463895912014-04-11T04:05:00.003-07:002014-04-11T05:39:00.867-07:00How does it feel to give a terrible conference talk?Have you been to a conference and sat through an awful presentation and wondered just how the hell someone got there? Me too!<br />
<br />
Recently I attended the <a href="http://www.accu.org/">ACCU</a> conference in Bristol and got to experience what it feels like to deliver something that went down like a lead balloon. One evening many moons ago, I thought I'd send in a proposal. By some small miracle I got accepted and was all set to run a 90 minute introduction to Haskell.<br />
<br />
I'd already run through the workshop once at a local user group. The material isn't amazing, but I was confident in delivering it and thought it offered people a chance to get a taste of Haskell and programming with functions.<br />
<br />
Then the problems started. It's ACCU. It's full of clever people, therefore I should level-up the material and assume more knowledge. Right? I should make it more hands-on, more interactive and better in every way.<br />
<br />
I prepared hard. I updated the slides. I added more and more. I wrote notes, I dug references and I was confident it would kick-ass.<br />
<br />
And then the day arrived.<br />
<br />
90 minutes seem like a long time. It isn't. I spent a good 15 minutes ensuring that everyone could run "hello world". Very rapidly 90 minutes because 60 minutes.<br />
<br />
Then my cleverness got the better of me. The <a href="https://en.wikipedia.org/wiki/Curry%E2%80%93Howard_correspondence">Curry-Howard isomorphism</a> is fascinating, but perhaps it's not the best subject matter within the first 30 minutes of any presentation. Trying to explain it under pressure with questions from an audience eager to learn makes it even worse. I probably lost another 20 minutes trying and failing to explain that <code>const :: a -> b -> a</code> only has one valid implementation in <a href="http://www.haskell.org/">Haskell</a>. And what the hell are the poor attendees going to do with this information? GAH!<br />
<br />
And so it continued. On to writing some code. I'd wanted to make it easier to compose higher order functions to produce results, so I'd made the initial data structures in the exercises a bit more complicated than those I'd shown in the example slides. Big mistake. This made it much harder for people to grok the syntax; I'd shown simple syntax but not given enough direction. 30 minutes rapidly disappeared and I'm now *way* behind schedule.<br />
<br />
At this point, I'd already realized the situation was going Pete Tong. But what'd you do? You can't just down tools and walk out the room (well, I suppose you could, but that'd be worse), so you just have to knuckle down and carry on. And carry on I did, through more examples (well over-egged) and then onto the Universality of Fold (brain, what the hell are you thinking?!). <br />
<br />
With 5 minutes left, there's plenty of time to through a demo of <a href="http://www.haskell.org/haskellwiki/Introduction_to_QuickCheck2">QuickCheck</a> in, right?But then, I realized I'm in an Emacs buffer. How'd I increase the font-size so people can read it? GNARGH!! It's over to Notepad and bump the fonts up in that. "Should have used vi!" went the audience. ARGH!<br />
<br />
And then the buzzer sounds (well, not really, but it's time to go). Bring things to a halt and escape to a corner of the building. I can't imagine that was particularly fun for the participants. A few people kept up (hurrah!) and there were a couple of positive things said, but I knew it'd gone wrong and it boy that doesn't feel good.<br />
<br />
So, at least now I know how it feels (bad, very bad) and I also learnt an important lesson. Keep the message simple! Focus on the single takeaway you want participants to have. I wanted people to leave knowing that Haskell isn't impenetrable and looking at how far you can get just by reading type signatures. However, I lost this in a noise of other random related things and tried (and failed) to communicate a million and one other features. <br />
<br />
<a href="http://en.wikipedia.org/wiki/KISS_principle">KISS</a>!<br />
<br />
Jeffhttp://www.blogger.com/profile/08195722595923882332noreply@blogger.com0tag:blogger.com,1999:blog-5743983044224833668.post-23580515111336160422014-04-09T00:18:00.002-07:002014-04-11T05:39:48.700-07:00Agile - What Next?I'm at <a href="http://www.accu.org/">ACCU</a> at the moment, and instead of preparing my talk on <a href="http://www.haskell.org/">Haskell</a> for Thursday, I'm writing up my notes from Bob Martin's talk on agile yesterday.<br />
<br />
Agile was originally founded by a bunch of programmers over a decade ago. The aim (from Kent Beck) was to devise a system that eliminated the trust divide between programmers and managers (them and us). Transparency was the aim of the game. Programmers would record velocity using story points. Managers would track number of story points per sprint and produce burn-down charts. Everyone is happy.<br />
<br />
Unfortunately, burndown and velocity charts track only one part of software development, features. There's a hidden part of software development that isn't captured by these charts, ability to change. If there's one thing for certain in software development it's that people will change their mind and features will need to adapt. It's no good having your software with the correct features today, if it can't have the correct features tomorrow. Arguably, a code bases ability to respond to change is the primary responsibility of the developers.<br />
<br />
In the original light-weight process, <a href="http://www.extremeprogramming.org/">XP</a>, this was kept in check by Ron Jefferies concentric circles.<br />
<br />
<img width="339" height="279" src="http://www.xprogramming.com/images/circles.jpg" alt="Concentric Circles"><br />
<br />
This, again, is part of transparency and trust. At the inner-level, TDD, pair-programming and simple design keep the software honest. A suite of tests gives transparency on the system functionality. Moving further out we reinforce these practices with collective ownership (transparency again, no siloed development). And so on, and so forth.<br />
<br />
Fast-forward a decade or so, and where are we now? Agile is the domain of the manager. There are no developers at agile conferences any more, it's all about the secondary value of a software product (shipping features) rather than the primary ability (reacting to change).<br />
<br />
The XP Practices have been forgotten. Scrum empowers teams to take ownership of their practices and opt out of ones that don't work. Of course, it's easier (in the short run!) to forget about TDD, simple design and refactoring. However, in the long run productivity grinds to a halt (see <a href="http://martinfowler.com/bliki/DesignStaminaHypothesis.html">Design Stamina Hypothesis</a>).<br />
<br />
Bob argues (<a href="http://blog.8thlight.com/uncle-bob/2014/03/28/The-Corruption-of-Agile.html">The Corruption of Agile</a>) that agile doesn't exist without the practices that support it. I agree; most agile teams aren't agile in their ability to react to change. Martin Fowler has a term for it "<a href="http://martinfowler.com/bliki/FlaccidScrum.html">Flaccid Scrum</a>" where we adopt the project management side of it, but not the underlying practices for ensuring that the code base becomes malleable and responsive to change.<br />
<br />
With all this in mind, the trust issues have reemerged. Dropping the velocity (number of story points per sprint) is a bad idea, so developers have rebelled. Let's just make the stories smaller. The points counted are the same, but the size of the stories is much smaller. Teams are wading through custard, developing features just as slowly as ever.<br />
<br />
The thrust against this has come in the form "software craftsmanship". This is trying to reimagine the circles from the inside out, but it's failed. It's failed because it doesn't attempt to bridge the divide between the managers and the coders. It might help the engineers to "do the right thing" more often, but it doesn't show transparency.<br />
<br />
And the talk ended there, no answers for the future and a little depressing. I've definitely seen the scenarios Bob describes, but what's the solution? It's probably not "kill all the project managers" as someone suggested. I'd love to make the "ability to change" a tangible concept that teams can explore and understand. It's not an easily measured property, but I think taking data-driven decisions about code is part of the answer. Project managers need options to meet business constraints. Sometimes it's OK to go quick and dirty, to spike a feature that may not live longer than a week, but you have to accept that the remedial cost of recovering from that burst of activity exists and understand the remedial cost.<br />
<br />
Right, now to finish off a few slides for this Haskell thing.Jeffhttp://www.blogger.com/profile/08195722595923882332noreply@blogger.com0tag:blogger.com,1999:blog-5743983044224833668.post-19327441010293728062014-02-01T13:22:00.001-08:002014-02-01T13:22:32.922-08:00The First International Conference on Software Archaeology <p>I recently attended <a href="http://ticosa.org/">The First International Conference on Software Archaeology</a>, much more memorably shorted to <a href="https://twitter.com/search?q=%23ticosa">#ticosa</a>.<br />
<br />
<p>It was a slightly strange conference, in that it was never particularly clear what software archaeology was, but that was a good thing as it gave a great variety of talks encompassing everything from metrics, to tools for understanding, to philosophical thoughts on the architecture of information.<br />
<br />
<h4>Process Echoes in Code</h4><br />
<p>Michael Feathers opened the proceedings with a question, what's the real point of version control systems? The most common answer is that VCS systems help you roll back to previous revisions should something go wrong, or support multiple different product lines. The truth is this doesn't really happen. If your team deploys something to production that goes wrong, then I imagine you'll revert the deploy (not the VCS) and simply deploy again. The real purpose of source control is providing change logging. By looking at those changes we can see the traces of the way we work that are indelibly written in the version control system. <br />
<br />
<p>Michael demonstrated a tool (<a href="https://github.com/michaelfeathers/delta-flora">delta-flora</a>) to explore the traces left in the source code. The tool was a simple <a href="http://www.ruby-lang.org/">Ruby</a> program that mapped the Git commit history (SHA1, files changed, author, code diff) into Method event objects (methods added, changed and modified). This is a simple transformation, but one that seems to yield a vast amount of useful information.<br />
<br />
<p>Exploring the temporal correlation of class changes seems like an incredibly useful way of identifying an area of related objects. I'm working on a large, badly understood code-base. We're already finding that adding features requires touching multiple files. By mining information from the past, maybe we can make more educated decisions in the future?<br />
<br />
<p>Another area Michael mentioned that sent my synapses firing was analysing classes by closure date. Even if you have a huge code=base, identifying the closed classes (those that haven't changed) helps reduce the surface area you have to understand. One particular graph he showed (graphing the set of active classes against the open classes) was particularly interesting.<br />
<br />
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjmcY11oijnIPD3pdFNNdpaJfBNcfZ_5L0cV-XLeez_1MoF06qWqvHnB-GCeWGTx_4u80e8p04SSKV5Ji6GAusxb3RyJlGWvcrWVfL0H9kwGSZR8nXVGKdDMkLUqp0sSibmfxlFi9-wb68/s1600/open_closed.png" imageanchor="1" ><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjmcY11oijnIPD3pdFNNdpaJfBNcfZ_5L0cV-XLeez_1MoF06qWqvHnB-GCeWGTx_4u80e8p04SSKV5Ji6GAusxb3RyJlGWvcrWVfL0H9kwGSZR8nXVGKdDMkLUqp0sSibmfxlFi9-wb68/s320/open_closed.png" /></a><br />
<br />
<p>I'd love to plot this on a real code-base, but my understanding is that whilst you've got open classes, chances are you haven't finished a feature and the code-base is in an unstable phase. Looking forward to trying this one out.<br />
<br />
<h4>Are you a lost raider in the code of doom?</h4><br />
<p>Daniel Brolund followed with a quick overview of the <a href="http://mikadomethod.org/">Mikado Method</a>. The Mikado Method provides a pragmatic way of dealing with a big ball of mud. We've probably all experienced the "shockwave" refactoring (or refucktoring?) where we've attempted to make a change, only to find that change requires another change, then another and before you know it you have a change set with 500 files in and little or no confidence that anything works. <br />
<br />
<p>The Mikado Method helps you tackle problems like this by recognizing that doing things wrong and reverting is not a no-op. You've gained knowledge. Briefly the method seems to consist of trying the simplest possible thing, using the compiler and more to find pre-requisites (e.g. If only that class was in a separate package...). By repeatedly finding the dependent refactorings you can arrange a safe set of refactorings to tackle larger problems.<br />
<br />
<p>I completely agree with this approach. Big bang refactorings on branches are no longer (if they ever were!) acceptable ways to work. Successful refactoring keeps you compiling and keeps you working in the smallest possible batch size. I liked the observation that the pre-requisites form a graph; before I've worked in pairs where we've kept a stack of refactorings (the <a href="http://en.wiktionary.org/wiki/yak_shaving">Yak</a> stack?) but it's an interesting observation that sometimes it's a graph.<br />
<br />
<h4>How much should I refactor?</h4><br />
<p><a href="http://blog.mattwynne.net/">Matt Wynne</a> gave a great metaphor for keeping code clean. If you imagine that software engineers are chefs and their output is meals, then the code base is the kitchen. What does your kitchen look like?<br />
<br />
<p>Matt had an exemplar code base (Cucumber rewrite), created as greenfield code, test-first, small-team, small commits and no commercial pressures. By analysing commits, a rough and ready guess was that 75% of commits were pure refactoring.<br />
<br />
<p>In answer to the question, how much should I refactor? The answer is simple.<br />
<br />
<blockquote>More than you currently do.</blockquote><br />
<h4>Code Metrics</h4><br />
<p><a href="http://peripateticaxiom.blogspot.co.uk/">Keith Braithwaite</a> gave us a talk about metrics and in particularly the dangers of not knowing what you are doing.<br />
<br />
<p>He gave some examples from earlier analysis that (allegedly) demonstrated that TDD exhibited bigger methods than test last. This doesn't fit our intuition and indeed analysing the results showed that they based the results on the mean. If we plot method length distribution, we'd find it's not a <a href="http://en.wikipedia.org/wiki/Normal_distribution">normal distribution</a> but a <a href="http://en.wikipedia.org/wiki/Power_law">power-law distribution</a>. Doing a more statistically sound analysis actually gives the opposite results.<br />
<br />
<p>The moral of the story for me was that reducing a data set without knowing what you are doing is very dangerous!<br />
<br />
<h4>Visualizing Project History</h4><br />
<p><a href="https://twitter.com/dmitrykandalov">Dmitry Kandalov</a> showed us an amazing analysis of a number of open source projects by mining the version control history (see <a href="http://dkandalov.github.io/code-history-mining/idea.html">here</a>). This was the highlight of the conference for me, seeing interactive history of real code bases. Neat!<br />
<br />
<p>I really enjoyed seeing the way <a href="http://www.scala-lang.org/">Scala</a> and <a href="http://www.clojure.org/">Clojure</a> have evolved. Scala has progressively added more complexity and more code. Clojure however, has stabilised. Draw from that what you will!<br />
<br />
<h4>Tools for Software Business Intelligence</h4><br />
<p><a href="http://stephane.ducasse.free.fr/">Stephane Ducasse</a> gave us an overview of some of the tools he used for software business intelligence. There was a call to action that we need dedicated tools for understanding code bases and I couldn't agree more with that. There were many interesting links:<br />
<ul><li><a href="http://www.moosetechnology.org/">Moose technology</a><br />
<li><a href="http://synectique.eu/">Synectique</a><br />
</ul>
<h4>Understanding Historical Design Decisions</h4>
<p>Stuart Curran gave a presentation on "Understanding Historical Design Decisions". Stuart's perspective was very different as he comes from an information architecture / design background and didn't consider himself a programmer.
<p>Some books to add to my ever-growing reading list:
<ul> <li><a href="http://www.amazon.co.uk/Information-Architecture-World-Wide-Web/dp/0596527349/">Information Architecture for the WWW</a></li>
<li><a href="http://www.slideshare.net/reduxd/beyond-the-polar-bear">Beyond the Polar Bear</a><br />
<li><a href="http://intersectionbook.com/">Intersection</a><br />
<li><a href="http://www.amazon.co.uk/The-Apollo-Guidance-Computer-Architecture/dp/1441908765/">The Apollo Guidance Computer</a><br />
<li><a href="http://www.amazon.co.uk/Information-James-Gleick/dp/0007225741/">The Information</a><br />
</ul>
<h4>Confronting Complexity</h4>
<p><a href="https://twitter.com/robsmallshire/">Robert Smallshire</a> gave a talk on Confronting Complexity and returned us back to metrics (see also notes from <a href="http://www.fatvat.co.uk/2013/10/software-architect-2013-day-1.html">Software Architect 2013</a>).
<p>We started by analysing how to calculate <a href="http://en.wikipedia.org/wiki/Cyclomatic_complexity">cyclomatic complexity</a>. One interesting observation was that cyclomatic complexity gives us a minimum bound on the number of tests we need to get code coverage. If you follow this through, then if you add a conditional statement once every fifth line then every five lines of code you write demands another test. Ouch.
<p>We looked at a simpler proxy for code complexity, Whitespace Integrated over Lines of Text (WILT). This is a really simple measure and incredibly quick to calculate so it lends itself to visualizing code data quickly.
<p>There was a really good quote attributed to Rob Galankis (technical director at Eve Online):
<blockquote>How many if statements does it take to add a feature?</blockquote>
<p>Again, this comes back to one of the recurring themes of the conference, Bertrand Meyer's <a href="http://en.wikipedia.org/wiki/Open_Closed_Principle">open-closed principle</a>. One of my takeaways from this was to pay much more attention to OCP!
<p>Rob mentioned that Refactoring Reduces Complexity and gave the example of "Replace switch with polymorphism". I'd agree with this for the most part, but there are exceptions. Rename for example preserves code complexity, but increases code comprehensibility: the two don't always align. It'd be interesting to hook in a plugin to refactoring tools to calculate WILT before and after refactorings and report on the cumulative benefits.
<p>Rob finished off by presenting an alternative model-driven approach to software engineering. The visualizations were neat and helped show the range of possibilities. That immediately seems like an improvement over other models such as <a href="http://en.wikipedia.org/wiki/COCOMO">COCOMO</a>. Interestingly, going back to COCOMO shows that developer half-life isn't considered in the model, nor is complexity of the code produced (I guess the assumption is that complexity of the product => complexity of the code?).
<h4>Lightning Talks</h4>
<p>Finally, we ended up with a set of lightning talks. <a href="https://twitter.com/natpryce/">Nat Pryce</a> gave a quick demo of using <a href="http://www.neo4j.org/">neo4j</a> to analyse a heap dump. Graph databases are cool!
<p>Ivan Moore gave a few opinions on how you can protect your software for archaeologists from the future.
<ul> <li>Ship your source with your product</li>
<li>Put your documentation be in source control</li>
<li>Put your dependencies in source control (reminded me of <a href="http://blog.ploeh.dk/2014/01/29/nuget-package-restore-considered-harmful/">nuget package restore considered harmful</a>)</li>
<li>Make sure you put instructions to build the product in source control (chef!)</li>
</ul>
<p>There was a presentation towards the end that showed how adding sound to a running program (initially for the purposes of accessibility) produced some interesting effects. I've done this kind of thing before (creating animations for log files). Sometimes you can just rely on your brain to find the interesting things when you present it in another way.
<h4>Conclusion</h4>
<p><a href="http://www.ticosa.org/">TICOSA</a> was a great conference. There was a good line up of speakers and lots of interesting content to muse over. What would I like to see next year? I'd really like to hear more war stories. I'd love to hear stories of archaeological digs. I'd especially love to hear about restorations. My general impression is that very few code bases start a restore process and come out better at the end (usually you hear about the big rewrite and sometimes <a href="http://www.joelonsoftware.com/articles/fog0000000069.html">those fail too</a>), but I'd love to hear otherwise!
<p>I'm looking forward to getting back to work on Monday and scraping through the commit logs to see what I can uncover!<br />
Jeffhttp://www.blogger.com/profile/08195722595923882332noreply@blogger.com0tag:blogger.com,1999:blog-5743983044224833668.post-24542002531476004132013-11-01T02:18:00.000-07:002013-11-01T02:18:17.318-07:00The Trouble with Scrum<p>Scrum. An iterative and incremental agile software development framework. It’s full of buzzwords. It frees us from the tyranny of waterfall development (not to say that ever existed anywhere anyway ). It’s based on the premise that the customer doesn't know what they want; we iterate quickly and deliver every sprint. We communicate with the customer often, inspect and adapt, and we build what the customer wants.</p><p>Sorted. We fired the <a href="http://en.wikipedia.org/wiki/No_Silver_Bullet">silver-bullet</a> and scored a direct hit.</p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgJ61Qvedj9c5yEZpxEbJPZexSBwJaQzljhDjr4DwEN5DmCu05haD3sUS6iYxIB9iKTa1zogfXu1EAKY0zVcTR1uXqjJIHDMIINhG7R324hNiPmDEGR6aU_V4cOadthch-S0qVE2nyvfBw/s1600/Untitled.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgJ61Qvedj9c5yEZpxEbJPZexSBwJaQzljhDjr4DwEN5DmCu05haD3sUS6iYxIB9iKTa1zogfXu1EAKY0zVcTR1uXqjJIHDMIINhG7R324hNiPmDEGR6aU_V4cOadthch-S0qVE2nyvfBw/s320/Untitled.png" /></a></div><p>Well no. Some things don't fit into our nicely delineated sprints. Where does user experience fit into a two-week sprint? Where does architecture fit? Where does overall product quality fit in?</p><p>Stories that we can’t estimate are known as epics. According to Scrum terminology, there’s nothing intrinsically wrong with epics, as long as they aren't high priority (!). We can’t directly work on epics. We can't put a story like "Fix User Experience" on the board. Project managers would go insane, blood would be spilt.</p><p>So what do we do? Well, if your experience is anything like mine then we try to break a story up into smaller bits. Perhaps we break “Great User Experience” into pithy little tasks such as "Move Button" and "Improve Dialog". Maybe the great architecture revisiting is broken into a small prod into the right direction. "Extract class for FooBar" or "Break X and Y Dependency".</p><p>Do these small tasks make sure that we get the best user experience? Do these tasks make sure that we've got an architecture to support the needs of the code over the next few months?</p><p>Of course not.</p><p>How do we make sure that cross-cutting concerns like user experience, quality and architecture are given adequate attention in an iterative development environment? I’m not sure I have the answer (I'm not sure that anyone does), but I have a suggestion.</p><p>A clearly communicated vision.</p><p>It doesn't matter whether it's user experience, product quality or software architecture. A clearly communicated vision gives you a tool for making the right decisions as you build software.</p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEge-qIUn2HxkI5oVIWR7U8PCxREhpR10NMC5Q73W6rK7JVu8BpluKG9ws3H06lFBwCSQPZYoKdXNNrVX-xxbp8SIvuTn9uHluPfLIlqExNkNlCDT-enIDBNo8hoKKXp1jtqgZM_JUD7J_A/s1600/map.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEge-qIUn2HxkI5oVIWR7U8PCxREhpR10NMC5Q73W6rK7JVu8BpluKG9ws3H06lFBwCSQPZYoKdXNNrVX-xxbp8SIvuTn9uHluPfLIlqExNkNlCDT-enIDBNo8hoKKXp1jtqgZM_JUD7J_A/s320/map.jpg" /></a></div><p>Am I suggesting the dreaded "Big Design Up Front"? No! It doesn't need all the minutiae just enough to navigate in the right direction. You might say, Just Enough Design Upfront.</p>Jeffhttp://www.blogger.com/profile/08195722595923882332noreply@blogger.com0tag:blogger.com,1999:blog-5743983044224833668.post-90199689272629047012013-10-30T01:24:00.000-07:002013-10-30T01:24:09.125-07:00Not all names are created equalI think everyone agrees that <a href="http://agilelikeacat.com/2013/05/13/old-possums-guide-to-programming/">naming things</a> is one of the hardest things you can do? Books like <a href="http://www.amazon.com/Clean-Code-Handbook-Software-Craftsmanship/dp/0132350882">Clean Code</a> devote whole chapters to naming. Names should convey meaning so that the next person reading the code has an easier job understanding what it does. After all, we read code far more than we write it. It's definitely OK to spend some time arguing about the right name. It's important.<br />
<br />
So that's it. Names are important. Job done? Of course not! There's more to the story than that.<br />
<br />
At <a href="agilecambridge.net/ac2013/">Agile Cambridge 2013</a>, I attended a session (<a href="http://www.fatvat.co.uk/2013/09/agile-cambridge-2013-day-1.html">Unpicking the Haystack</a>) where the source code was only available from decompiled byte code (some sad story involving not using version control, not backing up and all the things that no-one ever does). Our task was to recover what the original program actually did. When we're looking at decompiled code almost all the naming information has gone. By the time you've gone source code to binary to source code you've lost variable names. Unsurprisingly, trying to decipher what the code does with local variables such as <code>a1</code> to <code>a999</code> is very hard.<br />
<br />
With variable names gone, we have to look for other clues for programmer intent. So what else is there? Well, it certainly helps that <code>public</code> methods aren't lost. In this respect, method names are more important to get right than variable names. The naming is stickier. But something far more important gives us even more clues about this mystery code base.<br />
<br />
Enter types. Decompilation reveals the names of public types. Names of types can show much more information than the variable name. For example, <code>string s</code> reveals little, whereas <code>URL s</code> reveals much more. If we're disciplined followers of <a href="http://en.wikipedia.org/wiki/Domain-driven_design">domain-driven design</a> then our types align with the problem we are solving. I'd say that types sit right at the top of the most-important-things-to-name-correctly hierarchy.<br />
<br />
In this view of decompiled code, some names are more important than others. Parameter names and local variables are least important, whereas type names are the most important (with methods a close second).<br />
<br />
Coming at names from decompiled source is certainly a weird way to do it, but this seems to fit with Bob Martin's definition of name length.<br />
<br />
<blockquote class="twitter-tweet"><p>The length of a variable name should be proportional to its scope. The length of a function or class name is the inverse.</p>— Uncle Bob Martin (@unclebobmartin) <a href="https://twitter.com/unclebobmartin/statuses/39333982050975744">February 20, 2011</a></blockquote><script async src="//platform.twitter.com/widgets.js" charset="utf-8"></script><br />
<br />
I'd like to try to reinforce the view that types are by far the most important thing to get right. Crisply named abstractions matter more than almost anything else. To explore this area, we'll look at a strongly-typed static language, <a href="http://www.haskell.org/">Haskell</a>, and explore just enough syntax to understand its types. But first...<br />
<br />
What is a type? A type is a label that describes properties of all objects that are instances of this type. If you see <code>string</code> in C#, you know you are getting an immutable set of characters with certain methods available. If you see a <code>AbstractSingletonFactoryVisitorBean</code> then you know you've got problems. I'm kidding.<br />
<br />
Anyway, back to sensible types. <a href="http://www.fatvat.co.uk/2012/07/on-types-and-tests.html">Types describe program behaviour</a>. Don't believe me? Let's begin our detour into Haskell:<br />
<br />
<code><br />
-- Whenever you see "::" replace it with "is of type"<br />
-- When you see a capital letter variable then you've got a type<br />
-- add5 is of type Int, returning Int<br />
add5 :: Int -> Int <br />
add5 x = x + 5<br />
<br />
-- Parameters are separated by -><br />
-- For the purposes of this, let's just say the last one is the return<br />
-- type and the rest are the arguments<br />
-- add is of type (Int -> Int) returning Int<br />
add x y :: Int -> Int -> Int<br />
add x y = x + y<br />
<br />
-- Generics are represented with lower case paramets<br />
-- middle is of type three generic parameters (a,b,c) returning b<br />
middle :: a -> b -> c -> b<br />
middle x y z = y<br />
</code><br />
<br />
Let's look at that last one again. <code>middle :: a -> b -> c -> b</code>. From the name we might guess that it returns the middle argument (e.g. <code>middle 1 2 3</code> returns <code>2</code>). Is there any other definition of what the function could do? In Haskell, there's no such thing as type-casting, if all I know is that something could be any type, there's not many options. I can't add anything to it. I can't convert it to a string. In fact, I can't do anything with it other than return it. The types don't let me. Types constrain the implementation choices to a more sensible subset.<br />
<br />
Do the names matter? We know that the argument <code>x</code> has type <code>a</code>. Is there any more descriptive name? Probably not, from the type we have no idea what properties hold for the types so a long descriptive name is just wasting space. For all we know, the argument could be a function. Or it could be a monad. What are you going to call it? <br />
<br />
Is the method name important? It's definitely nice to have a good name, but is it essential? If I gave you <code>quux :: a -> b -> a</code> I'm betting you could tell me what it does?<br />
<br />
In fact, armed with just a little knowledge about types you can start to infer what functions do without even needing to see their definition. Here's a few random functions with really poor names; what do they do?<br />
<br />
<code><br />
bananaFactory :: a -> a<br />
<br />
-- (a,b) is a tuple of two elements of type a and type b<br />
spannerBlender :: (a,b) -> a <br />
<br />
-- (a -> b) is a function taking anything of type a and returning type b<br />
-- [a] is a list of items of type a<br />
omgWTF :: (a -> b) -> [a] -> [b]<br />
<br />
-- "Num a =>" says a must be an instance of the Num typeclass<br />
-- think of this as specifying an interface<br />
-- boing is a function of type taking two numbers and returning a number<br />
boing :: (Num a) -> a -> a -> a<br />
<br />
-- m is a type constructor that takes an argument of any type a<br />
mindBlown :: (a -> b) -> m a -> m b<br />
</code><br />
<br />
Armed with this basic knowledge of reading Haskell type signatures, you're now equipped to use <a href="http://www.haskell.org/hoogle/">Hoogle</a>. You can search for the type signatures given above (<a href="http://www.haskell.org/hoogle/?hoogle=a+-%3E+a">a -> a</a>, <a href="http://www.haskell.org/hoogle/?hoogle=%28a%2Cb%29+-%3E+a">(a,b) -> a</a>, <a href="http://www.haskell.org/hoogle/?hoogle=%28a+-%3E+b%29+-%3E+%5Ba%5D+-%3E+%5Bb%5D">(a -> b) -> [a] -> [b]</a> and <a href="http://www.haskell.org/hoogle/?hoogle=%28a+-%3E+b%29+-%3E+m+a+-%3E+m+b">(a -> b) -> m a -> m b</a>) and get a good idea of what these functions do.<br />
<br />
So that's why I think long variable names are less common in functional programming. It's because the languages are terser (Uncle Bob's rule still applies) and because the type signature gives you the power of reasoning, not the variable names.<br />
<br />
<blockquote class="twitter-tweet"><p><a href="https://twitter.com/kevfromireland">@kevfromireland</a> <a href="https://twitter.com/hairybreeches">@hairybreeches</a> variable names matter much less in FP because types. <a href="https://twitter.com/search?q=%23controversialopinion&src=hash">#controversialopinion</a></p>— Jeff Foster (@fffej) <a href="https://twitter.com/fffej/statuses/393807522194530304">October 25, 2013</a></blockquote><script async src="//platform.twitter.com/widgets.js" charset="utf-8"></script><br />
<br />
Names are important; but not all names are equally important.Jeffhttp://www.blogger.com/profile/08195722595923882332noreply@blogger.com0tag:blogger.com,1999:blog-5743983044224833668.post-74039738480535641672013-10-10T15:26:00.000-07:002013-10-10T15:26:20.697-07:00Software Architect 2013 Day #2<h4>What's wrong with current software architecture methods and 10 principles for improvement</h4>
<p><a href="http://www.gilb.com/">Tom Gilb</a> showed us a heck of a lot of <a href="http://gilb.com//dl603">slides</a> and tried to convince us that we must take architecture seriously. I don't disagree with this, our industry could definitely do with a bit more rigour. <a href="https://twitter.com/imtomgilb">Tom</a> was very forthright in his views, and I appreciated his candour.</p>
<blockquote>The system should be scalable, easy customizable and have a great user interface</blockquote>
<p>That's a typical "design constraint" that we're probably all guilty of saying. This is nothing more than architectural poetry (putting it politely) or complete and utter bullshit. In order to take architecture seriously we should measure. Architecture is responsible for the values of the system. We should know these values and be able to measure them. If a given architecture isn't living up to these values, we should replace it with something that does. Architecture exists solely to satisfy the requirements.</P>
<p>Real architecture has multi-dimensional objectives, clear constraints, estimates the effects of changes. Pseudo-architecture has no dedication to objects and constraints, no ideas of the effects and no sense of the relationship between the architecture and the requirements.</p>
<p>If we're going to take architecture seriously, then we need to start treating it as engineering. We must understand the relationship between our architecture and the requirements of the system. We must demonstrate that our architecture works.</p>
<p>And then the wheels came off.</p>
<p>I don't work with huge systems, but I can clearly see that understanding the relationship between an architecture and the requirements is a good thing. Unfortunately, Tom presented examples from a domain that was unfamiliar to me (300 million dollar projects). In the examples, incredibly accurate percentages were shown (302%). At that point, I lost the thread. Estimates are just that, and if experience has taught me anything, it's that estimates have HUGE error bars. I didn't really see how all that planning up front led to a more measurable design. I've got a copy of Tom's book, <a href="http://gilb.com/Books">Competitive Engineering</a> so hopefully I can fill in the blanks.</p>
<h4>Building on SOLID foundations</h4>
<p><a href="https://twitter.com/natprcye">Nat Pryce</a> and <a href="https://twitter.com/sf105/">Steve Freeman</a> gave a thought-provoking presentation entitled "Building on SOLID foundations" which explored the gap between low-level detail and high-level abstractions.</p>
<p>At the lowest level we have guidelines for clean code, such as the <a href="http://en.wikipedia.org/wiki/SOLID_(object-oriented_design)">SOLID principles</a>. At this level, it's all about the objects, but not about how they collaborate and become assembled into a functioning system. Even with SOLID principles applied, macro level problems occur (somehow all related to food metaphors), colourfully referred to as "raviolli code". Individual blocks are well organized, but as a whole it still looks like a mess. "Filo code" is code that's got so many layers you can't tell what's going on. "Spaghetti and Meatballs" code is an application with a good core, but the communication glue surrounding it is a huge mess.</p>
<a href="http://upload.wikimedia.org/wikipedia/commons/thumb/9/98/Spaghetti_all'_arrabbiata.jpg/320px-Spaghetti_all'_arrabbiata.jpg" imageanchor="1" ><img border="0" src="http://upload.wikimedia.org/wikipedia/commons/thumb/9/98/Spaghetti_all'_arrabbiata.jpg/320px-Spaghetti_all'_arrabbiata.jpg" /></a>
<p>At the highest level we have principles such as <a href="http://www.melconway.com/Home/Conways_Law.html">Conway's Law</a>, <a href="http://en.wikipedia.org/wiki/Robustness_principle">Postel's Robustness Principle</a>, <a href="http://en.wikipedia.org/wiki/CAP_theorem">CAP</a>, <a href="http://en.wikipedia.org/wiki/End-to-end_principle">end-to-end principle</a> and <a href="http://en.wikipedia.org/wiki/Representational_state_transfer">REST</a>.</p>
<p>But what's in the middle?</p>
<p>In the middle there are some patterns, such as Cockburn's, <a href="http://alistair.cockburn.us/Hexagonal+architecture">Hexagonal Architecture</a> that help us structure systems as an inner domain language surrounded by specific adapters converting that data to the needs of the client. The question remains though; what are the principles between low and high level design?</p>
<p>Nat and Steve assert that compositionally is the principle for the middle. We should adopt a functional type approach and build a series of functions operating on immutable data in a stateful context. That sounds complicated, so what does code written in this style look like? <a href="https://code.google.com/p/hamcrest/">Hamcrest</a> gives us some examples, where by using simple combinators (functions that combine data) you can build up complicated expressions from simple operations (see the <a href="https://code.google.com/p/hamcrest/wiki/Tutorial">examples</a>).</p>
<p>Having done a fair bit of <a href="http://www.haskell.org/">Haskell</a> I found it really easy to agree with this point of view. When there's no mutable state you can reason about code locally (and not checking for mutation). Local reasoning means that I can understand the code without jumping around. This is a hugely important part of a well-designed system.</p>
<p>I was slightly concerned to hear this style of programming as <i>Modern Java</i>. I hope it's not, because using Java like this feels like putting lipstick on a pig. One of the things I value in Haskell is that composition is a first class citizen. Partial application, function composition and first class functions mean that gluing simple code together to make something powerful is incredibly easy. I hope we're at that awkward point in language evolution where we're stretching our current languages to do things they don't want to do. Maybe this is finally the time when a functional language hits mainstream? (maybe it's <a href="http://www.clojure.org/">Clojure</a> or <a href="http://scala-lang.org/">Scala</a>.</p>
<p>We tried adopting this style of programming at Dynamic Aspects when building <a href="http://dynamicaspects.org/papers/op07-perera.pdf">domain/j</a> [PDF]. It was fantastic fun, and I really love Java's static imports for making the code lovely and terse (finding out $ is an operator also helps). Something about it feels dirty though. Haven't quite put my finger on what that was then, and hopefully with lambdas in Java 8 it's more natural.</p>
<p>So what is the bit in the middle? The bit in the middle is the language that describes your domain. Naming is everything and you should do whatever you can to make it easiest to understand. Eschewing mutable state and using functional programming to compose multiple simple operators seems to work!</p>
<h4>Agile Architecture - Part 2</h4>
<p><a href="https://twitter.com/allenholub">Allen Holub</a> gave a presentation on agile techniques for design. Allen examined the <a href="http://en.wikipedia.org/wiki/Fragile_base_class">fragile base class</a> in some depth, before recapping CRC cards (not used enough!). Allen is a good presenter, so it was great to have a recap and have a few more examples to stick in my brain!</p>.
<h4>Leading Technical change</h4>
<p><a href="https://twitter.com/ntschutta">Nate</a> closed out the day by giving a presentation on Leading Technical Change. It was well presented and focused on two things. How do you keep up with technology and how do you engage your organization to move to different technologies?</p>
<p>Nate presented some really disturbing statics about how much time Americans (and presumably other countries) waste on TV. Apparently the average American watches 151 hours of TV a month! Wow.</p>
<p>Nate introduced the audience to the idea of the <a href="http://www.thoughtworks.com/radar">technology radar</a> which allows you to keep track of technology that is hot for yourself or your organization. We're trying to build one at <a href="http://www.red-gate.com">Red Gate</a>. We've also experimented with skills maps too, and you can see an example from a software engineering point of view <a href="http://dev.red-gate.com/skills/">here</a> (love to know what you think?).</p>
<p>Introducing change is hard, and Nate presented the same sort of ideas that <a href="https://twitter.com/royosherove/">Roy</a> presented the previous day. Change makes things worse in the beginning, but better in the end. Having the courage to stick out the dip is a hard thing! (image from <a href="http://sethgodin.typepad.com/the_dip/2007/05/images_from_the.html">here</a>)</p>
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj0t39F-qYd2bw-cUNOMu6QGWNvfL00CEr23KDMvpRfrurzTQemBHRRkEsddf6xo5aQsmwE3WL5ySdTkEESOxJQ8fwsZrcwgsTsh32-1uhE7dNitqTz089lEFbP4IA1w5BuVsGcYLydGfw/s1600/the_dip.jpg" imageanchor="1" ><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj0t39F-qYd2bw-cUNOMu6QGWNvfL00CEr23KDMvpRfrurzTQemBHRRkEsddf6xo5aQsmwE3WL5ySdTkEESOxJQ8fwsZrcwgsTsh32-1uhE7dNitqTz089lEFbP4IA1w5BuVsGcYLydGfw/s320/the_dip.jpg" /></a>
<p>I have to admit, I didn't take many notes from this talk because I was enjoying it instead :) It was well presented and engaging with the audience. In summary, change is hard and it's all about the people. I think deep down I always knew that (people are way more complicated than code) but it was great to hear it presented so well!</p>
Jeffhttp://www.blogger.com/profile/08195722595923882332noreply@blogger.com0