{"id":217,"date":"2010-01-21T09:54:09","date_gmt":"2010-01-20T22:54:09","guid":{"rendered":"http:\/\/blog.mozilla.org\/nnethercote\/?p=217"},"modified":"2010-01-21T14:27:12","modified_gmt":"2010-01-21T03:27:12","slug":"safer-refactoring-in-nanojit","status":"publish","type":"post","link":"https:\/\/blog.mozilla.org\/nnethercote\/2010\/01\/21\/safer-refactoring-in-nanojit\/","title":{"rendered":"Safer refactoring in Nanojit"},"content":{"rendered":"

Recently I’ve been making a lot of refactoring changes to Nanojit that I think of as “code hygiene” — changes that clean up the code, but don’t change its behaviour at all.\u00a0 (Example 1<\/a>, example 2<\/a>.)\u00a0 When making changes like these I like to do the changes carefully and gradually to make sure that I don’t affect behaviour unintentionally.\u00a0 In Nanojit’s case, this means that the generated code shouldn’t change.\u00a0 But it’s always possible to make mistakes, and sometimes a particular change is disruptive enough that you can’t evolve the code from one state to another without breaking it temporarily.<\/p>\n

In such cases, if the regression tests pass it tells me that the new code works, but it doesn’t tell me if I’ve managed to avoid changing the behaviour as I intended.\u00a0 So I’ve started doing comparisons of the native code generated by Nanojit for parts of SunSpider using TraceMonkey’s verbose output, which is controlled with the TMFLAGS environment variable.\u00a0 For such changes, the generated code should be identical to an unchanged version.\u00a0 The way I organise my workspaces<\/a> is such that I always have an unchanged repository available, which makes such comparisons easy.<\/p>\n

Except, the generated code is never quite identical because it contains addresses and the slightest change in the executable can affect these.\u00a0 So I run the verbose output through this script:<\/p>\n

perl -p -e 's\/[0-9a-fA-F]{4,}\/......\/g'<\/pre>\n
It just replaces numbers with four or more digits with “……”, which is enough to filter out these address differences.<\/p>\n
I tried this technique on all of SunSpider, but it turns out it doesn’t work reliably, because crypto-aes.js is non-deterministic<\/a> — it contains a call to Date.getTime() which introduces enough non-determinism that TraceMonkey sometimes traces different code.\u00a0 A couple of the V8 benchmarks have similar non-deterministic behaviours.\u00a0 Non-determinism is a rather undesirable feature of a benchmark suite so I filed a SunSpider bug<\/a> which is yet to be acted on.<\/p>\n
Fortunately, experience has shown me that I don’t need to do code diffs on all of SunSpider to be confident that I haven’t changed Nanojit’s behaviour — doing code diffs on two or three of the bigger benchmarks suffices, as enough code is generated that even small behavioural differences are quickly found.<\/p>\n
In fact, this code diff technique is useful enough that I’ve started using it sometimes even when I do want to change behaviour — I split such changes into two parts, one which doesn’t change the behaviour and one which does.\u00a0 For example, when I added an opcode to distinguish between 64-bit integer loads and 64-bit integer floats I landed a non-behaviour-changing patch<\/a> first, then did a follow-up change that improved x86-64 code generation<\/a>.\u00a0 This turned out to be a good idea because I introduced a regression<\/a> with the follow-up change, and the fact that it was separated from the non-behavioural changes made it easy to determine what had gone wrong, because the regressing patch was much smaller than a combined patch would have been.<\/p>\n
I also recently wrote a script to automate the diff process, which makes it that much easier to perform, which in turn makes it that much more likely that I’ll actually do it.<\/p>\n
In summary, it’s always worth thinking about new ways to test your code, and when you find a good one, it’s worth making it as easy as possible to run those tests.<\/p>\n","protected":false},"excerpt":{"rendered":"
Recently I’ve been making a lot of refactoring changes to Nanojit that I think of as “code hygiene” — changes that clean up the code, but don’t change its behaviour at all.\u00a0 (Example 1, example 2.)\u00a0 When making changes like these I like to do the changes carefully and gradually to make sure that I […]<\/p>\n","protected":false},"author":139,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[528,617,467,649],"tags":[],"_links":{"self":[{"href":"https:\/\/blog.mozilla.org\/nnethercote\/wp-json\/wp\/v2\/posts\/217"}],"collection":[{"href":"https:\/\/blog.mozilla.org\/nnethercote\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blog.mozilla.org\/nnethercote\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blog.mozilla.org\/nnethercote\/wp-json\/wp\/v2\/users\/139"}],"replies":[{"embeddable":true,"href":"https:\/\/blog.mozilla.org\/nnethercote\/wp-json\/wp\/v2\/comments?post=217"}],"version-history":[{"count":0,"href":"https:\/\/blog.mozilla.org\/nnethercote\/wp-json\/wp\/v2\/posts\/217\/revisions"}],"wp:attachment":[{"href":"https:\/\/blog.mozilla.org\/nnethercote\/wp-json\/wp\/v2\/media?parent=217"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blog.mozilla.org\/nnethercote\/wp-json\/wp\/v2\/categories?post=217"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blog.mozilla.org\/nnethercote\/wp-json\/wp\/v2\/tags?post=217"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}