Eclipse Refactoring for legacy code

It has been quite sometime since I wrote anything on this blog, twitter probably spoiled me. If you are following me on twitter, you probably noticed announcement of  a small eclipse plugin for automated refactoring for Legacy Code. 


Before few months, I wrote an LTK refactoring mainly in Scala (yes, eclipse plugin in Scala language) to forward static method calls in a Java method to an instance method in same class. I am not really an expert at Scala (or functional programming for that matter) so the code is more of javaish Scala but I am improving and thats the best thing about Scala. This is also an attempt to prove how easy it is to write eclipse plugins in Scala and how seamlessly it integrates with Java source; Scala is not only a better language, it is much more suitable to deal with Eclipse APIs (you can define views etc. on extremely verbose interfaces like JDT AST).


Enough about Scala; the real purpose of the refactorings provided in this plugin would be to ease development with Legacy Java code, most of the code generators (think JavaCC) generate ugly Java code which is not only non unit testable but pain to comprehend and is often not usable with concurrent routines. Using this automated refactoring should make such code better. You can read more about the motivation behind the plugin in wiki.


For the performance heads who are worried about method chains introduced by this refactoring are encouraged to run their own "sane" micro-benchmarks to be sure JVM is really in-lining the method calls created by this refactoring.


The plugin is currently in beta and I am open for any new refactoring proposals. If you have any feedback, you are welcome to comment on this blog post or create a bug. The update site is here.

Real Refactoring

A lot of really good material has been written about refactoring. However, just like a lot of other things in software development, refactoring is also victimized by becoming the buzzword implying 'rewrites' and every other thing that is not refactoring. A lot of people really don't understand what they mean when they refer to 'refactoring', this post aims to provide a systematic explanation of refactoring for such audience.

Refactoring is merely a process of restructuring existing code so that its end result and meaning remains intact. However, this restructuring has a great potential to become far better design than the original one. (It is implied that the refactored design is easier to refactor and hence better) In rest of this post I will try to summarize how to approach and refactor a code in real life application development.

1. Get familiar with code

I am assuming here that we are almost always refactoring code written fully or partially by others, refactoring your own code should be much more simple exercise. Primarily you would be dealing with Legacy code or testable code. If you are lucky enough to get the code which is almost covered with unit tests, your job becomes substantially easy and you can start refactoring very fast without a lot of trouble. However if you are working with existing code which just exists and works somehow, you will have to start slowly. The first step is to understand the incomprehensible. There are several ways you can get familiar with code:

• Read the code

Reading code is one of the fundamental activity we developers do most of the time, if any employer is in the illusion that she is paying developers just to write code then they are paying 1/3 of the salary. Reading code is an art which develops over the time, reading good code can make you better developer (with a precondition that one possesses the ability to judge good and bad code for the language). The more you read bad code the more you familiarize yourself with the repeating mistakes in the code (duplication, absolute hostility towards testability, extremely fearful defensive checks and other hilarities). The goal is to grasp the mindset of the original developer(s) who wrote it. You will know it immediately how deeply thought out the code is in a few passes. Use any modern IDEs to navigate through files, ignore the comments which don't make sense.

• Write test

Once you have some understanding of inputs/outputs and/or interactions in code's life cycle you can write some tests just so that you can identify if you made some mistake in next step (Step 2). These tests might not be unit-tests (unless the code itself is unit testable). Writing test is incredibly useful way to get familiar with the code base.

2. Isolate the refactoring hot-spot (or "The Inflection points" as Michael Feathers calls it)

There has to be a really good reason why you are refactoring and it certainly can't be a full 'rewrite'. Irony will kill itself when it finds out that the problems in a bad code manifests itself in popular hot-spots. By isolating these hot-spots you should be able to mock out the uninteresting and untestable (or very slow) dependencies. If you have multiple inflection points; deal with them one by one in isolation. Isolation may become tricky because you may have to introduce some inevitable changes before you actually have any unit tests. For example, you might want to eliminate inherently untestable 'statics', extract methods/interfaces which you can stub with NullObjects/mocks. The tests created in earlier step will be valuable to identify any problems you introduced while isolating the code. The goal here is to not concern yourself with trivialities other than the refactoring hot-spots.

3. Write unit tests

Now that you have code which can be isolated, start writing unit tests. If you are working on Java sources I highly recommend using Mockito to mock the dependencies. Don't waste time on accidental complexities of other mock libraries. A reasonable analogy would be: if Agile software development is a good thing than other mocking tools are worse than Waterfall - Mockito is Agile.

Your primary goal is to cover code under refactoring as much as possible, there are no hard numbers on code/branch coverages (but anything beyond 90% should be good enough :)), use common sense and quality metrics like CRAP. The more unit tests you will write, more you will learn about the code base. There may be totally unused code for anti-YAGNI stronghold, or there might be bugs, Your unit test will reveal these naturally. Depending on the code in question, A good test suit generally reaches the size of the code under test.

4. Refactor

Refactoring is particularly painful when it has to be done in an environment with highly volatile code base and for the parts of code which are critical to overall application functionality. Make small changes; make sure all tests pass.

If you are using Eclipse - turn off auto build and add a ANT or Maven builder so that it runs your tests after each compile - you will be running tests a lot.

Use a real source control system, it might be much harder for you to revert the changes otherwise. You might consider using something like git in between upstream and your local repository if your primary SCM is not good enough. In practice, this becomes a lot more important than what IDE you use, you will need the ability to identify and rollback a commit specifically in code base with many hands. Commit after each change, the smaller the commit the lesser chance of breaking a lot of things together by a long shot.

With each commit going in repository without external complains (like "You broke my build!"), you will gain confidence to make bigger changes. You should continue writing tests while refactoring because if at this point you are making changes without violating functional contracts you are evolving a new design and doing something good. If you have broken something and your justification involves the word 'refactoring', you are doing it wrong! you either don't have enough tests or you failed to understand the code entrails.

So this is refactoring in real life. For the primary audience, please consider following these simple steps before tossing the word 'refactoring' around. If you disagree, then invent the new word and let me know :).

Unusual tip to find refactoring hot-spot

Disclaimer: This post contains generalization based on my experience, it may or may not be applicable in every state of code-base resembling this situation.

Most of you have probably been through this list of smells to refactoring. In Recent refactoring sessions I discovered a new (at least unusually new) way to identify "hot-spot" needing refactoring.

This might be in turn related to some other smells (like most of them), but here it is:

- A (set of) source file with significantly more revisions compared to it's dependencies demands refactoring.

The obvious question here is why?
- Number of revisions for a source file indicate that it is being modified more frequently than it should. There can be many reasons why a source file is being modified among:

• Imperative requirements change or
  
• A source file that tries to do so much than it should

Now, requirements change is something very common in everyday life. But generally speaking ( opps! ), Not all requirements ask for a significant change in control flow but data. Depending on the domain, you can externalize the portions which keep changing to helper classes or configurations.

A source file trying to do more than it could is always a problem and is generally perceived as 'God module', it's time to slice it down.

So next time you feel like hunting for bad code, look at the revisions and you will have hint where to start. Good hunting.

PS: Note that I've deliberately generalized this to 'source files' and not the configurations.

The art of not repeating yourself

I discovered a great bug in the my code today, It was weird hibernate related bug. If you have been working with hibernate enough you know you've to be careful about the ordering of statements relating to your persistent object model, that may sound little odd but that's how it is with hibernate. If you hit the SQL constraint exception and you've multiple joined entities, be cautious, you might want to isolate call to some many-to-one references earlier than populating your domain object.

There are several ways you'll force flush to hibernate session and that flush (a flush can be a batch update) will cause DB validation and if that flush happens before statements that set the non-null fields, you'll be greeted with boring exceptions. They are not very interesting exceptions to deal with and it might take sometime before you understand what's going on. In general, debugging hibernate code is never interesting unless you've less experience with some outstanding ORM patterns.

The problem that I hit today was slightly different than what I described though (but I've experienced similar problems many times), My problem was that I was inadvertently modifying a read-only row which had 'long' primary key with '0L' as value, if you've worked enough with hibernate types you know 0 (zero) is the default value for long and integer types. It is just because I was modifying that row by mistake hibernate decided to save that row with different primary key (new sequence number), it assumed 0(zero) as default value (dirty value in Identity Map) for long primary key type, this is bad assumption but there are less ways any ORM framework can find dirty objects. So from next time onwards be aware about these little details If you hit problem of similar nature.

How I prevented myself from repeating was the way of writing a simple JUnit test case which isolated the problem in my code and pointed me to the place where I should be looking without long (very very long) J2EE debugging session. Let me tell you, that's great relief! With that test case I was able to refactor the code and resolve this bug without too much pain. Which makes me and my manager happy :)

If you've been experimenting something similar, I would be glad to know..

Refactoring and Upfront Designs

Low level software design is one of my regular work, most of the time it turns out to be really interesting; so much so that I fail to sense environmental changes around me :). Most of the decisions in such designs, confined to refactoring, are upfront.

There were times when these decisions used to drive me nuts (for example, I used to extend framework at larger granularity, failed to identify framework extensions at right places etc.), with time (and few refactoring book skims) I learned how to simplify it. Some of them are separating the concerns of mammoth classes into few "aware" type of interfaces (Spring style), distribute methods, extract stateless routines, derive common abstract base class etc. These decisions are quite easy to tackle with upfront designs.

Refactoring is wonderful, refactoring to patterns is even more interesting. Recently, Ketan identified design smells in his GUI testing framework for SWT. I did a test drive into the initial framework to understand and tried the same upfront approach to make it smell free, I failed; but I understood the potential pain points. Then, We brainstormed and discussed the design on white board, espied the smells; asking questions, suggesting and rejecting alternatives designs. That cleared few confusing points and we came with a good enough, smell free design (Ketan, all we did was shifting and distributing the bottleneck somewhere else :), as it can't be eliminated - "SWT"- I wonder why SWT is not very cleanly designed, performance?).

The lessons learned? You can't refactor well if you're in confusion (requirements or other), refactoring in this case drags you away, and Upfront designs takes extensive experience on variety of problem domains.