It's inevitable. As more and more features are added to an application, the code base grows. The build has more to do: more compiling, more tests and metrics to run, more reports to generate. We've done numerous things to keep the build time down. In this entry, I'd like to describe one of them: running standard JUnit tests concurrently.

A colleague of mine came up with this idea. He first did a preliminary search to see if anything already existed.  Pretty much everything he found was intrusive, e.g., requiring extending a specialized TestCase.  He then attempted to combine Ant's Parallel and JUnit tasks. That is, he could simply nest N number of JUnit tasks inside a Parallel task. The problem here was coming up with a good way to divide the tests up into equal parts beforehand. My colleague quickly scrapped this idea in favor of a custom Ant task that wrapped the JUnit task.

The thought was to leverage the JUnit task as much as possible, but fronting it with the ability to run tests in parallel. The conceptual design looks like this:

There are N number of TestExecutors. A TestExecutor runs in a single JVM. Thus, there are N JVMs. Say we have a dedicated, 4 processor integration machine running builds. We may choose to configure our custom task with 5 TestExecutors (we'll add one assuming we're not 100% compute bound). Note: if you're familiar with the JUnit task, you may be wondering how its fork/forkmode attributes fit in. The answer is that they are eliminated in favor of this jvm-count attribute.

There's a single TestProvider. His job is to gather all the tests and hand them out one at a time to whichever TestExecutor is ready. A simple protocol exists to let a TestExecutor tell the TestProvider that he's done and ready for another test.  This should maximize parallelism.

The biggest negative with this approach is that since all tests run in a fixed number of JVMs, there's some loss of isolation here. Class level/global state set from a previous test could affect later tests. We're willing to trade this for increased performance.

The other possible downside is the use of a non-standard Ant task.

Using this approach we've successfully reduced our build time to usable levels.

1. AtLeastOneConstructor: Why? Code is smaller without an empty, no-args contructor.
2. AvoidInstantiatingObjectsInLoops: This one just seems to generate too many false positives. In addition, for short-lived objects, garbage collection is essentially free.
3. CallSuperInConstructor: super() is called implicitly and requiring it just adds more lines of code.
4. JUnitAssertionsShouldIncludeMessage: Most of the methods in Assert already generate good messages. I do include a message for those that don't (assertTrue(), assertFalse(), fail()).
5. LocalVariableCouldBeFinal: final for variables is usually overkill and it adds line length.
6. LongVariable: Clarity rules!
7. MethodArgumentCouldBeFinal: Same as LocalVariableCouldBeFinal.
8. OnlyOneReturn: I think it's clearer to exit early. It reduces the amount of things to think about below the exit.
9. PositionLiteralsFirstInComparisons: This really isn't that bad, but myString.equals("x") just reads better than "x".equals(myString).
10. ShortVariable: There are just too many cases where it's acceptable to have short variables in small methods .

 These two just missed the cut:

• ShortMethodName: I try to make my names expressive. I've never seen this rule fire.
• SignatureDeclareThrowsException: This is an okay rule, but for JUnit tests, who cares?

Let's quickly look at an example, the idea stolen from one of David Chelimsky's blog entries about specifying the behavior of a stack. We're focused here on the specification of an empty stack:

// notice the class name specifies the context
public class EmptyStack {
    private Stack stack = null;
    
    @Before
    public void setUp() {
        // set up the context
        stack = new Stack();
    }
    
    // notice the name focuses on the context
    @Test
    public void shouldBeEmpty() {
        assertTrue("not empty", stack.isEmpty());
    }
    
    @Test(expected=EmptyStackException.class)
    public void shouldComplainOnPeek() {
        stack.peek();
    }
    
    
    // more specification focused methods
}

The point here is that you can get many of the benefits of BDD (a focus on specification rather than testing) using the familiar JUnit framework.

Now, if you're a hardcore BDD'er, then you might complain that you still have to use a test-centric vocabulary. You still need those Test annotations and method calls like assertEquals rather than Dave Astels' preferred shouldEquals calls.

Also, from the legacy side of the fence, you lose the convention of method names starting with testSomething and class names ending with Test. It's sometimes hard to let go of that if you've been writing tests for a long time and it's super clear to spot those test methods and test classes if you're following a naming convention. Furthermore, Ruby on Rails has taught us that convention is a good thing. The test method naming convention doesn't really bother me so much. The @Test annotation makes things clear enough and shouldSomething really gets us focused on specification. However, I haven't let go of ending the class name with Test. Maybe it's more because Ant can pick those tests up easier if there's a standard naming convention.

The last negative I can think of is that of consistency. Having a mix of old style test centric tests and BDD style tests could bother some.

So, let's be honest. What am I really doing? Well, I am incorporating more and more BDD into my work. However, I still shy away from creating new classes. The BDD style really lends itself to many test classes (contexts) per class under test. In the case of Stack, as David Chelimsky points out, you'd also need AlmostEmptyStack, AlmostFullStack, and FullStack classes to fully specify the behavior. I just can't commit myself to writing all those. But I am focusing more on the set up of a context and the specification methods of that context. I just may cheat a little and combine contexts into a single class. So perhaps in the Stack example, I'd combine the empty and almost empty stack contexts under one test class. You know, I'd set up an empty Stack in setUp and for the almost empty context, I'd do a little more set up (push something) in the appropriate shouldSomething methods to make it almost empty.

I also revert back to legacy style testing sometimes. This is usually when I'm modifying an existing, old style test. It's just simpler and faster, and typically the context isn't set up too well for the BDD style.

So my advice is to use the right tool for the job. Use BDD style when that makes things clearer and you want to focus on specification. Use test centric style when that's easier. Try to do a better job of focusing more on specification and less on verification, especially when you're adding new behavior.

1. Extract Local Variable (possibly multiple times).
2. Extract Method
3. Inline the local variables created in step 1.

Let’s take the Range.equals() example from the previous blog. My brain dump would have resulted in a table like this:

Min	Max	Equals?
same	same	true
different	same	false
same	different	false

Yes, I would have jotted down other, exceptional cases for things like null and a non-Range object, but that’s out of scope here. For this blog, I’m just concentrated on comparing Range objects. I start with the positive case and come up with something like:

   private static final int MIN = 1;
   private static final int MAX = 7;

   private Range range = new Range(MIN, MAX);

To be honest, I probably would have had the MIN and MAX constants hardcoded in there, but at some point, I would have performed Extract Constant to come up with the above.

After I get that test to pass (by returning true), I’m ready to write the next test. I want to pick something simple again (of course in this example they’re all simple). But before I write the next test, I do something that is probably considered cheating by the hardcore test-driven development crowd. I’m not too concerned because I know from experience that I’m going to end up with tabular tests. In fact, the table is specified above!  So here goes the set of refactorings:

I first Extract Local Variable on MIN and MAX:

   public void testEqualsAnotherRange() {
      int testMin = MIN;
      int testMax = MAX;
      Range anotherRange = new Range(testMin, testMax);
      assertEquals(range, anotherRange);
   }

Now I know from my table that I need a boolean for my expected equals. There's no simple Eclipse refactoring so I manually make the change:

   public void testEqualsAnotherRange() {
      int testMin = MIN;
      int testMax = MAX;
      boolean expectedEquals = true;
      Range anotherRange = new Range(testMin, testMax);
      assertEquals(expectedEquals, range.equals(anotherRange));
   }

I highlight the last two lines and perform Extract Method:

   public void testEqualsAnotherRange() {
       int testMin = MIN;
       int testMax = MAX;
       boolean expectedEquals = true;
       testEqualsAnotherRange(testMin, testMax, expectedEquals);
   }

   private void testEqualsAnotherRange(int testMin, int testMax,
           boolean expectedEquals) {
       Range anotherRange = new Range(testMin, testMax);
       assertEquals(expectedEquals, range.equals(anotherRange));
   }

I want to get back to what it looks like in the table, so I Inline each local variable:

   public void testEqualsAnotherRange() {
      testEqualsAnotherRange(MIN, MAX, true);
   }

   private void testEqualsAnotherRange(int testMin, int testMax,
         boolean expectedEquals) {
      Range anotherRange = new Range(testMin, testMax);
      assertEquals(expectedEquals, range.equals(anotherRange));
   }

I just created the first row of the table by applying the set of refactorings that is the topic of this blog. At this point, I usually look at the parameterized method and make sure I’m happy with the signature. I’ve already done most of this when I extracted the method, but Eclipse doesn’t let me make the method static, if that’s appropriate (in this case it’s not). Eclipse also lists all the checked exceptions, which for tests, I’d rather generalize to Exception.  This doesn't apply in this case either.

During this set of refactorings, I would have ran the tests after a change or two just to make sure I didn’t break anything. I also do that because it feels good to get that green bar.

This is a pretty trivial example. Most of the time much more is going on in the parameterized method, but this still shows the removal of duplication.

I could have made these changes manually, but like I mentioned above, I like the automation; I’m less likely to break things.

   public void testEqualsAnotherRange() {
      testEqualsAnotherRange(MIN,     MAX, true);
      testEqualsAnotherRange(MIN - 1, MAX, false);
   }

I’ve got the red bar. Time to make it green.