RunWith helps JUnit

When we want to test with Junit4 we can spot some problems. Firstly I thought that this is a maven-surefire-plugin problem, but as I dig into, the problem shows in JUnit runner. We track this problem on simple example.

We create simple project.

$ mvn archetype:generate

We move to the project and we run test – everything is ok.

$ mvn test
...
[INFO] BUILD SUCCESSFUL

Now we change JUnit version in pom.xml from 3.8.1 to 4.4. We run test again and still everything is ok.
We continue by adding java 5 target compilation.

<build>
	<plugins>
	 <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-compiler-plugin</artifactId>
            <configuration>
                <source>1.5</source>
                <target>1.5</target>
                <encoding>UTF-8</encoding>
            </configuration>
        </plugin>
	</plugins>
</build>

Next we change testApp() method to something like this (JUnit4)

    /**
     * Rigorous Test :-)
     */
    @Test
    public void tapp()   {
        assertTrue( true );
    }

Test fails !!!. The stacktrace is :

-------------------------------------------------------------------------------
Test set: info.pietrowski.AppTest
-------------------------------------------------------------------------------
Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 0.036 sec <<< FAILURE!
warning(junit.framework.TestSuite$1)  Time elapsed: 0.008 sec  <<< FAILURE!
junit.framework.AssertionFailedError: No tests found in info.pietrowski.AppTest
	at junit.framework.Assert.fail(Assert.java:47)
	at junit.framework.TestSuite$1.runTest(TestSuite.java:97)
	at junit.framework.TestCase.runBare(TestCase.java:134)
	at junit.framework.TestResult$1.protect(TestResult.java:110)
	at junit.framework.TestResult.runProtected(TestResult.java:128)
	at junit.framework.TestResult.run(TestResult.java:113)
	at junit.framework.TestCase.run(TestCase.java:124)
	at junit.framework.TestSuite.runTest(TestSuite.java:232)
	at junit.framework.TestSuite.run(TestSuite.java:227)
	at org.junit.internal.runners.JUnit38ClassRunner.run(JUnit38ClassRunner.java:81)
	at org.apache.maven.surefire.junit4.JUnit4TestSet.execute(JUnit4TestSet.java:62)
	at org.apache.maven.surefire.suite.AbstractDirectoryTestSuite.executeTestSet(AbstractDirectoryTestSuite.java:140)
	at org.apache.maven.surefire.suite.AbstractDirectoryTestSuite.execute(AbstractDirectoryTestSuite.java:127)
	at org.apache.maven.surefire.Surefire.run(Surefire.java:177)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
	at java.lang.reflect.Method.invoke(Method.java:585)
	at org.apache.maven.surefire.booter.SurefireBooter.runSuitesInProcess(SurefireBooter.java:338)
	at org.apache.maven.surefire.booter.SurefireBooter.main(SurefireBooter.java:997)

No tests found !!!. What’s going on. The core thread-dump lines are:

 at org.junit.internal.runners.JUnit38ClassRunner.run(JUnit38ClassRunner.java:81)
 at org.apache.maven.surefire.junit4.JUnit4TestSet.execute(JUnit4TestSet.java:62)

This means that maven-surefire-plugin run Junit4TestSet as it should do, but unfortunately JUnit runs Junit38ClassRunner !!!. This strange behavior is due we still extends TestCase class. So the quickies solution is to drop this extend. We can not use now assert* methods, the solution is to static import org.junit.Assert.*. Everything backs to normal.

But what if we want to extend TestCase (DBUnit, Spring*TestCase). In case of Spring test classes there is special tree for JUnit4 and TestNG so every TestCase class has equivalent in JUnit4 and TestNG. So for example AbstractTransactionalJUnit38SpringContextTests, has equivalents AbstractTransactionalJUnit4SpringContextTests, and AbstractTransactionalTestNGSpringContextTests.

The problem still exists when library does not provide such equivalents. Let’s look to the code, the problematic function is

The procedure is simple, if class has @Ignore annotation than we skip this class, next we look if class has @RunWith annotation if so we get value of this annotation (we talk about it later). Next three is simple if class has suite method we run Suite runner called AllTest, if class is subclass of TestCase then we use JUnit38ClassRunner if everything else we run with JUnit4ClassRunner. We solved the mystery 😉

Our only hope is @RunWith annotation, we can use it with JUnit4ClassRunner and than everything is alright.

Of course we can use this annotation with any class that extends Runner. I hope this solves your problems. See you.

Pedro