Skipfish is active scanner so it first scan application, preparing the map of web site, than recursively ran different test, the last thing is report generation. Documentation is simple and has a lot of example we can start on. So let’s see that in action.

One of such command is:

$ skipfish -m 5 -LVJ -W /dev/null -o output_dir -b ie http://www.example.com/

During the scan, Skipfish is displaying statistics:

Scan statistics
---------------

       Scan time : 0:11:07.0068
   HTTP requests : 2446 sent (3.71/s), 16228.73 kB in, 659.18 kB out (25.32 kB/s)
     Compression : 0.00 kB in, 0.00 kB out (0.00% gain)
 HTTP exceptions : 34 net errors, 0 proto errors, 0 retried, 0 drops
 TCP connections : 2451 total (1.09 req/conn)
  TCP exceptions : 0 failures, 1 timeouts, 0 purged
  External links : 745 skipped
    Reqs pending : 219        

Database statistics
-------------------

          Pivots : 471 total, 94 done (19.96%)
     In progress : 323 pending, 38 init, 12 attacks, 4 dict
   Missing nodes : 54 spotted
      Node types : 1 serv, 269 dir, 46 file, 1 pinfo, 91 unkn, 63 par, 0 val
    Issues found : 70 info, 111 warn, 49 low, 1 medium, 13 high impact
       Dict size : 0 words (0 new), 0 extensions, 0 candidates

After few hours/minutes, it depends on the site we are scanning, we will got

[+] Copying static resources...
[+] Sorting and annotating crawl nodes: 1666
[+] Looking for duplicate entries: 1666
[+] Counting unique issues: 1158
[+] Writing scan description...
[+] Counting unique issues: 1666
[+] Generating summary views...
[+] Report saved to outputDir/index.html
[+] This was a great day for science!

The report consist of “crawl results”, “document type overview” and “issue type overview”. My last scan result has some finding, but also has a lot of false positives, it seams that a lot of work still waiting for a Skipfish team, but it looks promising.

We create simple project.

$ mvn archetype:generate

We move to the project and we run test – everything is ok.

$ mvn test
...
[INFO] BUILD SUCCESSFUL

Now we change JUnit version in pom.xml from 3.8.1 to 4.4. We run test again and still everything is ok.
We continue by adding java 5 target compilation.

<build>
	<plugins>
	 <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-compiler-plugin</artifactId>
            <configuration>
                <source>1.5</source>
                <target>1.5</target>
                <encoding>UTF-8</encoding>
            </configuration>
        </plugin>
	</plugins>
</build>

Next we change testApp() method to something like this (JUnit4)

    /**
     * Rigorous Test 
     */
    @Test
    public void tapp()   {
        assertTrue( true );
    }

Test fails !!!. The stacktrace is :

-------------------------------------------------------------------------------
Test set: info.pietrowski.AppTest
-------------------------------------------------------------------------------
Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 0.036 sec <<< FAILURE!
warning(junit.framework.TestSuite$1)  Time elapsed: 0.008 sec  <<< FAILURE!
junit.framework.AssertionFailedError: No tests found in info.pietrowski.AppTest
	at junit.framework.Assert.fail(Assert.java:47)
	at junit.framework.TestSuite$1.runTest(TestSuite.java:97)
	at junit.framework.TestCase.runBare(TestCase.java:134)
	at junit.framework.TestResult$1.protect(TestResult.java:110)
	at junit.framework.TestResult.runProtected(TestResult.java:128)
	at junit.framework.TestResult.run(TestResult.java:113)
	at junit.framework.TestCase.run(TestCase.java:124)
	at junit.framework.TestSuite.runTest(TestSuite.java:232)
	at junit.framework.TestSuite.run(TestSuite.java:227)
	at org.junit.internal.runners.JUnit38ClassRunner.run(JUnit38ClassRunner.java:81)
	at org.apache.maven.surefire.junit4.JUnit4TestSet.execute(JUnit4TestSet.java:62)
	at org.apache.maven.surefire.suite.AbstractDirectoryTestSuite.executeTestSet(AbstractDirectoryTestSuite.java:140)
	at org.apache.maven.surefire.suite.AbstractDirectoryTestSuite.execute(AbstractDirectoryTestSuite.java:127)
	at org.apache.maven.surefire.Surefire.run(Surefire.java:177)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
	at java.lang.reflect.Method.invoke(Method.java:585)
	at org.apache.maven.surefire.booter.SurefireBooter.runSuitesInProcess(SurefireBooter.java:338)
	at org.apache.maven.surefire.booter.SurefireBooter.main(SurefireBooter.java:997)

No tests found !!!. What’s going on. The core thread-dump lines are:

 at org.junit.internal.runners.JUnit38ClassRunner.run(JUnit38ClassRunner.java:81)
 at org.apache.maven.surefire.junit4.JUnit4TestSet.execute(JUnit4TestSet.java:62)

This means that maven-surefire-plugin run Junit4TestSet as it should do, but unfortunately JUnit runs Junit38ClassRunner !!!. This strange behavior is due we still extends TestCase class. So the quickies solution is to drop this extend. We can not use now assert* methods, the solution is to static import org.junit.Assert.*. Everything backs to normal.

But what if we want to extend TestCase (DBUnit, Spring*TestCase). In case of Spring test classes there is special tree for JUnit4 and TestNG so every TestCase class has equivalent in JUnit4 and TestNG. So for example AbstractTransactionalJUnit38SpringContextTests, has equivalents AbstractTransactionalJUnit4SpringContextTests, and AbstractTransactionalTestNGSpringContextTests.

The problem still exists when library does not provide such equivalents. Let’s look to the code, the problematic function is

The procedure is simple, if class has @Ignore annotation than we skip this class, next we look if class has @RunWith annotation if so we get value of this annotation (we talk about it later). Next three is simple if class has suite method we run Suite runner called AllTest, if class is subclass of TestCase then we use JUnit38ClassRunner if everything else we run with JUnit4ClassRunner. We solved the mystery

Our only hope is @RunWith annotation, we can use it with JUnit4ClassRunner and than everything is alright.

Of course we can use this annotation with any class that extends Runner. I hope this solves your problems. See you.

Pedro