To launch an external program we’ll use a ProcBuilder. The run method
builds and spawns the actual process and blocks until the process exits.
The process takes care of writing the output to a stream (as opposed to the standard
facilities in the JDK that expect the client to actively consume the
output from an input stream:

ByteArrayOutputStream output = new ByteArrayOutputStream();
 
new ProcBuilder("echo")
        .withArg("Hello World!")
        .withOutputStream(output)
        .run();
 
assertEquals("Hello World!\n", output.toString());

The input can be read from an arbitrary input stream, like this:

ByteArrayInputStream input = new ByteArrayInputStream("Hello cruel World".getBytes());
 
ProcResult result = new ProcBuilder("wc")
        .withArgs("-w")
        .withInputStream(input).run();
 
assertEquals("3", result.getOutputString().trim());

If all you want to get is the string that gets returned and if there
is not a lot of data, using a streams is quite cumbersome. So for convenience
if no stream is provdied the output is captured by default and can be
obtained from the result.

ProcResult result = new ProcBuilder("echo")
                            .withArg("Hello World!")
                            .run();
 
assertEquals("Hello World!\n", result.getOutputString());
assertEquals(0, result.getExitValue());
assertEquals("echo \"Hello World!\"", result.getProcString());

For providing input there is a convenience method too:

ProcResult result = new ProcBuilder("cat")
   .withInput("This is a string").run();
 
assertEquals("This is a string", result.getOutputString());

Some external programs are using environment variables. These can also
be set using the withVar method

ProcResult result = new ProcBuilder("bash")
                            .withArgs("-c", "echo $MYVAR")
                            .withVar("MYVAR","my value").run();
 
assertEquals("my value\n", result.getOutputString());
assertEquals("bash -c \"echo $MYVAR\"", result.getProcString());

A common usecase for external programs is batch processing of data.
These programs might always run into difficulties. Therefore a timeout can be
specified. There is a default timeout of 5000ms. If the program does not terminate within the timeout
interval it will be terminated and the failure is indicated through
an exception:

ProcBuilder builder = new ProcBuilder("sleep")
        .withArg("2")
        .withTimeoutMillis(1000);
try {
    builder.run();
    fail("Should time out");
}
catch (TimeoutException ex){
    assertEquals("Process 'sleep 2' timed out after 1000ms.", ex.getMessage());
}

Even if the process does not timeout, we might be interested in the
execution time. It is also available through the result:

ProcResult result = new ProcBuilder("sleep")
        .withArg("0.5")
        .withTimeoutMillis(1000)
        .run();
 
assertTrue(result.getExecutionTime() > 500 && result.getExecutionTime() < 1000);

By default the new program is spawned in the working directory of
the parent process. This can be overidden:

ProcResult result = new ProcBuilder("pwd")
        .withWorkingDirectory(new File("/"))
        .run();
 
assertEquals("/\n", result.getOutputString());

It is a time honoured tradition that programs signal a failure
by returning a non-zero exit value. However in java failure is
signalled through exceptions. Non-Zero exit values therefore
get translated into an exception, that also grants access to
the output on standard error.

ProcBuilder builder = new ProcBuilder("ls")
                            .withArg("xyz");
try {
    builder.run();
    fail("Should throw exception");
} catch (ExternalProcessFailureException ex){
    assertEquals("ls: xyz: No such file or directory\n", ex.getStderr());
    assertEquals(1, ex.getExitValue());
    assertEquals("ls xyz", ex.getCommand());
    assertEquals("ls: xyz: No such file or directory\n", ex.getStderr());
    assertTrue(ex.getTime() > 0);
 
}

Input and output can also be provided as byte[].
ProcBuilder also copes with large amounts of
data.

int MEGA = 1024 * 1024;
byte[] data = new byte[4 * MEGA];
for (int i = 0; i < data.length; i++) {
    data[i] = (byte) Math.round(Math.random() * 255 - 128);
}
 
ProcResult result = new ProcBuilder("gzip")
   .withInput(data)
   .run();
 
assertTrue(result.getOutputBytes().length > 2 * MEGA);

The builder allows to build and spawn several processes from
the same builder instance:

ProcBuilder builder = new ProcBuilder("uuidgen");
String uuid1 = builder.run().getOutputString();
String uuid2 = builder.run().getOutputString();
 
assertNotNull(uuid1);
assertNotNull(uuid2);
assertTrue(!uuid1.equals(uuid2));

For convenience there is also a static method that just runs a
program and captures the ouput:

String output = ProcBuilder.run("echo", "Hello World!");
 
assertEquals("Hello World!\n", output);

Also there is a static method that filters a given string through
a program:

String output = ProcBuilder.filter("x y z","sed" ,"s/y/a/");
 
assertEquals("x a z\n", output);

After they saw a bit of it, they asked me how they could separate out the testdata. I was a bit stunned, because, well it was all there – at least to me as a java developer. I had separated out everything into a method somewhat like this:

 createNewContact("Felix", "Leipold");

So I could test for other inputs as well:

 createNewContact("", "Invalid");
 createNewContact("Invalid", "");
 createNewContact("Holden", "Caulfield");

Obviously the expectation of QAs was to have an external file, e.g. a spreadsheet that feeds the data into the test code. I am very critical of this approach, but I felt like playing with junit 4 a bit. Junit 4 allows you to specify a runner four your test and indeed they have got one prepackaged that does parameterized tests. So I thought how about having a runner reads a spreadsheet and feeds the data into the test. As an example I use a very simplistic function:

    double add(double a, double b) {
        return a + b;
    }

So to test this I created this table and saved it as mathTest.xls:

Then I went on to create a test case, like this:

@RunWith(SpreadsheetDriven.class)
@Spreadsheet(name = "mathTest.xls")
public class MathTest {
    double a;
    double b;
    double expectedSum;
 
    public MathTest(Double a, Double b, Double expectedSum) {
        this.a = a;
        this.b = b;
        this.expectedSum = expectedSum;
    }
 
    @Test
    public void testAdd(){
        assertEquals(expectedSum, add(a,b));
 
    }
 
    double add(double a, double b) {
        return a + b;
 
    }
}

Essentially this thing parses the file specified in the annotation and passes it into the constructor. It’s not doing any fancy type conversions. All numerics are passed in as double, booleans as booleans, and everything else should be a string. Implementation and tests are to be found here and depend on apache poi being on the classpath for reading the excel file.

I am also quite proud of the reporting I get:

I am wondering, if all this is a good idea. But if it helps keeping people from using fit, it’s a start. What I realised by now is that actually table or data driven testing is much more appropriate for unit tests. Funnily enough that people actually do it the other way round. For unit tests I would actually recommend not to use a spreadsheet, but the original JUnit Parameterized runner, which lets you define your data like this:

    @Parameterized.Parameters
      public static List getParameters(){
        return Arrays.asList(new Object[][]
                {
                        {2,2,4},
                        {3,4,7},
                        {4,2,6},
                        {2,4,6}}
        );
    }