You are here: JWalk Home / User Guide / JWalkUtility /

The JWalkUtility User Guide

The original and classic tool in the JWalk 1.1 Tool Suite is the command-line driven unit testing tool JWalkUtility. This is provided for those who prefer command-line console interaction and plain text output. The style is similar to the old JWalk 0.8 Beta release, but uses the new JWalker core test engine.

Please note that you must first have a valid license to run this tool, which is obtainable free for most academic and industrial evaluation purposes. For this and the software bundle, please follow the clear instructions given at the JWalk Download Centre.

Launching JWalkUtility

The following assumes that you have chosen your JWalk working directory, and have installed the software bundle: JWalk11.jar and an accompanying license: JWalkLicense.txt in this directory.

Launching the Unbundled Software

There is no option to run this tool directly from the compressed JAR-file download bundle, so you must unbundle the software archive first. Open a command console window and, in this console, change to your chosen JWalk working directory, in which the software bundle was placed. Unpack the archive using the command:

   jar -xf JWalk11.jar

This will create a Java package directory structure under your working directory. You may now launch the JWalkUtility tool using the Java command:

   java org.jwalk.tool.JWalkUtility  [ <testClass> ]
      [ <strategy> | <modality> |  <testDepth> | 
        <generator> | <probeDepth> | <stateDepth> | 
        <convention>]*

This will invoke the main() method of the class JWalkUtility in the sub-package org.jwalk.tool, and pass to it the parameters listed on the command line, which are explained below. After the license is recognised, the tool banner should appear in the console, followed later by the program's output. Warning: if you do not have a valid license, the program will terminate immediately and a license violation message will be printed on the console!

Command Line Parameters

By default, executing the JWalkUtility tool with no command line parameters merely prints out a message about correct usage. In normal usage, the only mandatory parameter is testClass, which must appear first. All remaining parameters are interpreted as further test settings, which may be presented in any order, and default to suitable values. Please refer to the main User Guide section for an explanation of what these settings mean:

testClass - the name of the test class, which:
- if unqualified, names a class in the current working directory
- if package-qualified, names a class in a package under the working directory,
  or names a class in a package known to the Java CLASSPATH
strategy - the test strategy, one of:
- protocol - exercise all method protocols
- algebra - exercise all algebraic constructions (default)
- states - exercise all states and transitions
modality - the test modality, one of:
- inspect - just inspect the test class's interface
- explore - also exercise the specified test series (default)
- validate - also validate the results of the tests
testDepth:
- t0..tn - the maximum length of test sequences (default = t3)
- 0..n - means the same as t0..tn
generator - the name of a custom generator, which:
- must end with the substring "Generator"
- if unqualified, names a class in the current working directory
- if package-qualified, names a class in a package under the working directory,
  or names a class in a package known to the Java CLASSPATH
probeDepth:
- p0..pn - the maximum length of probe sequences (default = p12)
stateDepth:
- s0..sn - the maximum tree-depth for object state comparison (default = s0)
convention - the convention on including Object's methods, one of:
- standard - exclude all of Object's methods (default)
- custom - interleave with some of Object's methods
- complete - interleave with all of Object's methods

Parameters are processed in the following way: The first supplied parameter is treated as the testClass. Any integral parameter is treated as the testDepth. Any integer prefixed by one of the letters t, p or s is treated respectively as the testDepth, probeDepth or stateDepth. Any parameter containing the substring "Generator" is treated as the name of a generator. Any other parameter is expected to name a strategy or modality or convention (see above for the expected lower-case string values). No other settings can be altered by the JWalkUtility tool.

Standard Usage

The most comonly used testing options available through the command-line interface are described here. For details of how to upload custom generators into the JWalkUtility tool, please refer to the Custom Usage section of this guide. For advice on how best to conduct testing using the JWalkUtility tool, please refer to the main User Guide section of this website.

Default Exploration

Invoking JWalkUtility with only the test class named on the command line will cause the tool to execute with default settings for the remaining parameters. For example, the following exercises all algebraic constructions of a compiled Stack class (in the current working directory) to a test depth of 3:

   java org.jwalk.tool.JWalkUtility Stack

The output will contain a Protocol Report, an Algebra Report and several Cycle Reports for each test cycle, from depth 0 to depth 3 inclusive, followed by a Summary Report summarising the results of exploration (including how many sequences executed normally, or terminated with exceptions).

Explicit Exploration

The test strategy, modality and depth of exploration may be changed by supplying further parameters on the command line. For example, the following explores all high-level states and transitions of a ReservableBook to a test depth of 4:

   java org.jwalk.tool.JWalkUtility ReservableBook states 4

The output will contain a Protocol Report, a State Report and several Cycle Reports for each test cycle, and for each detected state, from depth 0 to depth 3 inclusive, followed by a Summary Report summarising the results of exploration (including how many sequences executed normally, or terminated with exceptions).

Inspection Only

The following example merely inspects the protocol of a Wallet class, in which case the test depth parameter for exploration is not relevant:

   java org.jwalk.tool.JWalkUtility Wallet protocol inspect

The output will only contain a Protocol Report. In the algebra strategy, inspection will also include an Algebra Report. In the states strategy, inspection will also include a State Report.

Explicit Validation

The following example causes the tool to interact with the tester. It validates all algebraic constructions of the Stack class to a test depth of 3, asking the tester interactively to confirm or reject the outcomes of certain test sequences. These are used to build up a test oracle, which may be reused in subsequent testing to predict these, and further test outcomes:

   java org.jwalk.tool.JWalkUtility Stack algebra validate 3

The interactive validation process is described below. The output will contain a Protocol Report, an Algebra Report and several Cycle Reports for each test cycle, from depth 0 to depth 3 inclusive, followed by a Summary Report summarising the results of validation (including how many outcomes were manually confirmed, or were automatically passed). Please refer to the main User Guide section for an explanation of test oracles and how they are used.

Interaction with JWalkUtility

In the validate modality, the tool interrupts its normal batch execution style to interact with the tester. In each test cycle, it selects certain key test sequences, displays these on the console and requests confirmation of their outcomes from the tester, who must enter a single-character response from:

y - meaning yes, to confirm the outcome
n - meaning no, to reject the outcome
q - meaning quit, to abort the current test series

A carriage return is considered shorthand for y. If the tester chooses to quit, the current test series is aborted and the tool produces the Summary Report directly.

An interactive query is presented at the console:

   Stack target = new Stack();
         target.isEmpty();
      ==> true
      
   Confirm? (y|n|q):

and the tool waits for the tester to enter a valid response. If the tester enters an invalid response, the query prompt is repeated. When all the results for one test cycle are complete, the tool prints these out in a Cycle Report. At the end of a validation series, the test oracle is saved in an oracle data file in the current working directory. The tester will be asked to acknowledge the creation of this file:

   The oracle file `Account.jwk' was written to:
   /programs/java/jwalk/

   OK? (y):

Entering a y or carriage return is sufficient. Other kinds of warning may also need similar acknowledgement.

Custom Usage

Sometimes the tester will want to customise the behaviour of the JWalkUtility tool, to adapt it to the needs of a particular test class. The most significant custom option is the ability to upload CustomGenerators, which control how test inputs are synthesised for the test class. Other options allow you to change the depth of probing during initial dynamic analysis, or change the meaning of object state comparison from shallow to deeper equality, or interleave the test class's methods selectively with methods inherited from the root Object, which are suppressed by default. Currently, it is not possible to change the default locations of directories for test oracles or custom generators using command line parameters.

Custom Generators

The tester may supply the name of a CustomGenerator class on the command line, which is recognised by virtue of its name ending in "Generator". A number of pre-defined CustomGenerators are supplied with the tool kit distribution in the package org.jwalk.gen, which may be uploaded using their package-qualified names. For example, RedirectInGenerator is useful if you wish to redirect standard input to read from a prepared file. Note that four CustomGenerators are always loaded. For general advice on when you should construct and load your own CustomGenerators, please refer to the section on how to create Custom Generators for JWalk.

Example Custom Generators

The following example exercises all method protocols of a Vector class, to a test depth of 2, using a user-supplied custom IndexGenerator that generates integer indices in pairs, to allow observation of successive put() and get() operations at the same index. This is different from the monotonically-increasing integer sequence generated by default for the int type:

   java org.jwalk.tool.JWalkUtility Vector protocol 2 IndexGenerator

It is assumed that IndexGenerator.class is a compiled Java class file that is accessible to Java. This custom generator will be installed as a delegate of the MasterGenerator used by the tool, and will intercept requests to generate int values. For an explanation of how it does this, please refer to the section on how to create Custom Generators for JWalk.

The following example explores the class Main, whose methods expect text input from the command line:

   java org.jwalk.JWalkUtility Main protocol 3 org.jwalk.gen.RedirectInGenerator

This initially generates a file input.txt, in the current working directory, which is used to supply text lines of input for the test class. The tester may edit this file, so that it contains the kinds of input expected.

Custom Depth Parameters

By default, integer values are always interpreted as the testDepth parameter. Integers may be prefixed by single characters: t, p or s to indicate testDepth, probeDepth or stateDepth explicitly. The following example alters the depth to which initial probing is performed, before exploring the algebraic structure of a Wallet class:

   java org.jwalk.JWalkUtility Wallet algebra p14 t5 explore

This initially probes the Wallet to depth 14 (to determine the state effects of methods over a much larger sample of test sequences) and then explores it to depth 5 during the main test series.

The following example alters the tree-depth to which object states are compared, when exploring the algebraic structure of a StackProxy class, which has no local state, but indirectly references another stack with local state:

   java org.jwalk.JWalkUtility StackProxy algebra s1 t5 explore

This compares object clusters to a tree depth of 1 (no longer shallow equality), while exploring the algebraic structure to a depth of 5 during testing.

Interleaving Object's Methods

The following example interleaves the test class's methods with some of the standard methods inherited from the root class Object, which are usually suppressed by JWalk:

   java org.jwalk.JWalkUtility Stack algebra custom 3

The custom parameter will cause JWalk to explore test sequences that interleave the inherited methods equals(), hasCode(), toString() and getClass() with Stack's local methods. Other parameters include standard and complete.

Output Format

The JWalkUtility tool prints all results on standard output. The kinds of output supported are similar to the kinds of Test Report described for the JWalkTester tool, the only difference being that all output is sent to the console as plain text.

Test Reports

The test results will appear as blocks of output text, preceded by a banner. Different blocks may appear for different test strategies, modalities and depths. In interactive modes, the output blocks will be interrupted by interactive queries to confirm or reject test outcomes. The Test Reports contain the following information:

Protocol Report - a static analysis of the test class's public constructor and method protocols (provided in all cases);
Algebra Report - a dynamic analysis of the test class's algebraic structure (only in the algebra strategy);
State Report - a dynamic analysis of the test class's high-level states and state cover (only in the states strategy);
Cycle Report - all the results for one test cycle, conducted to a given test depth (and for a given state, in the states strategy);
Summary Report - test summary statistics, giving execution behaviour and (in the validate modality only) test validation outcomes.

The kinds of information presented by these reports are discussed in the documentation for the package org.jwalk.out.

Handling Exceptions

Any notifications or warnings must be acknowledged by the tester (typing carriage return is understood to mean "OK"). Any exceptions that cause JWalkUtility to terminate will be printed on standard error.

Regent Court, 211 Portobello, Sheffield S1 4DP, United Kingdom

JWalk software testing tool suite