|
|||||||||
| PREV PACKAGE NEXT PACKAGE | FRAMES NO FRAMES | ||||||||
See:
Description
| Class Summary | |
|---|---|
| AbstractDialog | AbstractDialog is the abstract superclass of custom dialogs. |
| AbstractPanel | AbstractPanel is the abstract superclass of panels that supply settings. |
| ClassFinderPanel | ClassFinderPanel is a kind of panel for selecting the test class. |
| ConfigureDialog | ConfigureDialog is a kind of dialog for configuring advanced settings. |
| CustomPanel | CustomPanel is a kind of panel that launches dialogs for custom settings. |
| GeneratorDialog | GeneratorDialog is a kind of dialog for selecting custom generators. |
| JWalkTester | JWalkTester is a simple GUI for conducting JWalk tests. |
| JWalkUtility | JWalkUtility is a command-line utility for lazy, systematic unit testing. |
| QuestionMaster | QuestionMaster implements the question-response interaction with the tester. |
| ReportPublisher | ReportPublisher implements the reporting of test results to the tester. |
| TestExecutePanel | TestExecutePanel is a kind of panel for running the JWalker test engine. |
| TestSettingsPanel | TestSettingsPanel is a kind of panel for selecting the test settings. |
This sub-package for JWalk 1.1 contains two JWalk class unit testing tools, © Anthony J H Simons, 2006-2011. To use this software, you will require a license from the developers. Free licenses are available for individual research purposes and for restricted commercial trials. Please see the JWalk Website for more details.
This package provides two finished testing tools, implemented on top of the JWalk 1.1 lazy systematic unit testing toolkit. These are the JWalkTester tool, a GUI-based test generator and execution tool; and the JWalkUtility tool, a command-line driven utility that prints results to standard output. In future releases, further tools may be added to this package. Other parts of the JWalk 1.1 tool suite are described in the following packages:
org.jwalk is the entry point into the
JWalk component kit.org.jwalk.core contains the core JWalk
test engine.org.jwalk.out contains the test reports
emitted by JWalk.org.jwalk.gen contains custom test
input generators.org.jwalk.test contains sample test
classes.org.jwalk.tool contains finished JWalk
testing tools.The following concepts are common to all the tools in the JWalk 1.1 lazy systematic unit testing toolkit.
testClass,
which must always be supplied by the tester and uploaded by the testing tool.
Other test parameters have suitable default settings, which the tester may
choose to override.
The testing can follow different strategies. The strategy
parameter is one of: protocol, meaning all interleaved method
protocols; algebra (the default setting), meaning all algebraic
constructions; or states, meaning all high-level design states.
The effect of choosing one of these strategies is the following:
protocol strategy: JWalk creates test sequences corresponding
to every possible interleaved ordering of the test class's constructors and
methods;
algebra strategy: JWalk creates test sequences corresponding
to every novel algebraic construction, that is, sequences which drive the test
object into novel concrete states;
states strategy: JWalk creates test sequences corresponding
to the state cover, transition cover, and n-switch cover, which drive the test
object through all of its high-level states and transitions.
true or false results.
Testing can be conducted to different levels. The modality
parameter is one of: inspect, meaning to stop after analysing the
structure and behaviour of the test class; explore (the default
setting), meaning to exercise and display the result of every test sequence
and let the tester interpret each result; or validate, meaning to
verify the result of exercising each test sequence semi-automatically, using
an incrementally derived test oracle.
Testing can be conducted to different depths. The depth
parameter corresponds to the maximum length of generated test sequences (the
default is 3). In the protocol and algebra modes,
the depth is the same as the maximum number of methods to be invoked after
initial construction. In the states mode, the number of methods
is counted starting from each high-level design state instead (a prefix
sequence from the state cover is used to reach each high-level state, which
is extended by each test sequence).
Testing can be customised in various ways. The most common reason for
doing this would be to take control of how test inputs are synthesised.
To do this, the tester must supply one or more CustomGenerators
that synthesise values in the manner and order desired by the tester.
Other settings can be customised in the different testing tools.
Each installation of the JWalk tool suite requires a specific license for that user and installation. The license may be obtained, in most cases for free, from the developers of the software. See the JWalk website for details. The license is a text file which must be placed in the working directory from which the JWalk tools are launched. The license is not transferrable, either to a different user, or different installation.
The classic JWalk 1.1 lazy systematic testing tool is JWalkTester, a GUI-based class unit testing tool. This is the default tool that is launched when the JAR-file download bundle is executed directly, either by double-clicking on the icon (if your platform supports this), or by using the Java command:
java -jar JWalk11.jar
Warning: if you do not have a valid
JWalk license, the software will terminate immediately.
In direct-launch mode, this will also happen silently.
Rather than launch the JAR-file directly, the user may wish to unbundle the JAR-file archive first, and then run the tool. The JAR-file archive may be unbundled using the file extraction command:
jar -xf JWalk11.jar
and then the JWalkTester tool may be launched using
the Java command:
java org.jwalk.tool.JWalkTester
which will access the tool from within the package structure created in
the current directory. The main frame for the tool should appear, using
the standard look-and-feel of the operating system, centrally in the
tester's monitor screen.
Warning: if you do not have a valid
JWalk license, the software will terminate immediately,
with a short error message indicating the kind of license violation
detected.
The tester must supply a test class. The tool gives you the choice of browsing for a class, or entering the name of the class directly. Because of the likelihood of classes being contained in named packages, the test class is selected in two stages. Firstly, the tester must set the working directory; then select the test class, which may be in the working directory or in some named sub-package under the working directory. If an incorrect path to the test class is specified, the tool will not be able to load the test class.
If the test class exists in the default package, then either of the Browse-buttons may be used to select the class directly, and the tool will sort out the location and the name of the class. If the test class exists in a named sub-package, then the top Browse-button must be used to set the location, and the bottom Browse-button to select the test class within its sub-package. The first text field displays the directory from which the test class is loaded; and the second text field displays the package-qualified name of the test class.
A test class may be specified by typing its package-qualified name directly into the lower text field. Any named packages must either be listed on the standard java CLASSPATH, or exist under the current working location.
The tester may then supply test settings for the current test series. These control how the tool executes. The standard settings may be chosen from combo boxes and a number spinner:
To execute the chosen test series, click on the button featuring the iconic "JWalking Jimbo" cartoon character. Initially, "Jimbo" is in a standing posture, indicating that the tool is idle and ready. Moving the mouse cursor over him causes "Jimbo" to get ready to run. Clicking on the button will activate the "JWalking" animation, which continues until testing is over, whereupon "Jimbo" reverts to standing.
If the animation starts to slow down, this indicates that the tool is beginning to run out of memory. If the animation freezes completely, there will be a short delay, then the tool will recover from the out-of-memory condition, reporting this to the user via a dialog. The user may always click on the "Cancel" button to terminate execution.
Clicking on the test button causes the current test class and test
settings to be uploaded to the core JWalk test engine. The first time
the tool is used, there may be a slight delay as Java loads the relevant
classes. Subsequent re-testing will be quicker. In the
ALGEBRA and STATES test strategies, the tool will
also perform an initial dynamic analysis of the test class, by executing
probing sequences. This may add a further noticeable delay to the
perceived response time. This can be reduced by altering the
probeDepth parameter in the custom settings.
This is launched by clicking the Generators-button on the Custom Settings panel. The tester may upload any number of custom generators, in two stages. First, a java class implementing the generator must be selected in the upper panel, in the same manner that the test class was selected above; then, this may be added to the current set of custom generators using the Add button in the lower panel.
The lower panel allows the tester to add or remove custom generators from the current set. Clicking on any generator in the current working set makes this the selected generator in the upper panel. Clicking the Add button adds the currently selected generator to the working set. Clicking the Remove button removes the currently selected generator from the working set. The modifications are uploaded to the core JWalk test engine when the tester clicks the OK-button. Clicking the Cancel-button will exit the dialog without making any changes.
This is launched by clicking the Configuration-button on the Custom Settings panel. The tester may modify a number of additional settings which usually don't need changing. These may be chosen using a combo-box and a couple of number spinners:
Convention parameter affects whether built-in methods from the
root class Object are included in tests (the default is for these
to be excluded).
You can choose to include just the four methods
equals(),
hashCode(),
toString() and
getClass();
or all ofObject's methods, including the thread synchronising
methods (which raise exceptions in a single thread).
The probeDepth parameter controls how deeply the test class is
explored in the initial probing phase. This can be raised, if states are
not being discovered, or lowered if memory is being exhausted.
The stateDepth parameter controls whether object states are
compared for equality in a shallow, or deeper way (the default is 0, meaning
shallow equality). Raising this causes clusters of reachable objects to be
compared. The changed configurations are uploaded to the core JWalk test
engine when the tester clicks the OK-button. Clicking the
Cancel-button will exit the dialog without making any changes.
The test results will appear as tabbed panes in the right hand output panel. Different tabs may appear for different test strategies, modalities and depths. The tabs stack in rows. Clicking on any tab will bring the contents of the tab to the front, and also bring its row of tabs to the front row. The tabs contain the following information:
org.jwalk.out.
During execution, the tool will sometimes interact with the tester in various ways. The main dialogs are:
The original JWalk 1.1 lazy systematic testing tool is JWalkUtility, a command-line utility program for class unit testing that prints results to standard output. This tool may be executed by first unbundling the JAR-file archive, using the file extraction command:
jar -xf JWalk11.jar
and then launching the JWalkUtility tool using the java
command:
java org.jwalk.tool.JWalkUtility
which will access the tool from within the package structure created in
the current directory. A banner should be printed on standard output,
indicating the authorship and version of the tool.
Warning: if you do not have a valid
JWalk license, the software will terminate immediately,
with a short error message indicating the kind of license violation
detected.
testClass and any
remaining parameters will be interpreted as further test settings. These
can be supplied in any order. Suitable default values are chosen for any
missing parameters. The compiled class file for the test class must be
accessible to Java (on the CLASSPATH; or in the current directory).
The remaining parameters are any of the test strategy, test
modality (values supplied as lower-case strings), the test
depth (supplied as a positive integer) and the name of any custom
generator (supplied as a string in capital-case, as per the usual
naming style for Java classes). Parameters are processed in the following way:
any integral parameter is treated as the depth. Any parameter containing the
substring "Generator" is treated as the name of a generator. Any other
parameter is expected to name a strategy or modality
(see above for expected values).
Invoking the command line utility with only the test class named on the
command line will cause the tool to execute with default settings for the
remaining parameters. For example, the following exercises all algebraic
constructions of a compiled Stack class (in the current
working directory) to a depth of 3:
java org.jwalk.JWalkUtility Stack
The test strategy, modality and depth of exploration may be changed by
supplying further parameters on the command line. For example, the
following explores all high-level states and transitions of a
ReservableBook to a transition depth of 4:
java org.jwalk.JWalkUtility ReservableBook states 4
The high-level states will be determined by natural predicates, such as
isOnLoan() and isReserved(), in the test class's
API. Results will be generated for each state and each cycle of increasing
transition path length.
The following example merely inspects the protocol of a Wallet
class, in which case the depth parameter for exploration is not relevant:
java org.jwalk.JWalkUtility Wallet protocol inspect
The following example causes the tool to interact with the tester. It
validates the algebraic structure of the Stack class,
exploring to depth 3 and also asking the user to confirm or reject the
outcomes of certain test sequences:
java org.jwalk.JWalkUtility Stack algebra validate 3
When operating in validation modality, the tool prints out selected test
sequences and requests confirmation of the test outcomes from
the tester, who enters 'y', 'n' or 'q' at the console (carriage return is
interpreted as shorthand for 'y'; entering 'q' quits the current test series).
The tester can take control of how test input values are supplied to
the test class's methods by supplying one or more custom generators. The name
of each custom generator is supplied as a command line parameter in capital
case, ending with the substring "Generator" to be correctly identified by
the tool. The name should refer to a class that satisfies the interface of
CustomGenerator. The class must be accessible to Java
(on the CLASSPATH; or in the current directory).
The following example exercises the method protocols of a compiled
Vector class, using a custom IndexGenerator that
generates integer indices in pairs, to allow observation of successive
put() and get() operations at the same index.
java org.jwalk.JWalkUtility Vector protocol explore IndexGenerator
It is assumed that IndexGenerator.class is a compiled Java class file that is accessible to Java. This custom generator will be
loaded and installed as a delegate generator. Test exploration will be
to the default depth of 3.
The following example alters the depth to which initial probing is
performed, before exploring the algebraic structure of a
Wallet class:
java org.jwalk.JWalkUtility Wallet algebra p14 t5 explore
This initially probes the Wallet to depth 14 (to determine
the state effects of methods) and then explores it to depth 5 during the
main test series.
The following example alters the tree-depth to which object states are
compared, when exploring the algebraic structure of a
StackProxy class, which has no local state, but indirectly
references another stack with local state:
java org.jwalk.JWalkUtility StackProxy algebra s1 t5 explore
This compares object clusters to a tree depth of 1 (no longer shallow
equality), while exploring the algebraic structure to a depth of 5
during testing. Note the character prefixes p, s, t to
denote probe depth, state depth and test depth, respectively. If no
prefix appears before an integer value, test depth is assumed.
Other kinds of file path settings cannot be changed with the JWalkUtility tool.
The tool prints all results on standard output. After the intitial banner, the result of inspection is printed, displaying the protocol of the test class (optionally followed by an analysis of its algebraic structure; or an analysis of its state space, including the state cover test set). After this, one or more test cycle reports may be printed, for test cycles containing sequences of increasing length. Finally, a test summary report may be printed, giving summary statistics. In interactive modes, the results will appear in blocks, after the tester has completed the confirmations for that cycle.
Any notifications or warnings must be acknowledged by the tester (typing carriage return is understood to mean "OK"). Any oracle file will be saved in the same directory as the test class. Any exceptions will be printed on standard error.
|
|||||||||
| PREV PACKAGE NEXT PACKAGE | FRAMES NO FRAMES | ||||||||