|
|||||||||
| PREV PACKAGE NEXT PACKAGE | FRAMES NO FRAMES | ||||||||
See:
Description
| Interface Summary | |
|---|---|
| Generator | Generator is the interface implemented by every test input generator. |
| QuestionListener | QuestionListener is the listener interface for receiving question events. |
| ReportListener | ReportListener is the listener interface for receiving report events. |
| Class Summary | |
|---|---|
| Channels | Channels is the API through which client programs set up two-way communication channels with a JWalker. |
| Console | Console provides a default channel for communication via the command line. |
| GeneratorLoader | GeneratorLoader is a class loader for loading custom generators. |
| JWalker | JWalker is the top-level API class through which third-party software communicates with the different lazy systematic unit testing algorithms for unit testing Java classes. |
| QuestionEvent | QuestionEvent is a kind of event that notifies about a Question. |
| ReportEvent | ReportEvent is a kind of event that notifies about a Report. |
| Settings | Settings is the API through which client programs set up the various test parameters for a JWalker. |
| TestClassLoader | TestClassLoader is a class loader for loading the test class. |
| Enum Summary | |
|---|---|
| Convention | Convention enumerates how many built-in methods are to be included in tests. |
| Error | Error enumerates the different kinds of JWalkException that can be raised. |
| Modality | Modality enumerates the levels of static, dynamic and interactive analysis. |
| Strategy | Strategy enumerates the protocol, algebra and state-based test strategies. |
| Exception Summary | |
|---|---|
| ExecutionException | ExecutionException signals when a test sequence did not execute properly. |
| GeneratorException | GeneratorException signals when a Generator could not synthesise a value. |
| JWalkException | JWalkException is the top-level exception handled by JWalk. |
| LoaderException | LoaderException signals when a test class or generator could not be loaded. |
| PermissionException | PermissionException signals the refusal to execute a class or a generator. |
| SettingsException | SettingsException signals when incorrect test settings have been supplied. |
This package contains the JWalk 1.1 lazy systematic unit testing tool suite, © Anthony J H Simons, 2006-2011. This is the root package for the JWalk 1.1 tool suite, containing the component kit for building your own JWalk 1.1 single- or multi-threaded unit testing applications. The various sub-packages found within this package contain further components, organised by their function; and one sub-package contains the finished testing tools JWalkTester and JWalkUtility, which may be executed directly. In overview:
org.jwalk is the entry point into the JWalk
component kit.org.jwalk.core contains the core JWalk
test engine.org.jwalk.out contains the test reports
emitted by JWalk.org.jwalk.gen contains custom test
input generators.org.jwalk.test contains sample test
classes.org.jwalk.tool contains finished JWalk
testing tools.org.jwalk.tool.
This package is provided as an executable JAR-file, which may be launched directly on most platforms. On some platforms, double-clicking on the JAR-file icon is sufficient, while on other platforms, it may be necessary to launch Java explicitly, specifying the JAR-file on the command line. Since the JAR-file is a Java archive, the user may also choose to unbundle the archive, creating a package directory structure containing the compiled class files of the JWalk 1.1 tool suite. More than one entry point is possible, when executing the unbundled software.
In all cases, the user must have a valid JWalk license to use the software, which may be obtained from the developers of the software, and is free to academic and trial commercial users. Please refer to the JWalk website for further details:
org.jwalk.tool.JWalkTester,
one of the two tools currently in the JWalk tool suite. Launching the
JAR-file directly will therefore execute the flagship tool
JWalkTester, a GUI-based Java class unit tester. This
tool is described in detail in the sub-package org.jwalk.tool.
The other tool contained in the package is JWalkUtility, a
command-line version of the same test engine.
It is also possible to unpack the JAR archive as a tree of packages (this package and its included sub-packages) and invoke the Java runtime on either tool. The GUI-based tool JWalkTester is invoked by:
java org.jwalk.tool.JWalkTesterand the command-line based tool JWalkUtility is invoked by:
java org.jwalk.tool.JWalkUtilityFurther tools may be added in later releases.
In general, this package and its sub-packages (not including
org.jwalk.tool) constitute a toolkit, out of which developers
may construct bespoke JWalk testing applications. The rest of this package
documentation describes how to incorporate the toolkit in such third-party
applications.
The following assumes that the developer wishes to build a third-party
application, which incorporates the JWalk 1.1 testing
engine. The application is expected to provide its own user interface and
display facilities. The application will need to interface with some of
the modular components in this package, org.jwalk, and with
some of the components from sub-packages, in particular
org.jwalk.out.
This introduction provides only an overview of how the modular API can be
used. Please refer to the more detailed API descriptions for the main
JWalker test engine and its component APIs Settings
and Channels. Many more parameters can be fine-tuned than are
described in this brief introduction.
The third-party program will want to load a test class, set the test
parameters and specify how to receive call-back communications submitted
by a JWalker in the form of QuestionEvent and
ReportEvent events. To do this, the third-party program must
create an instance of the main JWalker test engine; and then
access its Channels API to set up the communication
channels; and then access its Settings API to supply the
desired test parameters and upload the test class.
The following fragment illustrates the style of code necessary to set
up the test class, construct a JWalker, register third-party
listeners, set test parameters and execute a test series. This example
will explore all method protocols of a Stack class to a
depth of 4, and will execute as a single-threaded application.
Class<?> testClass = Stack.class; // or by other means
JWalker walker = new JWalker();
Channels channels = walker.getChannels();
Settings settings = walker.getSettings();
channels.addQuestionListener(new MyQuestionListener());
channels.addReportListener(new MyReportListener());
settings.setTestClass(testClass);
settings.setStrategy(Strategy.PROTOCOL);
settings.setModality(Modality.EXPLORE);
settings.setTestPathDepth(4);
walker.execute();
The classes MyQuestionListener and
MyReportListener
are supplied by the third-party program, according to the interface
specifications in QuestionListener and
ReportListener.
These classes may interact with the tester through dialogs, and display the
contents of reports in any desired manner. See the package documentation for org.jwalk.out for further details of kinds of reports
emitted by the JWalk 1.1 tool suite. Protocol testing to any great depth will soon lead to OutOfMemoryError exceptions, but in single-threaded execution, these are handled safely by the engine.
The following fragment illustrates the style of code necessary to
initiate the algebraic exploration of a Stack class to a
depth of 3, using the JWalker API as before, but this time
supplying slightly different test settings, and running the test engine in a separate asynchronous thread. This can be useful if the GUI launching the engine should remain active, while the test engine runs in the background. However, third-party GUI designers must then also ensure that all GUI threads can recover from OutOfMemoryError exceptions, since you cannot then predict in which thread this condition will arise.
String className = "Stack"; // or by other means
JWalker walker = new JWalker();
Channels channels = walker.getChannels();
Settings settings = walker.getSettings();
channels.addQuestionListener(new MyQuestionListener());
channels.addReportListener(new MyReportListener());
settings.setTestClass(className);
settings.setStrategy(Strategy.ALGEBRA);
settings.setModality(Modality.EXPLORE);
settings.setTestPathDepth(3);
Thread thread = new Thread(walker);
thread.setPriority(Thread.NORM_PRIORITY);
thread.start();
The test engine will run as a parallel worker thread with the same priority as the code that launched the thread. Reducing the priority by one or two will make the worker thread run in the background. All exceptions thrown by the worker thread must be handled within the thread, but will notify the
third-party supplied QuestionListener about any exceptions, just like any other notification, such that the third-party system may choose to report the fault. The worker thread eventually sends a silent Notification to signal when it has finished testing.
See the package documentation for org.jwalk.out for further details of how to process notifications.
This example also illustrates how to set the test class by name,
rather than by supplying the class object directly. The test class will be
loaded in the manner specified in
Settings.setTestClass(String).
In particular, Java will search for the test class in the directory currently specified by
Settings.getTestClassDirectory(). See the
Settings API for details of how to change this.
The following fragment illustrates how a custom generator may be supplied
via the Settings API; and how the directory in which oracles
are saved may be set explicitly (the default is the test class directory).
This fragment will perform state-validation on a Vector class
to a transition path depth of 2, using a user-supplied custom
IndexGenerator as the generator. This example runs in a single thread, and interacts with the human tester on standard input and output.
Class<?> testClass = Vector.class; // or by other means
Class<?> custom = IndexGenerator.class; // or by other means
JWalker walker = new JWalker();
Channels channels = walker.getChannels();
Settings settings = walker.getSettings();
Console console = new Console();
channels.addQuestionListener(console);
channels.addReportListener(console);
settings.setTestClass(testClass);
settings.addCustomGenerator(custom);
settings.setOracleDirectory(new File("test/oracles/"));
settings.setStrategy(Strategy.STATES);
settings.setModality(Modality.VALIDATE);
settings.setTestPathDepth(2);
walker.execute();
This uses a default Console object to fulfil both the roles
of the QuestionListener and the ReportListener
interfaces.
Console is supplied with the toolkit and it unpacks the various
communicated events and pipes them to standard output, expecting responses
from the tester on standard input. If the third-party program fails to set
any listeners, a Console is then used by default.
Custom generator classes may be supplied by name, as well as by value,
in a similar manner to the test class. More than one custom generator may
be supplied. In this example, the oracle data file will be saved in the
file: "test/oracles/Vector.jwk". The oracle data file is valid
for retesting using the same generators, but not necessarily for other
generators, which may synthesise other test inputs in different orders.
While the JWalk test engine automatically handles a number of exceptional cases, such as when one of the tested methods of the test class fails, or when Java's heap memory capacity is exhausted during testing, or when the tool fails to detect all the expected high-level states, third-party software must also be able to recognise and possibly handle the following five kinds of exception. The first two are raised while setting test parameters, while the last three may be raised during test execution.
LoaderException - raised if the test class, or a custom
generator could not be loaded (wrong pathname given)SettingsException - raised if the third-party software
supplies invalid test settings (bad constants, out of range values)PermissionException - raised if the test class refuses
permission to be executed (wrong visibility, security permissions)GeneratorException - raised if test inputs could not be
synthesised, or a generator fails (unknown types, user generator)ExecutionException - raised if any constructor or method
could not be invoked, or behaved randomly (security, nondeterminism)MyQuestionListener is notified, and may respond appropriately. All of the above exceptions are
subclasses of JWalkException, so third-party software may
choose to trap just the one exception, or handle them more selectively.
Each kind of exception provides access methods to inquire about the cause
of the exception more finely, which may be used to tailor error messages.
|
|||||||||
| PREV PACKAGE NEXT PACKAGE | FRAMES NO FRAMES | ||||||||