The JWalker Toolkit

The JWalk 1.1 Tool Suite provides a kit for building your own testing applications. You build around the JWalker core test engine. All the other featured tools were built from this kit, including this early beta release of the JWalkEditor, a combined Java editor, compiler and testing tool.

The following assumes that you have chosen your JWalk working directory, and have installed the software bundle: JWalk11.jar and an accompanying license: JWalkLicense.txt in this directory.

Unpacking the Download Bundle

To use the toolkit, you must first decompress the JAR-file download bundle. Open a command console window and, in this console, change to your chosen JWalk working directory, in which the software bundle was placed. Unpack the archive using the Java console command:

   jar -xf JWalk11.jar

• org.jwalk - the interface to the toolkit;
• org.jwalk.core - the core test engine;
• org.jwalk.gen - a custom generator kit;
• org.jwalk.out - a test result reporting kit;
• org.jwalk.test - some example test classes;
• org.jwalk.tool - JWalkTester and JWalkUtility.

Unpacking the Documentation

Detailed documentation has been created for each of these packages, supplied as a separate download bundle. This is identical to the documentation found under the Look Inside section of this website. However, if you wish to have your own local copy of this, you should download the documentation bundle, a JAR-file archive JWalk11doc.jar, placing this wherever you wish to browse hypertext files. Unpack the archive using the Java console command:

	jar -xf JWalk11doc.jar

Key Toolkit Components

The following assumes that you wish to build an application, which incorporates the core JWalker test engine. Your application is expected to provide its own user interface and display facilities. Your testing application will want to load a test class, set various test parameters and specify how to receive call-back communications dispatched by a JWalker, which will cause your application to interact with the tester, or display test results. To do this, your application will create a single instance of the JWalker test engine; and then access its Channels API to set up the communication channels; and access its Settings API to supply the desired test parameters and upload the test class.

The JWalker's Channels API observes Java's event-based model for communication. This allows the core test engine to be decoupled from different kinds of front-end for handling input and output. From version 1.1, the core test engine may also be run in a separate worker thread, in the background. Your application must register suitable event listeners, which are capable of handling the events dispatched by the engine. The two kinds of event are QuestionEvent, encapsulating an interactive query or notification, and ReportEvent, encapsulating a test report. Your application will register objects that satisfy the QuestionListener and ReportListener interfaces with the Channels object, in order to process these events.

The JWalker's Settings API offers fine-grained control over all test settings, ranging from the directories in which files are found and placed, a flexible facility for loading (and re-loading) the test class and custom generators, to complete control over the strategy, modality, convention, test depth, probe depth and state depth parameters. In particular the test class may be re-loaded multiple times into the same Java runtime (after being modified) and will execute in its modified form. The tester may supply one or more generators satisfying the CustomGenerator interface, to take control over how test input values are synthesised.

The following examples indicate how your own testing application might interface with the components in the JWalk 1.1 Tool Suite. The application will eventually create a single instance of the JWalker core test engine, set up callbacks, upload test settings and invoke the JWalker's main execute() method. Warning: if you do not have a valid license, the application will terminate. If it was launched from a console, a license violation message will be printed on the console; otherwise it will terminate silently!

Example 1: Protocol Exploration

The following fragment illustrates the style of code necessary to set up the test class, construct a JWalker, register suitable listeners, set test parameters and execute a test series. This example will explore all method protocols of a Stack class to a depth of 4, and will execute as a single-threaded application.

   Class<?> testClass = Stack.class; 

   JWalker walker = new JWalker();
   Channels channels = walker.getChannels();
   Settings settings = walker.getSettings();
	
   channels.addQuestionListener(new MyQuestionListener());
   channels.addReportListener(new MyReportListener());

   settings.setTestClass(testClass);	
   settings.setStrategy(Strategy.PROTOCOL);
   settings.setModality(Modality.EXPLORE);
   settings.setTestDepth(4);
	
   walker.execute();

This first example assumes that your application has supplied a reference to the test class directly - in other words, it was loaded into memory by your application. Note that Java may cache a loaded class and this may prevent you from reloading it, unless you provide a customised ClassLoader, or allow the Settings API to reload the class by name (see below), using its own ClassLoader.

Example 2: Algebraic Exploration

The following fragment illustrates the style of code necessary to initiate the algebraic exploration of a Stack class to a depth of 3, using the JWalker API as before, but this time supplying slightly different test settings, and running the test engine in a separate asynchronous thread. This can be useful if the GUI launching the engine should remain active, while the test engine runs in the background. However, third-party GUI designers must then also ensure that all GUI threads can recover from OutOfMemoryError exceptions, since you cannot then predict in which thread this condition will arise.

	String className = "Stack";   // or by other means
	
	JWalker walker = new JWalker();
	Channels channels = walker.getChannels();
	Settings settings = walker.getSettings();
	
	channels.addQuestionListener(new MyQuestionListener());
	channels.addReportListener(new MyReportListener());
	
	settings.setTestClass(className);
	settings.setStrategy(Strategy.ALGEBRA);
	settings.setModality(Modality.EXPLORE);
	settings.setTestPathDepth(3);
	
	Thread thread = new Thread(walker);
	thread.setPriority(Thread.NORM_PRIORITY);
	thread.start();

This example also illustrates how to set the test class by name, rather than by supplying the class object directly. This allows the test engine to load and reload the test class from a compiled file, even if it has been modified since. The test class will be loaded in the manner specified in setTestClass(String). Java will seek to load the class from the directory specified by getTestClassDirectory(). If the test class is packaged, you should load it using its full package-qualified name from the working directory above the package. See the Settings API for details of how to change the working directory.

Example 3: State-Space Validation

The following fragment illustrates how a custom generator may be supplied via the Settings API; and how the directory in which oracles are saved may be set explicitly (the default is the test class directory). This fragment will perform state-validation on a Vector class to a transition path depth of 2, using a user-supplied custom IndexGenerator as the generator. This example runs in a single thread, and interacts with the human tester on standard input and output.

	Class<?> testClass = Vector.class;       // or by other means
	Class<?> custom = IndexGenerator.class;  // or by other means
	
	JWalker walker = new JWalker();
	Channels channels = walker.getChannels();
	Settings settings = walker.getSettings();

	Console console = new Console();
	channels.addQuestionListener(console);
	channels.addReportListener(console);
	
	settings.setTestClass(testClass);
	settings.addCustomGenerator(custom);
	settings.setOracleDirectory(new File("test/oracles/"));
	settings.setStrategy(Strategy.STATES);
	settings.setModality(Modality.VALIDATE);
	settings.setTestPathDepth(2);
	
	walker.execute();

In this example, the oracle data file will be saved in the file: "test/oracles/Vector.jwk". The oracle data file is valid for retesting using the same generators, but not necessarily valid for other generators, which may synthesise different test inputs in different orders.

This example uses a default Console object to fulfil both the roles of the QuestionListener and the ReportListener interfaces. Console is supplied with the toolkit and it unpacks the various communicated events and pipes them to standard output, expecting responses from the tester on standard input. If the third-party program fails to set any listeners, a Console is then used by default.

Handling JWalk Exceptions

While the JWalk 1.1 Tool Suite recovers automatically from a number of anticipated exceptional situations, such as when methods of the test class fail, or memory is exhausted during testing, or when the tool fails to detect all the expected high-level states, your application will need to recognise and possibly handle the following five kinds of exception. The first two are raised while setting test parameters, while the last three may be raised during test execution.

• LoaderException - raised if the test class, or a custom generator could not be loaded (wrong pathname given)
• SettingsException - raised if the third-party software supplies invalid test settings (bad constants, out of range values)
• PermissionException - raised if the test class refuses permission to be executed (wrong visibility, security permissions)
• GeneratorException - raised if test inputs could not be synthesised, or a generator fails (unknown types, bad user-supplied generator)
• ExecutionException - raised if any constructor or method could not be invoked, or behaved randomly (security, nondeterminism)

The following gives more details about how to construct listeners that satisfy the QuestionListener and ReportListener interfaces. Further information is also given about the Report and Question objects returned inside events. For full details, please refer to the package documentation for org.jwalk.out.

The Contract for ReportListener

A ReportListener is designed to publish test reports emitted by the JWalker test engine. It must provide the single method void publish(ReportEvent event) which accepts a ReportEvent and publishes the contents of the report that this contains. That is, processing a ReportEvent should cause your application to display the contents of a test report in the desired manner. A simple skeleton implementation might be:

   void publish(ReportEvent event) {
      Report report = event.getReport();
      System.out.println(report.getContent());
   }

   void publish(ReportEvent event) {
      Report report = event.getReport();
      switch(report.getEdition()) {
         case Edition.PROTOCOL_REPORT :
            handle((ProtocolReport) report); break;
         case Edition.ALGEBRA_REPORT :
            handle((AlgebraReport) report); break;
	  ...
      }
   }

The Contract for QuestionListener

A QuestionListener is designed to interact with the human tester in order to elicit a response to a question; so it must be able to handle both input and output. It must provide the single method: Answer respond(QuestionEvent event) which accepts a QuestionEvent and returns an enumerated value from the Answer type, indicating the user's answer to the question. That is, processing a QuestionEvent should cause your application to ask the tester a question, and accept a reply from the tester, which can be converted into an Answer. A simple skeleton implementation might be:

   Answer respond(QuestionEvent event) {
      Question question = event.getQuestion();
      System.out.println(question.getContent());
      do {
         System.out.println(question.getQuestion());
         /* Get a response from the user */
      }
      while( /* Response not acceptable */ );
      /* Convert response to an Answer */
      return answer;
   }

   Answer respond(QuestionEvent event) {
      Question question = event.getQuestion();
      switch(question.getEdition()) {
         case Edition.CONFIRM_DIALOG :
            return handle((Confirmation) question);
         case Edition.NOTIFY_DIALOG :
            return handle((Notification) question);
	  ...
      }
   }

The Report and Question Components

All information emitted by the JWalker test engine is some kind of Report. A Question is merely a kind of report that solicits a response from the tester. Both of these produce formatted output strings using getContent(); and questions also produce a formatted query prompt strings using getQuestion(). The various kinds of Report and Question emitted by the tool suite may be classified in a hierarchy:

• Report - the abstract ancestor of all reports
  • ProtocolReport - reports a class's method protocols
  • AlgebraReport - reports a class's algebraic structure
  • StateReport - reports a class's high-level states
  • StatisticalReport - abstract ancestor of test results
    • CycleReport - gives the results of a single test cycle
    • SummaryReport - summarises a complete test series
  • Question - the abstract ancestor of all questions
    • Confirmation - a request to confirm or reject a test outcome
    • Notification - a notification requiring an acknowledgement

Raising Different kinds of Dialog

The JWalker test engine raises five kinds of exception, and deals with other conditions by dispatching Notification objects to warn the tester. From version 1.1, when the test engine is launched in a separate thread, a SILENT notification is dispatched to signal the end of the thread's execution. This allows a third-party GUI to synchronise itself with the test engine. Also, exceptions raised by the test engine in the spawned thread can only be communicated back to the main thread as Notification objects. Third-party code should distinguish different kinds of Notification and pop up a suitable dialog:

        void handle(Notification notice) {
            switch(notice.getUrgency()) {
                case Urgency.SILENT :
                    // A silent notification, no response needed
                case Urgency.NOTICE :
                    // A notice, trigger an information dialog
                case Urgency.WARNING :
                    // A warning, trigger a warning dialog
                case Urgency.ERROR :
                    // An exception, trigger an error dialog
            }
        }