The JWalk Weblog, 2011

This is the JWalk blog for 2011! It describes the development history of the revised multi-threaded version of the JWalk tool, against a background of a year which included the Royal Wedding between Prince William of Windsor and Catherine Middleton. The main entries for the year cover the period from April to June, when this project was in progress. The end result was the shiny new JWalk 1.1 tool suite, created in response to feedback from our registered Beta-users. This significantly improved the user-experience, with a new multi-threaded user interface that supports more concurrent styles of interaction while testing is going on. We also added some on-board tool-tips and upgraded the images on the website to show the latest Windows 7 and Linux deployment of the tools.

So, what did the new version do? It allowed the user to interact immediately with tabbed panes containing test results as they were produced, while the test engine was still computing further test results. The animated icon of "Jaywalking Jimbo" now provided a better indication of how the tool is working. The figure is stationary when the tool is idle, and walking while the tool is active. The speed of walking also reflects how hard the tool is working, slowing down as the memory exhaustion limit is approached. There was also a new option to cancel a test series in progress, before the memory limit was reached. The tools still recovers automatically from memory exhaustion - this was the most difficult thing to achieve under multi-threading!

Wednesday, 21 December 2011

JWalk rules, when it comes to testing states and transitions. I would still like to come up with a better way of populating the test sequences with suitable test inputs. The default input generators are fine, but from time to time, we get feedback from a beta-user who wants more specific inputs for their example. Now, this gives me an idea. Why not have a community challenge to develop new custom generators for JWalk? It is possible that several users might want to create something similar, so why not pool the effort?

If you're interested in this idea, I suggest you first read this website's guide to customising JWalk with your own custom generators. If you come up with a generically useful custom generator and would like to share it with the JWalking community, I will be happy to share it here. You will get credit, of course!

Tuesday, 29 November 2011

Finally, we pushed JWalk out into the curriculum today. After a big re-write of three modules on Human-Computer Interaction, Database Systems, and Object-Oriented Design, we have come up with a big module called Human Centred Systems Design. However, it took two attempts to get the content properly balanced. Last year, we ended up with no room left for software testing. This year, we have a whole double hour devoted to it. So, I was able to cover validation, verification and testing from several viewpoints - random, code-based and model-based testing. I showed the class how to construct test cases for equivalence-partition testing. I showed them how to construct test cases to explore all states and transitions. Then, I showed them JWalk.

Guess which approach they wanted to try the most? By the way, that's about 100 potential new JWalkers.

Thursday, 1 September 2011

We're happy that the new JWalk release is running smoothly. So, I hear you say, what's new in the JWalk 1.1 tool suite? For the first time, we have multi-threaded the user interface of the JWalkTester and we launch the test engine as a background worker process. This allows the user to interact immediately with tabbed panes containing test results as they are produced, while the test engine is still computing further test results. The animated icon of "Jaywalking Jimbo" now provides a better indication of how the tool is working. The figure is stationary when the tool is idle, and walking while the tool is active. The speed of walking also reflects how hard the tool is working, slowing down as the memory exhaustion limit is approached. There is also a new option to cancel a test series in progress, before the memory limit is reached. The tools still recover automatically from memory exhaustion.

For the JWalkUtility tool, we have increased the number of options that can be set on the command line. In addition to setting the test depth, it is now possible to set the prior probe depth for algebraic exploration, and the state depth for testing the equality of object trees, using new command-line switches. The keywords standard, custom, complete are also recognised as switches for including none, some or all of root Object's inherited methods in interleaved sequences. Standard directories are still assumed for the test class and its oracle file.

Wednesday, 22 June 2011

Today, I finished taking a whole new series of screen captures of the JWalk 1.1 tools. This time, featured graphics show the JWalkTester tool running under Windows 7; and the JWalkUtility tool running inside a command console in Linux, running the Gnome desktop. I also managed to get a capture of the idle state, featuring the new introductory tab; and I had to run the tool under the protocol strategy to quite high test depths, so that I could capture Jimbo while he was still running, and also show off the enabled cancel-button. See the images on this blog for examples. I also updated the user guide to include these new images. I have included more screenshots on each manual page than for the previous version, which may make these pages a bit more attractive.

Friday, 10 June 2011

Finished the rewrite of the JWalk help pages today. Mostly, this was a reworking of the manual pages describing each of the tools and especially describing how to use the new toolkit API, with the option of multi-threading. The changes to the documentation were quite involved, in that a client application that runs the JWalk engine in the background now has to deal with caught exceptions in a different way, by unpacking the notifications dispatched to the third-party listener. I still have to update the JWalk blog for 2011. I also need some new screen-shots of the latest version of the tool, which now also runs under Windows 7.

Thursday, 9 June 2011

Succeeded in installing the latest updated release of the JWalk 1.1 tool suite on the website, together with regenerated Javadoc documentation for the Look Inside Page, which required writing new package documentation to supplement the automatically generated API descriptions. Most effort went into describing the new facility to run the core engine as a background worker process. After that, I updated the the Download Page and the the User Guide Page, together with the home page, to announce the availability of the new version.

Monday, 23 May 2011

Trials of a frustrated developer: Success at last! I have finally found a way of restoring the animated "Jaywalking Jimbo" icon after cleaning up from an OutOfMemoryError that occurs inside the Image Animator thread. Java experts will recall that this thread is not created directly by the programmer, since it is part of some process that is started deep inside Sun Microsystem's original implementation of the AWT toolkit and Swing set. The problem was this: you cannot restart a broken thread, but only launch a new thread; and you cannot reload the animated GIF using getImage(), since this simply fetches the cached version of the icon, whose thread has already broken. The trick is to use createImage() directly instead, which bypasses the cacheing.

This was extremely hard to find out. There were four or five separate postings on developer websites like Stack Overflow that had reported similar problems trying to restart broken animations over which the programmer had no direct control, and no-one had suggested simply reloading the whole image from scratch using this more primitive method. It works; because it bypasses cacheing and restarts a new Image Animator thread, when the loader detects that the GIF is animated. It may also start a new System Timer thread that schedules when the different frames of the GIF should be displayed.

Saturday, 21 May 2011

Trials of a frustrated developer: I've been running the JWalk Tester prototype all day, deliberately trying to make it run out of memory, to see how many threads actually exist, and can break, when running the multi-threaded version of the tool. I've now worked out that we have at least four different threads in which an OutOfMemoryError may arbitrarily occur, and we need to recover from each of them. The threads are: the AWT Event Queue (the main thread for the GUI), the Image Animator thread, a secret thread created by the AWT toolkit to animate a GIF, the System Timer thread and the spawned JWalker Thread running as a worker in the background.

To recapitulate for those following this glorious saga, we have turned the JWalk Tester tool into a multi-threaded application, so that the user interface does not lock up while the test engine is running. One of the hidden benefits of the old single-threaded execution model was that we always knew where in the test engine software the OutOfMemoryError condition was reached, so we could always recover and clean up at that point. However, in the multi-threaded version of the tool, we have absolutely no control over the thread in which the OutOfMemoryError condition might first be reached. If this is in the JWalker Thread, we can recover as before. If this is in any other thread, we need a new way of recovering.

The current solution is to grab a chunk of junk memory just before the test engine is launched. Then, if memory is exhausted in any of the user interface threads, we release that chunk to give the tool a chance of continuing, notify the test engine that memory is now exhausted and immediately yield to the JWalker thread, which allows this thread to terminate early and clean up. However, whereas the AWT event queue is robust and always restarts, I have found that the image animator remains broken after such a recovery. For the time being, I have put in a substitute static icon to display, when the Image Animator thread has broken. No-one on the networks seems to know how to access this proprietary part of Sun's implementation of the AWT.

Tuesday, 18 May 2011

Trials of a frustrated developer: I have now completed the changes to the exception handling mechanism for the new multi-threaded version of the JWalk Tester tool. This required some updates to the way in which Notification objects were used by the specific question listener for the tool, called QuestionMaster. Basically, a separately-run worker thread cannot throw caught exceptions. This is because the run() method of the interface Runnable does not declare any thrown exceptions. So, the new JWalker, which implements Runnable, must also handle its own raised exceptions, when executing under run() (it may still throw exceptions from execute() in a single-threaded application). And yet ... we still want to notify the user of their mistake, when they have done something unconscionably silly with the tool, but how?

Well, the answer was to redesign the Notification object to encapsulate tool-raised exceptions, as well as other kinds of messages sent as notifications to the user. This means that QuestionMaster listens more carefully to the Urgency of each received notification, and any tagged as Errors are handled by calling the same method in the user interface that processes caught exceptions. I have now streamlined this, so that the same handlers get to see these exceptions, whether they were thrown in the old single-threaded mode, or whether they were encapsulated inside notifications and passed as events to the question listener. Extending Notification was the right way to do this, since the tool already reports information, warning and error messages. We have just standardised on what we mean by a warning and an error!

Saturday, 14 May 2011

Trials of a frustrated developer: The prototype multi-threaded version of the JWalkTester engine is causing me some headaches. Although I've now made the test engine run as a worker thread in the background (by making the JWalker implement the Runnable interface), and I can even adjust the priority of this worker thread so that the user interface dominates in the time-share, I still have two outstanding thorny issues to tackle.

The first is that I now have to process every exception raised during the execution of the test engine locally, inside the worker thread. This is because you cannot throw exceptions from the worker thread back to the user interface thread. For the moment, I'll just print out such faults on standard output; but in the long run, I want to find some way to communicate these exceptions back to the user interface. We already have a notification mechanism for warning and error dialogs, which could be adapted for this purpose. Other kinds of exception, to do with bad settings, are still thrown before the main execution loop, so it is only these three: PermissionException, GeneratorException and ExecutionException that need to be trapped specially.

The second problem is how to handle OutOfMemoryError exceptions. In the old single-threaded model, it was always clear where in the test engine's code this out of memory condition would arise; and we could recover by cancelling the current test cycle and reporting all the saved results so far. However, in the new multi-threaded model, we have no control over the thread in which such out of memory conditions arise. This means that the tool can recover (currently) only if memory is exceeded in the worker thread. I need to experiment with ways of recovering in the AWT Event Queue main thread, without requiring the Java runtime to grab more memory, while processing the out of memory condition!

Tuesday, 3 May 2011

How hard can it be to make the JWalk animated icon "Jaywalking Jimbo" do his running while the tool is executing, and stand still while the tool is idle?. You would think it is simply a matter of programming. Yes, except that this is rather like excavating two new sub-basement levels under a hundred-storey skyscraper. This tale is quite fascinating for agile programmers, who think everything is just a matter of fixing the code, so read on...

I had always wanted the animation to indicate the working state of the tool. However, in the original JWalk 1.0 release, whenever the test engine was started, this blocked all activity in the user interface until the engine had finished work. This was OK as a mode of working, but it meant that the animated "Jaywalking Jimbo" icon froze up, while the test engine was executing. It provided some kind of visual feedback - you had to wait for Jimbo to start walking again, to know that testing was over. But why not arrange things the other way around? The metaphor of walking through the software would be better represented by having the walking animation of Jimbo activate during a test run. Also, at least one of my Beta users reported that having the running man constantly in view on his computer, even when he was not running the tool, was visually irritating. This kind of feedback is extremely useful - and I'm very grateful. So, now I'm trying to do something about it.

So, how hard can it be? Well, the main trick was to multi-thread the application, so that the main test engine runs as a worker thread in the background, freeing up the user interface. I first looked into the new possibilities in Java 1.6 with the recently-introduced SwingWorker class; but later I found that just making the JWalker implement Runnable with its own run() method was much simpler, and allows third-party applications to launch the JWalker inside a new thread. This way, the animated icon doesn't freeze up while the test engine is running. That's at least a partial success.

The next problem is how to make sure that the image displayed correctly reflects the working state of the tool. The main GUI thread can switch the button image from a static image of Jimbo standing, to the animaged GIF of him walking. However, this thread doesn't know when to switch back to the static image. When you have concurrent threads, there's also the possibility that the user could initiate a second test series, while the first is already executing! So, I killed two birds with one stone: the switching between static and animated Jimbo was allied to the active and disabled states of the button. Starting a test cycle disables the test button, and so switches to the walking icon. However, how to switch back again to the static icon once testing was finished? The best solution seemed to be something based on the event-forwarding mechanism already in place. Eventually, I created a special silent notification event dispatched from the test engine back to the GUI, to signal that testing has finished. The GUI responds to this by re-enabling the test-button and so switching back to the static icon.

Saturday, 30 April 2011

I found a lot of relevant stuff on the web, including the discovery that the Java 1.6 Swing set now includes a new SwingWorker class, supplied expressly for the purpose of providing a worker thread that is decoupled from the GUI. There seems to be quite a lot involved in the contract of this class. I may need to go back and re-read all my notes on Java multi-threading just to see if this is the right way forward. I seem to be spending all my Easter vacation working ... not a good idea. However, I've had to do so much other stuff this year (including a viva for a PhD) that there's not been so much time for JWalking as usual.

BTW, the Royal Wedding was huge fun, even for people slightly jaded about royalty. Catherine managed not to look like a meringue and William looked the part in his guards' uniform. Wills is an OK bloke, flying search-and-rescue helicopters to save the likes of you and me. (His father Charles also used to fly Whiskey-Alpha-something in the Royal Navy; and is uncle Andrew actually served on helos in the Falkands War). Best man Prince Harry looked like a Manga cartoon character with sticky-out exploding orange hair. Nice to see someone not taking it too seriously. But we all wish the newly-weds well; and it looks as if they stand a better chance than some manufactured marriages of state.

Friday, 29 April 2011

It's Royal Wedding day today, when Prince William marries Catherine Middleton. (If you're well acquainted, or just any old newspaper hack, you get to call them Wills and Kate, informally). We get an extra statutory day's holiday. However, I'm busy working on ideas for the next release of JWalk. I've been looking into techniques similar to those in Java that allow you to pop up a progress bar recording how far a given process has completed, to see whether this could be adapted to indicate the progress of testing with the JWalk Tester tool. I'm thinking of decoupling the user interface from the test engine, and using the walking icon of "Jaywalking Jimbo" as an indicator that testing is in progress. At the moment, he freezes up as soon as you start testing.

It's 11.30, so I'm off for a break to watch all the pomp and circumstance on TV. They have promised that there will also be a flypast by the Battle of Britain commemorative flight, which I last saw (in real life) in 2008 at the Waddington Air Show in Lincolnshire.

Back to earlier entries...