The JWalk Weblog, 2012

This is the JWalk blog for 2012! It recounts mostly a steady take-up in industry and academia of the JWalk 1.1 tool suite, the latest version of the JWalk lazy systematic unit testing toolkit. This includes two finished tools: JWalkTester is the flagship tool, a GUI-based Java class unit tester, which now operates in multi-threaded mode to give a more satisfying user experience. JWalkUtility is the original command-line utility, which executes in a standard console window, interacting with the user on standard input and printing results to standard output. The download bundle also includes the complete JWalk Toolkit for building your own JWalk testing applications around the core test engine. Built using run-anywhere Java, the tools configure themselves to the host's operating system, whether Windows (see the featured JWalkTester running on Windows 7), Mackintosh, Unix or Linux.

The tools support the revolutionary lazy systematic unit testing method, in which specifications are learned incrementally from prototype code; and from these, systematic test-sets are generated to validate the production code. The method is particularly suited to agile software development methods, such as eXtreme Programming, which have a strong emphasis on software testing, but no time to provide specifications. So, where do the unit tests come from? With the JWalk 1.1 tool suite, you can continually update the test oracle for each prototype Java class, by exercising the class inside the tools, and responding to queries about particular test outcomes. The tools use strong state-based and algebra-based test generation algorithms to ensure that the Java class is thoroughly tested, up to the current inferred specification.

Wednesday, 19 December 2012

Well, that's it for this year. I'm signing off now, to take a couple of weeks' vacation. We've just uploaded drafts for some EU deliverables for Broker@Cloud. I guess we will be back to discussing these in the New Year.

Thursday, 22 November 2012

The British Computer Society has visited us today, and re-accredited all our degree programmes. This is a mammoth exercise, in terms of making all information about our teaching programme accessible to the assessors. The BCS is the Royal Chartered Institute for Computing and Information Systems, so it is important to achieve this level of professional recognition. So we're happy it went well! I wonder how long it will be before practising software engineers must all have their CEng by law in the UK? It affects a couple of states in the USA at the moment. But we need to regulate this profession more seriously, with the waste due to badly managed and untested software out there.

Friday, 16 November 2012

I had some useful feedback today from Michael Keene from Avocent. Apparently, he is testing Java classes that link to other classes in different packages. He is experiencing some difficulty when using JWalk in combination with specific development environments. Apparently, these raise issues with cross-package importing, that is, when using development environements such as NetBeans or Eclipse that upload packages as compressed JAR files and manage the class cross-references internally on their build-paths, it is somehow not possible to make all of this information available to JWalk. I have not yet replicated this problem locally yet; but am waiting for a small canonical example.

In other respects, JWalk is quite happy with the package structure of Java. You need to run the tool in an environment that is above all of the packages that you wish to use, and then specify the fully-qualified name of the test class. This is then loaded by a specialised ClassLoader that supports repeated reloading of the test class, but not of other related classes, within the same JWalk runtime. If all of the related packages are

Monday, 5 November 2012

The kick-off meeting went quite well. I think that the co-ordinator SINTEF is very experienced and will help keep the work on track. There is a quite diverse collection of academic and commercial interests in our consortium; I hope it will be possible to get some kind of focus in the work we do. I also met our new Research Associate, Mariam Kiran. She has plenty of experience of EU projects which will help get us started quickly. We have to create two deliverables by the end of February 2013, which is a pretty short timescale. At the moment, this is the state-of-the-art in Cloud brokerage, and an as-is assessment of two pilot platforms provided by SingularLogic and CAS Software AG.

Wednesday, 31 October 2012

I'm off to Oslo today to attend the kick-off meeting of our EU Project, Broker@Cloud. It will be great to meet with Iraklis, Dimitrios and Kostas again; and also to meet the other members of the consortium, whom I don't yet know. The project officially kicks off on 1 November, so we will have a two-day meeting with introductions and planning for the work to be done. Hopefully, we will have some brainstorming too.

Wednesday, 24 October 2012

I got a request for JWalk from a company called Ruckus Wireless, which sounded rather fun, so I looked them up. They're a mobile technology outfit with a headquarters in Sunnyvale, California. They have other bases around the world. So it was good to have an entry into the mobile telephony market. It was 1995 when I was last on the West Coast of the USA, helping out on the OOPSLA programme committee, with Mary Loomis in charge.

Friday, 19 October 2012

I got a number of requests for JWalk licenses from students at UFSCAR, the Universidade Federal de São Carlos, Brazil. The requests came via private gmail accounts. Unfortunately, my licensing software insists on certain properties of beta-users, one of which is that they belong to an accredited university or commercial company. So you have to apply with a company or university email address, rather than with a gmail, hotmail or yahoo account. Sorry about this, but we have to ensure serious users, who will give us solid feedback after usage of the tool. That reminds me, a few more of you out there still owe me a report!

Wednesday, 17 October 2012

I was contacted by a representative of Avocent, based in Fort Lauderdale in Florida, who is looking into better testing practices for their software. I'm happy to offer JWalk to Avocent for a trial run. Avocent is a subsidiary company of Emerson Corporation.

Thursday, 11 October 2012

JWalk hits Wall Street! Today, I got a request from an analyst at CPSI, 6600 Wall Street, who would like to evaluate the tool. Maybe this will help plug some gaps in the financial sector, who knows?

Monday, 8 October 2012

I was contacted today by a developer at Aspect Software, Chelmsford, Massachusetts. She is investigating the possible automation of their Java application and peripheral Web Services that keep the application in synch with some other business applications in-house. She would like to try JWalk to test this package. I think Aspect Software is a company mostly involved in CRM applications.

Sunday, 7 October 2012

I don't normally write at the weekends, but just had to report that SpaceX have launched another ISS resupply mission today. The launch could be watched in real-time on the web. This is the second actual resupply mission with the Falcon 9 launcher and Dragon capsule, but the first "official" mission, since the previous one was technically the second of two feasibility tests. This mission will bring back some biological samples from the astronauts in the ISS for examining on Earth.

What was particularly satisfying about this launch was that one of the engines on the Falcon 9 failed just before max-Q, but the launch continued successfully on the remaining eight engines. Apparently the Falcon 9 can make orbit even in a two-engine failure mode. That's robust engineering, folks!

Tuesday, 2 October 2012

I had a request for JWalk from a PhD student at the University of Coimbra, Portugal today. She will try out JWalk as one of several testing tools in a project to determine what kinds of quality attributes are tested in different testing approaches. Well, for those interested, JWalk falls into the conformance-testing class of tools, where you are trying to determine that the software performs completely according to some kind of specification. The only difference is that, with JWalk, the tool learns the intended specification from you, the tester, incrementally. If you alter the code, the specification must also change, implicitly. But everything you tell the tool will be used again to verify the code you are testing.

Tuesday, 25 September 2012

I had a request for JWalk from a student intern at Capgemini in München. Part of our discussion involved whether or not you could export the tests to an external file. Initially I misunderstood the request. I said that JWalk doesn't export tests in the manner of JUnit, because this runs counter to JWalk's philosophy of continuously regenerating tests, whenever you change the compiled code for the tested object. However, it might be possible to do this, for the sake of some who want to use JWalk as a test generation engine, rather than a test generation and execution engine. This would be a new project.

Eventually, I realised that my correspondent was actually only asking whether the test results from JWalk's test engine could be exported to a file. Naturally, they can. If you use the command-line version of the tool, all results appear on standard output, and you can pipe these somewhere else. However, if you want to interact with the oracle generator, and not have this pollute the test results, you can provide your own components to implement the QuestionListener and ReportListener interfaces, and have these send some text to a pop-up dialog, and other text to a formatted output file. This is all intentional in the way the main JWalk engine was designed separately from the standard tool's user interface.

Monday, 27 August 2012

I had two more requests for JWalk licenses from graduate students at North Carolina State, Raleigh, USA. I seem to be supporting a whole class of students on this course - see the earlier entry for 20 January this year.

Monday, 9 July 2012

Great news! We have now heard that the EU will definitely be funding our project: Broker@Cloud: Enabling continuous quality assurance and optimization in future enterprise Cloud service brokers. We know that project has been allocated €3.388M altogether, across the seven partners, and that Sheffield will receive a share of €404K, which will allow me to appoint a Research Associate. Our partners at SEERC are happy to be supported to the level of €425K, which will keep two of my PhD students in gainful employment there!

Wednesday, 4 July 2012

Today I had a request for a tool to perform equivalence-partition testing! This was from an academic at UPM in Malaysia who had read about an experimental student project with Peter Lamb to build this facility on to JWalk. While Peter did a great job, the result wasn't as habitable as current JWalk, which looks at state-based testing for some spread of inputs, rather than exhaustive input category-based testing. Then, we did not have the technology to perform the guided exploration of the input space. But maybe we do now. This could form the basis for the next JWalk research project.

Friday, 25 May 2012

Another academic request for JWalk came in today, from an undergraduate at the University of Minnesota, Duluth. He is investigating the usefulness of automated systematic testing, versus random testing. So I was happy to oblige. We have done some work in this area already, if you look at the Smeets and Simons research report on the JWalk Publications Page. It will be interesting to see if this study comes up with similar conclusions.

Tuesday, 22 May 2012

Could not resist this, because it is so exciting. Today saw the first launch of the SpaceX Falcon 9 launcher and Dragon capsule to rendezvous with the International Space Station. This will be the first commercial resupply mission, with re-entry capability. Since the demise of the Space Shuttle, only the Russian Soyuz has this capability. The European Automated Transfer Vehicle (ATV) and Russian Progress capsulse can deliver supplies, but burn up on re-entry. Now the Dragon will be able to dock, unload supplies and bring other stuff back to Earth. A later version will be crew-capable as well.

What is so good about SpaceX is their attitude to design: keeping it simple! They've learned a lot from NASA - both what to do, and what not to do! Note that this is the second launch attempt - they tried to go up on Saturday, 19 May but aborted a few seconds after ignition and safely shut down the booster. Then, they identified the fault within hours and reloaded for the successful launch today. This is properly robust engineering folks!

Monday, 14 May 2012

Arrived to find another license request from a software engineering student from Mäardalens Högskola, Sweden. He's doing a master's thesis on unit testing, and would like to include JWalk in his analysis. Fine by me!

Wednesday, 9 May 2012

I received a request from Insys Group, New Jersey today for a trial license for JWalk. Apparently, they are working on a Java product for an unnamed client, who insisted that they use JWalk to test the product before it was delivered! How about that? Now the end users are insisting on it! Of course, I gave them a trial license. We're still negotiating with epiGenesys about marketing arrangements for the tool. Mostly it's a matter of getting the attention of their MD.

Thursday, 5 April 2012

Great news! We have now heard that the EU will most likely be funding our project: Broker@Cloud: Enabling continuous quality assurance and optimization in future enterprise Cloud service brokers. Thanks to Iraklis for phoning me up from SEERC in Thessaloniki to let me know. It seems that they have folks there whose ears are close to the news about what goes on in Brussels.

Monday, 19 March 2012

I'm back again, with a terrible lingering chest infection. I have to say that Sri Lanka was wonderful, with a most attractive hotel on the sea-front in Columbo, and a very good team helping us run the Engineering Recruitment Fair. I spoke continuously to interested applicants for four days, and lost my voice. The aircon in the hotel was a bit fierce, so I caught a chill, which only worsened when we left for Nepal. This country is so poor, it seems almost ridiculous for anyone to be able to afford to pay study fees for the UK. I met so many deserving kids who just needed a little financial assistance to get them started. One lad was so smart that he built his own version of a chip and bus (a bit like the Arduino product ArduPilot) that could drive a robot with a web-cam. He made it onto national Nepali TV. He was smarter than most of his college lecturers, but can't afford the UK at all.

Wednesday, 7 March 2012

I received a request for JWalk from the Royal Bank of Canada today. They use a number of Java tools and are looking for a suitable Java testing product. They think JWalk offers suitable features, so I gave them an evaluation license. This will be my last post for a couple of weeks, as I am about to head off on a recruitment trip to Sri Lanka and Nepal.

Tuesday, 6 March 2012

JWalk is now being used at MIT! I had a request from Benjamin Weissmann, who wishes to use JWalk in the MIT Undergraduate Course 6-3, in the Department of Electrical Engineering and Computer Science. My regards to Ben, and also to Daniel Jackson, with whom I have had several conversations about the Alloy analyzer.

Thursday, 23 February 2012

I received a request for JWalk from a student at the National University of Singapore, who is interested in automatic program assessment systems. Basically, he can take the JWalk tester and supply alternative QuestionListener objects to mark the behaviour of the tested software, according to whether it conforms to the learned oracle, or not. This is the same strategy that we built in the prototype JWalkMarker, which needs some tidying up before we release it.

Wednesday, 22 February 2012

I received a request for JWalk from a private IT consultancy that has worked in commercial IT with British Airways and the University of Edinburgh. They are interested in comparing automated testing with expert test development. I was happy to oblige with one of the candidates for this experiment!"

Monday, 6 February 2012

Another JWalk license request came from industry for JWalk today. This time, it was from the Irish company Digisoft, which makes TV equipment. Their R&D department would like to look at new ways of automating test coverage.

Friday, 20 January 2012

I had two more requests for JWalk licenses from graduate students at North Carolina State, Raleigh, USA. It's good to be able to support their masters projects in software engineering.

Tuesday, 17 January 2012

We submitted our proposal to the EU Framework Programme 7 today, to undertake research into the quality control of brokered Cloud services. The project is called Broker@Cloud: Enabling continuous quality assurance and optimization in future enterprise Cloud service brokers. The consortium includes SINTEF, Oslo; SEERC, Thessaloniki; ICCS, Athens; CAS Software AG, Karlsruhe; SingularLogic, Athens; SAP Research, Karlsruhe; and The University of Sheffield. The Sheffield contribution will be to develop state-based specifications of Cloud services that can be offered publicly and used for automated testing.

Monday, 16 January 2012

I've been going through the backlog of applications for a JWalk license today. Sorry for the wait! This is because of tomorrow's deadline for EU project proposals for Framework VII, call #8. I can now tell you that we have our first user from eBay Inc. This adds to the long list of adopters, which includes people who say they were recommended JWalk by Google - but I've not had a formal request for a license from Google? Maybe someone should own up!

Thursday, 12 January 2012

Over the last few months, new work on JWalk has been in abeyance while we have been putting together a proposal for a national UK centre for software testing. This will include search-based testing (Phil McMinn's specialty), and state-inference from code (Kirill Bogdanov's specialty), as well as my interests in model-based testing from state, algebra and functional models. Watch this space!

Friday, 6 January 2012

Happy New Year to all our JWalkers! I can hardly believe it is 2012 - this sounds like some kind of science fiction date, fondly anticipated during the 1960s. Well, here it is already; and the future isn't what it used to be. Consider all the software failures that happened last year! Maybe we should do more rigorous testing?

Back to earlier entries...