JADE Unit Testing Framework

[0.0]

I am wondering:

1) How many developers have taken up the unit testing framework feature in JADE?
2) How developers are structuring their tests and what is the rationale behind the structure chosen? (what schema(s) JadeTestCase subclasses are added to and where they exist in the schema hierarchy).

Cheers, 0.0

[murray]

1) one here - just started on them in the last few weeks, to try them out on my current project.

[Jason]

We've recently started to use the unit testing framework as part of a wider adoption of lean/agile principles.

So far, we've found it to be adequate for writing and running tests. The initial schema and class structure we've chosen is as follows:

Tests are written in a subschema of the schema being tested. This means you keep a clear separation between the code under test, and the test code. This is important as it allows you to subclass production classes in order to mock them (I recommend reading Roy Osherove's The Art of Unit Testing to find out more about mocking).

Within test schemas, we have the following class structure:

Code: Select all

JadeTestCase [schema_name]TestClasses [schema_name]IntegrationTests [schema_name]UnitTests [class_A]Tests [class_B]Tests [class_Z]Tests

The first class below JadeTestCase is there for common functionality that may evolve as tests are written and refactored.

We then have two abstract classes for unit tests and integration tests. Once again, I'd suggest you read The Art of Unit Testing to understand what a unit test is and what an integration test is. But briefly a unit test does not depend on anything other than the logic being tested. So a unit test should not call a web service, it should not access a file (including ini files), it should not depend on persistent data etc. Also, a unit test should test a class in isolation. Therefore, if a unit test ever fails, you know it's because the logic being tested is wrong (i.e. a change has introduced a bug). On the other hand, integration tests test multiple classes, and they may call web services, access files etc. Therefore, an integration test will be slower to run, and may fail for reasons other than defective code.

Naming test methods is probably the hardest part of your organisation strategy. This is due to Jade's 30 character name limit. We've found that 30 characters is not enough to be able to convey enough information about a test to make it meaningful to other developers. I'm going to refer The Art of Unit Testing again to give you an example - you really should read it . In it, Osherove recommends the following naming convention for test methods:

[methodName]_[stateUnderTest]_[expectedBehaviour]

Clearly, this is not feasible in Jade. We've therefore chosen this convention:
[scenarioUnderTest]_[expectedResult]

Where this exceeds the 30 character limit, we omit the [expectedResult]. This is not ideal as it makes it more difficult for other developers to understand what a test method is testing without having to read the code (which will be a large overhead when you have hundreds of tests).

To compensate for the difficulty of naming test methods clearly, we've started adopting a convention of writing our tests using the Given/When/Then style of tests. This means that our test methods look like this:

Code: Select all

exampleTestMethod() updating, unitTest; begin givenACalculator(); whenAdding(3, 4); thenResultShouldBe(7); end;

The advantage of this is that your tests are very easy to read. The details of how the test is implemented are neatly encapsulated in the methods. In this example, the Given/When/Then methods would like similar to this:

Code: Select all

givenACalculator() updating, protected; begin create self.myCalculator transient; end; whenAdding(firstNumber, secondNumber : Integer) updating, protected; begin self.actualResult := self.myCalculator.add(firstNumber, secondNumber); end; thenResultShouldBe(expectedResult : Integer) protected; begin assertEqualsMsg("Result should be " & expectedResult.String & " but is " & self.actualResult.String, expectedResult, self.actualResult); end;

You can then use a teardown method (one with the unitTestAfter method option) to delete transients created in your given methods.

Hope that helps you My final piece of advice would be (once again) to read The Art of Unit Testing. It is an easy read and a very good introduction to unit testing. The examples are in C# but are easy to follow even if you're not familiar with the language. Of course, the majority of concepts discussed apply to Jade as they would to any other language. There's also tonnes of blogs and articles on the net about unit testing and test driven development - just search on google and go from there!

[0.0]

Hi Jason.

Thank you very much for providing such a detailed answer. Lots of good advice in there.

A couple more quick questions in relation to running the tests, if you don't mind.

These questions apply to the scenario where you have a number of testing schemas to test different parts of a complex schema hierarchy...

1) In terms of running tests, do you run tests in batch mode?

If yes to Q1, do you use
2) multiple invocations to jadclient within a batch file? (where each invocation runs app=JadeUnitTestBatch)
3) mutiple batch files to manage your testing?
4) the xml configuration file to manage your testing?

Thanks a lot.

0.0

[Jason]

We're actually in the process of setting up automated running of unit tests. We're still trialling our setup, so I'll answer your questions based on what we're currently using.

We currently invoke jadclient per schema. This is because we've struggled to get the test results reported correctly when running multiple schemas in one invocation of jadclient. We're not sure why this is, but are planning to report our findings to Jade Support shortly. Doing it this way means we have one results file per schema.

We are using CruiseControl.NET to manage scehduling and running unit tests. CruiseControl is a continuous integration server. It's very flexible and ideally suited for running tests. One of the features that attracted us was the CCTray utility it comes with. Basically, this app runs in the task tray of each developer and indicates the current status of a build. If a build breaks (i.e. a test fails), then the icon changes from green to red and notification is displayed. It's much better than emails!

I've not got the XML config file with me at the moment, but it's pretty straightforward.

We've set up a test run per schema, e.g.

Code: Select all

<tests name=SchemaA > <schema name=SchemaA runAllTests=true /> </tests> <tests name=SchemaB > <schema name=SchemaB runAllTests=true /> </tests>

The syntax might not quite be right but hopefully that gives you the gist

One other thing you might want to do is implement the JadeTestListenerIF interface to output your test results. We've done this to output them in XML. Eventually we'd like to mimic the NUnit test result formats, so that CruiseControl can display them and track trends etc. This is still hypothetical at this stage.