Transient cleanup and DictIterator

[ConvertFromOldNGs]

by cdshearer >> Thu, 7 Nov 2002 19:14:03 GMT

The reason I went for the known objects thing, is that the list of objects is pretty much known for a particular version of JADE. Sure, in the future, the list might expand or contract, but at least for now, if I filter out the known objects, I don't have to wade through the list each time to check that only the known ones are there - in effect, I've shielded myself from a false positive result.

Craig

[ConvertFromOldNGs]

by cnwrwm1 >> Thu, 7 Nov 2002 23:30:05 GMT

If you have JSM Start (CardSchema) there is a shut down option you can set in the [CardLog] section of your ini file : CheckTransientsOnShutDown=true. Basically when set this checks all non-system classes in your schema hierarchy plus certain RootSchema classes for undeleted transients at shut down time, and logs the count of transients by class in the cn_log.log file.

The current implementation doesn't encompass checks for EVERY class of object that might exist, nor does it help identify the source of the leak, but it's an easy way to get started checking for leaks.

A word of warning : the operations involved in these checks are expensive, and will substantially lengthen the time taken to close down an application. This option should normally be switched off for production systems.

Richard Mitchell
Jade Software Corp
Systems Management Development Centre

[ConvertFromOldNGs]

by cdshearer >> Fri, 8 Nov 2002 0:14:00 GMT

Just took a look at the code in CardSchema::cnCheckForTransients, and interestingly enough, it leaks a transient too!

The following line creates a collection instance which is never cleaned up:

classcoll := currentSchema.getAllClasses(false); // excludes system classes

Additionally, it checks for instances of certain RootSchema classes, but doesn't check for subclasses of them. So, it checks for undeleted transient instances of Iterator, but it wouldn't find any DictIterator instances, because it's a subclass of Iterator.

Perhaps this code needs some review Richard!

Craig

[ConvertFromOldNGs]

by JensRichnow >> Fri, 8 Nov 2002 7:31:27 GMT

Good point, although I don't have and hence don't use the card schema.

One point which concernce be re transient cleanup is the emphasis on the clean-up at shut-down time of the application. I think that managing transients throughout the entire life-time of the application is equally if not more important. Apps are running typically weeks if not month during which time undeleted redundant transients accumulate, overflow the transient buffer, need r/w to disk, etc. If you run multiple Jade applications on one server, as typically is done, you multiple the story...

Any thoughts on this?

[ConvertFromOldNGs]

by cdshearer >> Fri, 8 Nov 2002 8:18:16 GMT

Well yeah, I agree that it's important that you don't leak transients during the running of the app, but shutting down the app gives a convenient point at which to check for undeleted transients - any that are left over must have been created during the run of the app and weren't properly deleted.

The method is an "ambulance at the bottom of the cliff" sort of thing. Hopefully you'd get it all sorted out before the app went into production.

Having said that, it wouldn't be difficult to arrange for the method to be called at strategic points in the app apart from the fact that it's a very expensive method to run.

Actually I don't see why it should be so expensive to run. Presumably it's expensive because for it to work, it has to search for transient instances of a class and all its subclasses. So, if you start at Object, then for the average schema, there's going to be hundreds or thousands of iterations of the code that checks for transient instances of a class. I'm just guessing here, but surely all transients are held in a common pool in memory. So the code would have to iterate through all the transients and filter them by their class. They could provide an alternate implementation that just returned ALL transients, and surely this would be much more efficient to execute, meaning that it could be executed reasonably frequently during the running of the app. (Of course, long running processes should be in their own thread, which in JADE means a separate app - but then that app wouldn't have access to the transients it was trying to detect! Doh!)

Craig

[ConvertFromOldNGs]

by JensRichnow >> Sat, 9 Nov 2002 9:14:47 GMT

For database-centric applications, and by that I mean most business-type applications, which deal mostly with managing persistent instances the cleanup at the shutdown of the app might seem to be sufficient.

However, there are a number of applications (or tasks within an app), such as parsers, compilers, statistical analysis and various modelling techniques, which almost entirely rely on transient objects. This is obviously for reasons of speed due to transaction state overheads, etc. Depending on the task (see above) the number of transient objects easly run in the hundreds of thousands. If you don't want to choke your application you really look after your transients at strategic points, as you say.

For these types of apps and/or tasks I'm more worried about the run-time state of the app. Don't get me wrong, we have to cater for all cases, including the shut-down clean-up. And for these types of apps/tasks an underlying garbage collection would be desirable.