Transient leaks

[ConvertFromOldNGs]

by Robert >> Thu, 14 Jul 2005 3:35:39 GMT

To correctly implement garbage collection Jade would have to hold a reference count on objects.

Alan points out that it's perfectly valid for a transient object to live it's life with no references, e.g. an object that is responsible for it's own demise. How does AGC know not to clean up?

This implies that the developer must be able to determine whether auto-delete is applied, which in turn implies:
1. "transparent and retrofittable" can't be achieved
2. a language extension is required

N'est pas?

[ConvertFromOldNGs]

by Patwos >> Thu, 14 Jul 2005 5:45:54 GMT

"Alan points out that it's perfectly valid for a transient object to live it's life with no references, e.g. an object that is responsible for it's own demise. How does AGC know not to clean up?"

Actually, he points out that in his situation he chose not to maintain a reference to this transient object and this implies that in his finalise logic he's utilising "firstTransientInstance" to get back a reference to this transient singleton in order to delete it during application finalise.

Whether or not this is to be considered "valid" could easily become a vigourous/religious discussion very quickly. Suffice to say, from my perspective anyway, I would have kept a reference to this object, probably on app, even though I seldom needed to directly reference the object to avoid the use of firstTransientInstance in my production code.

To my mind, if we're going to have a language extension to go in partnership with AGC, the better option would be an extension for those specific cases where you want a non-referenced transient to be excluded from AGC. This allows a developer to make a conscious choice in those relatively few situations where they would not want AGC invoked, while still retaining the ability to just "switch the setting on" without any changes required to historical applications.

That's another $0.02 for me,
Pat.

[ConvertFromOldNGs]

by cnwjhp1 >> Thu, 14 Jul 2005 10:12:37 GMT

This implies that the developer must be able to determine whether auto-delete is applied, which in turn implies:
1. "transparent and retrofittable" can't be achieved
2. a language extension is required

I agree a language extension could be useful to explicitly exclude a transient from being deleted.

As for 'transparent and retrofittable', I would be surprised if any of the apps I have worked on for the last few years would have any trouble with turning on such an option. If there were any problems, they would quickly surface in testing and be easy to remedy.

[ConvertFromOldNGs]

by Patwos >> Thu, 14 Jul 2005 5:55:24 GMT

What would be the disadvantage of retaining a reference to that singleton connection instance, say off app, rather than leaving it as an unreferenced transient?

You could then avoid any use of firstTransientInstance in your production code to "find" this transient in order to delete it during your application's finalise processing.

Pat.

[ConvertFromOldNGs]

by cnwjhp1 >> Thu, 14 Jul 2005 10:04:09 GMT

Cleaning up locally created references automatically would not be appropriate. I have at times created a singleton Connection sub-class object (as a local variable so that relevant attributes can be set) whose function is to sit in the background listening and processing. I do not assign this as a reference to any other object - cleanup is performed in the finalize method by accessing the singleton again then deleting it. There are implications with creating it as shared transient (transaction state being one).
With the methodTransient the control / intent of the created object remains under the control of the developer, where I believe it should be. I think most other automated solutions are going to have cases where we wished there was an exception to the rule.

I agree the developer should be able to specify, but the standard syntax should be used for the normal case, and the unusual syntax for the unusual case. So we need a new term for 'keep-it-around-after-the-method-finishes', perhaps appTransient (ie, same lifetime as app).

When I create a transient that will stay around until the end of the app, I tend to put a reference to it on app or another transient control object.

[ConvertFromOldNGs]

by John Beaufoy >> Wed, 6 Jul 2005 11:12:53 GMT

a new 'methodTransient' command

I dont agree, the better option is to come up with a solution that is transparent to the code (like the Jade 6 cache-coherency). Introducing new statements like methodTransient will add complexity which we should be trying to remove. While Brendan has outlined these points already in detail, I'll reiterate the point that it would do nothing to help us with our existing systems.

transparent / AGC

While I would be very interested to see how the implementation would be put together, I wouldn't be dissapointed if it didn't appear in the product in the next year or two.



There are many points to consider, particularly:

- covering all the bases where a transient could actually get used again

- debugging, who deleted the transient, AGC or the app code

- when would the cleanup occur, would it invoke the delete methods
- at what levels could it be overriden.



My reluctance to jump on the AGC bandwagon is two fold, we might create as many issues or as much confusion as we hope to address, and the other unknown is what this would do to the performance of the interpretor/jom/cache mgmt etc.



Some interim measures then:



Jade provides warnings at compile time where it's obvious a transient isnt getting deleted (the iterator issue being a candidate here). Its not going to catch everything, but it's a good start and a good reminder to keep on top things. It could even suggest where to insert the delete into your code.


At runtime, Jade's diagnostic tools are enhanced to clearly expose transient leaks. To start with, this would be a running total of the number of transients for a process - a single count. If this number continues to grow while you are using the app, you know you have a problem. So you then want to see a breakdown of the totals by class exposing the problem ones. Then, you want to know which method created each of these transients, and method stack that goes with it.



Cheers,

John

NWI

[ConvertFromOldNGs]

by cnwjhp1 >> Fri, 8 Jul 2005 5:03:19 GMT

John,

Doesn't my suggestion (the post immediately prior to yours) address all these concerns? The only one I did not specify is whether the delete method is called. I'm sure someone can chime it with a reason it has to be one way or another.

Re your interim measures Jade does not currently have a warning message mechanism aside from the rather annoying msgBox, so something would have to done to address that. Regarding Jade diagnostics to track these transients, performance would be better if you simply delete them at the end of the method or delete of the parent object.

Cheers,
John P

[ConvertFromOldNGs]

by John Beaufoy >> Sun, 10 Jul 2005 12:56:07 GMT

Doesn't my suggestion (the post immediately prior to yours) address all these concerns? The only one I did not specify is whether the delete method is called. I'm sure someone can chime it with a reason it has to be one way or another.

In most cases yes. Not all transient leaks are orphans though, if you keep on adding transients to a collection that is hooked up to a 'long-term' transient an AGC would never remove these. What I mean is, I think a diagnostic tool would be useful whatever the long-term solution.

Re your interim measures Jade does not currently have a warning message mechanism aside from the rather annoying msgBox, so something would have to done to address that.

Lets hope they could do something better than this. Jade 6.1 has a new and more powerful control for the method source area, Im sure this would not be a problem.

Regarding Jade diagnostics to track these transients, performance would be better if you simply delete them at the end of the method or delete of the parent object.

Performance is not an issue for the diagnostics in the same way its not for using the profiler ie. you switch it off.

Regarding Jade diagnostics to track these transients, performance would be better if you simply delete them at the end of the method or delete of the parent object.

What do you mean by parent object? If you mean parent is terms of cascade delete then yes it will all happen pretty quick. The problem is these will often be peer-peer, so each time the AGC deletes an object, it has to also reevaluate objects referenced by that object to check again if they can now be deleted and so on. AGC may ease development, but having to perform these checks after every method surely cannot be as efficient as deleting manually.

Maybe while the application is in an idle state, jade/jom could rip through its cache working out all transients that can be accessed from your basic starting points eg. app, forms etc. All others can be simply blasted without any further need for 'inter-object' checking.

[ConvertFromOldNGs]

by cnwjhp1 >> Thu, 14 Jul 2005 9:54:10 GMT

What do you mean by parent object? If you mean parent is terms of cascade delete then yes it will all happen pretty quick. The problem is these will often be peer-peer, so each time the AGC deletes an object, it has to also reevaluate objects referenced by that object to check again if they can now
be deleted and so on. AGC may ease development, but having to perform these
checks after every method surely cannot be as efficient as deleting manually.

By 'parent object' I mean the method or class where the reference is defined. It would be super-quick. There is no need to check if it is referenced. If a local method variable was used to 'create transient', then it is deleted at the end of the epilog. There would be a variant of the create statement so you can exempt it from being deleted. Or you could 'create app.myRef', which would not be automatically deleted, as it is not a local method variable.

[ConvertFromOldNGs]

by John Beaufoy >> Fri, 15 Jul 2005 1:24:43 GMT

By 'parent object' I mean the method or class where the reference is defined. It would be super-quick. There is no need to check if it is referenced. If a local method variable was used to 'create transient', then
it is deleted at the end of the epilog. There would be a variant of the create statement so you can exempt it from being deleted. Or you could 'create app.myRef', which would not be automatically deleted, as it is not a
local method variable.

I don't believe it is as simple as monitoring whether it's a local variable or not when the create is called. Consider the following:



create user transient;

app.myUser := user;



user is a local variable, but Jade shouldn't delete it. Jade would have to keep a list of all transient objects created in a method as you suggest, but it would _also_ have to check which ones have been assigned to global/parameter transients, and remove these from the garbage list. Then consider calling the method a second time, the original transient is now orphaned and will miss the AGC.



Again I think a periodic check is the way todo it. Ini settings and application/class level would allow default settings to be overridden, and even a variant of the create method or self.dontGCMe() as I think you were suggesting may be useful. All these are provided to handle any issues AGC creates, or turn it off if performance is affected. Any transients that have notifications, timers or are listening on connections should not be deleted, leaving firstProcessTransient as the only stumbling block. Decision to amend this code or not use AGC, no-brainer really.