JADE Discovery Kit: Erewhon NotificationSubscriber class and

[ConvertFromOldNGs]

by cnwep1 >> Mon, 26 Aug 2002 4:17:33 GMT

Hello,

For those of who are familiar with the Erewhon demo system and know a bit about Jade:

Why would your cache synchronisation class register for Object_Delete_Event notifications? I can understand why you'd want to register for Object_Update_Events, but why 'deletes'?

Cheers,

Eric in Dunedin (snow on the hills but remarkably mild)

[ConvertFromOldNGs]

by cdshearer >> Mon, 26 Aug 2002 6:01:19 GMT

Hello Eric

This caused me to ponder a little too... and I haven't fomally investigated this, but I think it's really to ensure early identification of Object Not Found when you try to access an object.

It an object is in a client's cache, and is subsequently deleted (by some other client, and thus, on the server) then clients with the object in their cache would still be able to access the object as if it existed. I guess this isn't generally desirable, hence the notification which would cause it to be removed from cache.

Does this sound plausible?

Craig (in Auckland, where there's never any snow on the hills)

[ConvertFromOldNGs]

by dcooper@jade.co.nz >> Mon, 26 Aug 2002 6:53:25 GMT

You're correct, Craig.

The notification itself won't cause an object not found exception, but it will mark the object buffer in cache (if it's there) as obsolete. This will force the next access to try to fetch the object again from the db server which will result in an object not found exception.

Without the notification, you could perform certain operations on the object from the cached edition without getting an exception. The notification is useful if you want your app to detect/respond to exceptions sooner rather than later, and if you want exceptions be raised consistently regardless of whether or not a deleted object is in cache.

Note though that there is always a latency (generally very small) between the event that causes a notification to be sent (in this case one process committing a delete) and another process receiving the notification. If this isn't acceptable then you need to consider using resynchObject or a lock, but beware of a shotgun approach all over the system as overuse of these can effectively defeat cache.

Dean.

[ConvertFromOldNGs]

by cnwep1 >> Mon, 26 Aug 2002 21:43:35 GMT

Hello,

<SNIP>

Without the notification, you could perform certain operations on the object from the cached edition without getting an exception. The notification is useful if you want your app to detect/respond to exceptions sooner rather than later, and if you want exceptions be raised consistently regardless of whether or not a deleted object is in cache.

If you have a group of classes synchronised using a NotificationSubscriber thing, and others that are not, then won't you be introducing inconsistency? For the former you'd be more likely to get ObjectNotFound, for the latter less likely. (Unless you go lock/resynchObject all over the place.) So if you have a mixture of approaches for cache synchronisation would it be more consistent not to register for Object_Delete_Events?

Not a big issue, except unfortunately in one system a developer had registered for Updates and Deletes. The updates were okay because there were relatively few of those, but occasionally the system would do thousands of deletes in a short space of time which impacted performance.


Hmmm....

Eric in Dunedin (snow patches still on the hills but plenty of affordable quality homes and no traffic snarl ups)

[ConvertFromOldNGs]

by cnwdad1 >> Mon, 26 Aug 2002 22:33:21 GMT

Generally you would only register for deleted objects where you don't expect lots of deletes. If you do expect lots of deletes, then you would probably want to implement the handling of this differently, such as a specific exception handler perhaps.

It's a case of horses for courses, in other words, the implementation should suit the situation. There's no one silver bullet that can cater to all situations.

regards,
darrell.

[ConvertFromOldNGs]

by dcooper@jade.co.nz >> Tue, 27 Aug 2002 1:09:14 GMT

It seems we're talking about two issues here.

Volatility aside, whether or not you choose to register for Object_Delete_Events in the first place depends on the application. If you're happy to access cached state on occasion, or deal with invalid objects entirely with exception handlers, or make specific (ie: limited, not shotgun) use of resynchObject or locks, then you're correct that Object_Delete_Events aren't necessary.

However, the problem you're describing sounds like it's more to do with the use of system *class* notifications where instances of the class can be affected by high-volatility operations. In general, class notifications are not recommended in these situations. The issue isn't really the Object_Delete_Event as you would run into the same problem with Object_Update_Event if the batch process happened to update thousands of objects in a short space of time.

Often a good technique to address this is to switch to user notifications as you can be more specific with targets and events, and you have more control over when/if the notifications are dispatched. For example, you could implement a transient NotificationManager (say) object per process that manages a list of notifications to be sent at the end of each transaction. This list is cleared when a transaction starts (helps if you have your transactions centralised somehow) and model operations register notifications with the NotificationManager (ie: causeEvents aren't done directly out of the model - this means the NotificationManager can also filter duplicate events). Just before each transaction commits, it asks the NotificationManager to dispatch the events. For batch operations, you can then ask the NotificationManager to not dispatch any events (or ignore the registration calls from the model entirely).

User notifications can be used to implement a cache synchronisation strategy, you just have to do the resynchObject yourself (passing an invalid object reference to resynchObject doesn't raise an exception, so this approa ch will still work for user-defined delete events). Ensure you use resynchObject, *not* resynch. resynchObject is preferable to resynch in general, and resynch on an invalid object that isn't in cache *will* raise an exception.

Dean.

[ConvertFromOldNGs]

by allistar >> Tue, 27 Aug 2002 3:08:12 GMT

Dean,
We are in the process of removing our class notifications (because
they are hindering the scalability of our product). We have ensured
the integrity of updating objects by locking an object before any updating method is called on it (this will ensure that we never use
out of date data when updating objects, especially when the update if
of the type: object.decimalProperty := object.decimalProperty +
number).

The trouble we are having is in coming up with a strategy of always displaying the most up to date information. It's a fine balance
between using cache and not using cache, and we want to be somewhere
in the middle. Using resynchObject and beginLock/endLock during our loading methods is basically ignoring the cache, so we have come up
with three possible solutions:

A) We don't want to resynch the same object more than once in a single read operation, so we make our own "resynchObject" method that
remembers the object we are resynching so we only resynch it once per read (the collection of remembered objects gets flushed at the start
of a read).

B) We know what types of objects we will be reading, so we do a class.resynchInstances on those classes before the read happens. This will ensure that we never read old objects.

C) Don't worry about it and allow the user to click on a "flush cache" button. (Although I don't like this solution).

Both A and B have their pros and cons. One thing to keep in mind is
that our system is quite large (40,000 methods, 800,000 lines of
code).

(BTW, this is what we will be discussing when you come up next week).

Regards,
Allistar.

[ConvertFromOldNGs]

by CarlRanson >> Tue, 27 Aug 2002 3:29:46 GMT

C) Don't worry about it and allow the user to click on a "flush cache" button. (Although I don't like this solution).

I don't know....it would be pretty cool if you hooked it up to a toilet flushing noise.

And you could market an optional extra chain pull peripheral.

CR

[ConvertFromOldNGs]

by cnwep1 >> Tue, 27 Aug 2002 4:06:04 GMT

I don't know....it would be pretty cool if you hooked it up to a toilet flushing noise.

Ahhh Reminds me of the good old days of a small DOS program we occasionally called from unsuspecting users' autoexec.bat files. Used to come up with messages like:

What has been detected in the A: drive
Please wait while the system spin dries drive A:

followed by a crescendo of whirring sounds before winding back down again and a message like:

Drive A: has now been dried out. You may now continue.



And some people used to believe it!

I wonder if there's a Windoze version of it....


Ignore me,

Eric in Dunedin (so drizzly and wet I must have water on the brain)