Deferred transaction updates idea.

[ConvertFromOldNGs]

by Carl Ranson >> Mon, 22 Mar 2004 5:59:55 GMT

Ok, I've just been dealing with a situation which has prompted me to moot this idea.

Imagine you had a company and employees class. There is a one to many relationship between company and employees.
Employee has an integer yearly sales figure. Company has a integer attribute on it called topEmployeeSales that is a copy of the yearly sales figure for the employee with the largest yearly sales. Imagine this is done for performance reasons (many employees) and that topEmployeeSales is used as a dictionary key.

The question is, how do you ensure the topEmployeeSales is updated correctly in ALL cases AND that you don't do excessive overcalculation of it.

You can update it whenever the yearly sales is set for any employee and when an employee is added or removed, but that involves calculating it each time. This is fine if you change them one at a time, but is silly if you are doing batch updates of employees.

You can have some background process that causes it to be periodically updated but that means it won't be guaranteed correct if read between the employee update and the company update.

An elegant solution would be a mechanism in Jade whereby an object can be flagged as dirty and a special "transactional" method can be called just before the commit transaction. I would envision something similar to a causeEvent call except that it gets called before the transaction is complete.

Then you could calculate the derived fields once and only once per transaction.

it code it might look something like.

employee.setYearlySales(sales : Integer);begin

yearlySales := sales;
myCompany.causeTransactionalEvent(Event_Update_Top_Sales);
end;

company.transactionalEvent(eventType: Integer);begin

if eventType = Event_Update_Top_Sales then
// calculate and store top yearly sales...
endif;
end;

and then you could have
begin
Transaction
foreach employee in company.allEmployees do
employee.setYearlySales(employee.calculateYearlySales);
endforeach;
commitTransaction // causes single company.transactionalEvent to be called.

Note: In this example above it would be easy enough to have a manual call to company.recalcTopEmployeeSales or something along those lines. The problem with this approach is there is NO COUPLING between changing the source data and changing the dependant data. This makes it very easy to accidentally update the source data and not the derived, not to mention all the extra work if you happen to add a second derived field.
I want the fact that there is a data dependencies between employee.yearlySales and company.topYearlySales to be encapsulated in the company and employee classes.

What do ya think?
CR

[ConvertFromOldNGs]

by allistar >> Mon, 22 Mar 2004 7:56:48 GMT

If you have a transaction agent class (or similar) that maintains the transaction handling (beginning, aborting and committing) then you could call your "company.transactionalEvent" in your framework. When the value on an employee is updated you could have a boolean in the transaction agent class called something like "needToRecalculateYearlySales" which is set to "true". In the framework that does the committing you then need to test the value of that Boolean, and if true do the actual update. This way the calculation of the company value will happen only when it needs to (and only once).

Essentially with the transaction agent you are wrapping thebegin /abort/commit transaction calls with your own methods, which allows
you to do call other code whenever a transaction is committed.

I may be misunderstanding the problem, but wouldn't a better approach be to always calculate the Employee.yearlySales figure online (whenever a sales transaction is added/changed/removed from that employee). Have a dictionary of Employees keyed by the topYearlySales decimal property. Now the first object in that collection (make sure it is keyed descending) will always be the Employee with the highest sales. Therfore to key the sales value for the top employee all you need to do is:

value := allEmployeesByYearlySales.first().yearlySales;

Using this approach there is no need to do any calculation on the commit transaction as the updating of the individual yearly sales value for the Employee could reshuffle its position in the collection of Employees (because the inverse maintenance would occur when the key property is changed).

Regards,
Allistar.

--
------------------------------------------------------------------
Allistar Melville
Software Developer, Analyst allistar@silvermoon.co.nz
Auckland, NEW ZEALAND

Silvermoon Software
Specialising in JADE development and consulting
Visit us at: http://www.silvermoon.co.nz
*NEW* Simple web access to Jade at: www.silvermoon.co.nz/jhp.html ------------------------------------------------------------------

[ConvertFromOldNGs]

by carlranson >> Mon, 22 Mar 2004 8:39:33 GMT

If you have a transaction agent class (or similar) that maintains the transaction handling (beginning, aborting and committing) then you could call your "company.transactionalEvent" in your framework. When the value on an employee is updated you could have a boolean in the transaction agent class called something like "needToRecalculateYearlySales" which is set to "true". In the framework that does the committing you then need to test the value of that Boolean, and if true do the actual update. This way the calculation of the company value will happen only when it needs to (and only once).

Essentially with the transaction agent you are wrapping thebegin /abort/commit transaction calls with your own methods, which allows
you to do call other code whenever a transaction is committed.

Yes, in a well designed system the transaction manager could take care of this, but i have legacy system that doesnt quite qualfy.

Anyone care to provide an example pattern of a transaction manager in Jade?

I may be misunderstanding the problem, but wouldn't a better approach be to always calculate the Employee.yearlySales figure online (whenever a sales transaction is added/changed/removed from that employee). Have a dictionary of Employees keyed by the topYearlySales decimal property. Now the first object in that collection (make sure it is keyed descending) will always be the Employee with the highest sales. Therfore to key the sales value for the top employee all you need to do is:

value := allEmployeesByYearlySales.first().yearlySales;

Using this approach there is no need to do any calculation on the commit transaction as the updating of the individual yearly sales value for the Employee could reshuffle its position in the collection of Employees (because the inverse maintenance would occur when the key property is changed).

This was just an example. The system im dealing with has calculations that are a lot more complicated than just finding the maximum.

There is still a need to do the calculation in the problem I stated as Company.topEmployeeSales is itself used as a dictionary key.

[ConvertFromOldNGs]

by Trailing Drones >> Mon, 22 Mar 2004 21:29:09 GMT

[ConvertFromOldNGs]

by Trailing Drones >> Mon, 22 Mar 2004 21:32:39 GMT

First reply was in html - trying again in plain text!

Yes, in a well designed system the transaction manager could take care of this, but i have legacy system that doesnt quite qualfy.

ditto

In the system that I'm working on, most of the updating is done in-line rather than through agent classes. Recently, I've needed to add some costly updates where the trigger for the update might be a low-level persistent class update that could be called multiple times for multiple subordinate objects.

For example, allocating cash from multiple receipts to multiple invoices - I need to review the debtor's position at the end of the transaction. There are many places in the system that could do the cash allocation and I don't want to put similar code into each one. The ideal place for me to do the update is in CashAllocation.setMyInvoice - but that could be called many times.

My solution has been to add a collection of "postponed updates" to my session object. These get processed on the commit. I have managed this by replacing begin/commitTransaction with app.beginManagedTransaction and app.commitManagedTransaction. I keep a flag on app that tells me when I'm in a managed transaction state, and the constructor for the PostponedUpdate raises an exception if it's not inside a managed transaction.

So in my example I can add a line to CashAllocation.setMyInvoice to create the postponed update object - PostponedUpdate is subclassed into PostponedDebtorReview, with a reference to the debtor.

Some of the code follows...

<<from app.beginManagedTransaction>>
vars
ppu:PostponedUpdate;begin

if transactionMode <> TransactionMode_None then
app.raiseApplicationException(PostponedUpdateException,EXCEPTION_220_POSTPON ED_UPDATE,
"Transaction Mode is "&transactionMode.String&" at "&method.name,null);
endif;
foreach ppu in allPostponedUpdates do
ppu.handleHangover; // this would typically raise an exception
endforeach;
transactionMode:=TransactionMode_Managed;
end;

<<from app.commitManagedTransaction>>
vars
ppu:PostponedUpdate;begin

if transactionMode <> TransactionMode_Managed then
app.raiseApplicationException(PostponedUpdateException,EXCEPTION_220_POSTPON ED_UPDATE,
"Transaction Mode is "&transactionMode.String&" at "&method.name,null);
endif;
while true do
// keep selecting first in case new ones are added during the processing if allPostponedUpdates.isEmpty then
break;
endif;
pu:=allPostponedUpdates.first;
pu.processSelf; // will delete itself
endwhile;
transactionMode:=TransactionMode_None;
end;

<<from PostponedUpdate superclass constructor>>begin

if not app.isInManagedTransaction then
app.raiseApplicationException(PostponedUpdateException,EXCEPTION_220_POSTPON ED_UPDATE,
self.class.name & " outside managed transaction",null);
endif;
end;

<<from app.cnUserCleanUp (called in the CardSchema exception handlers on an abort)>>
allPostponedUpdates.purge;
transactionMode:=TransactionMode_None;