Relational Population Service (RPS) and database logs

[Michael.Duraj]

I have a client with a Jade database 370+ GB using SDS to replicate to a secondary system and then RPS to export data to SQL Server.

Its taken a while but SDS Manager on the second server now reports my databases are "synchronised"

BUT

My primary system D:\...\System\logs\current directory contains 180 GB of db000xxxxxxx.log files
My secondary system D:\...\System\journals\current contains 128 GB db000xxxxxxx.log files

These have appeared since I have setup RPS on the clients Live system, and the Primary System folder prior to implementation was about the 230+ GB mark.

When does Jade clean up these log files? I thought that when SDS had synchrnised that all log files would be marked as used and the log files would be deleted?
How do I remove these files? jdbutil ?

[Michael.Duraj]

So the Primary SDS node uses Archive Recovery hence the increasing number or .log files?

If in the Jade ini file I set;

JournalCloseAction=Remove

In the Primary?? and or Secondary?? Server will that resolve my issue
or do I have to

JournalArchiveDirectory=<insert location>
JournalCloseAction=Move

And have a batch file that periodically deletes the contents of JournalArchiveDirectory

Please note, my Secondary Server sole reason for existance is RPS (Pushing data into SQL Server)
It is not going to be used for recovery purposes.

[BeeJay]

When does Jade clean up these log files?

Having SDS, or RPS enabled means recovery journals will be retained as per setting EnableArchivalRecovery=true. However, for a production system in theory this shouldn't be an issue as you should already have EnableArchivalRecovery=true anyway for production systems, to maximise your recovery options and minimise data loss in the event of a catastrophic failure.

I thought that when SDS had synchrnised that all log files would be marked as used and the log files would be deleted?

Unfortunately this is not the case, although there is a way to get the secondary DBs to remove recovery journals when finished with for replay purposes as per the comment below.

How do I remove these files? jdbutil ?

You can get your secondary DB to remove recovery journals that are no longer required by specifying JournalCloseAction=Remove. Once a recovery journal is no longer required for replay purposes, the secondary will then remove the recovery journal. However, depending on your backup strategy for your secondary DB, you may or may not want this to happen automatically.

For your primary DB, you'll probably want to include steps in your backup strategy for clearing out older recovery journals. A common approach I've seen used previously is to remove anything older than 7 days, but this may not be sufficient if for any reason you have your secondary DB offline from your primary for > 7 days. You may need to implement additional checks using relevant methods on the JadeDatabaseAdmin class to ensure you only remove recovery journals from your primary that are no longer required by any secondary DBs.

You don't need to use jdbutil to remove any recovery journals you identify as no longer required, they can be removed via standard file delete processes.

Hope that helps,
BeeJay.


EDIT 13:32: Don't you just love it when another posting comes in on a thread while you're writing a reply.... especially when you leave it unfinished while you take a lunch break.

[Jade Support]

If the secondary is only being used as an RPS node for populating an MS-SQL database then its journals can be removed once they are no longer in use. Our recommendation is to set JournalCloseAction=Move which will move them to the archive directory (defined by JournalArchiveDirectory) when they are finished with, and then to manually clear this directory every so often.

However the primary journals should be backed up regularly because they may be required in the event that you need to restore your database from backup. Refer to the "JADE Database Admin Guide" (especially Chapter 3) for further information. Again it is recommended to set JournalCloseAction=Move so that JADE moves the primary journals once its finished writing to them.

You may also find the White Papers "Developing a Backup Strategy", "Relational Population Service (RPS)" and "Synchronized Database Service (SDS)" helpful..

[Michael.Duraj]

Thanks BeeJay and JadeSupport

Problem was ArchiveRecovery was never setup at client, I introduced RPS which enabled ArchiveRecovery and because of the volume of transactions bloated size of Jade system.

Current Setup is now;

Primary Server
------------------
[PersistentDb]

EnableArchivalRecovery=true
JournalArchiveDirectory=L:\Logs_Recovery
JournalCloseAction=MoveAndCompress

Secondary Server
--------------------

[PersistentDb]

EnableArchivalRecovery=true
JournalArchiveDirectory=D:\Logs_Recovery
JournalCloseAction=Remove

We have a scheduled that deletes log files on the Primary system older than 7 days

Code: Select all

forfiles /p " L:\Logs_Recovery " /s /m *.* /D -7 /C "cmd /c del @path"