Introduction
Physical, Meta and Logical certification of JADE databases should be undertaken on a regular basis to help ensure the integrity of each database. It has become apparent that not all customers are regularly running the certification process on their systems, with performance characteristics sometimes quoted as the reason.
A prerequisite of upgrading a JADE 6.3 database to JADE 7.0 is that the system is free of any logical inconsistencies.
The amount of work that the certification process needs to perform is relative to the size and complexity of the database concerned. This is because the logical certifier performs a number of checks to verify the referential integrity of user data. e.g. object A has a reference to object B, but object B’s inverse back is null, so the relationship is only known about from one side.
So if you have large numbers of objects combined with a complex model then the certification process may take a significant amount of time.
A JADE Logical Certify is by its nature an I/O intensive process and requires an I/O subsystem that is responsive.
Running the JADE 6.3 Logical Certify in the standard manner can take significant time depending upon the individual system, hardware and environment configuration. The performance improvements in JADE 7.0 have reduced the time taken for JADE logical certifies to complete. In addition to these improvements in the core product, a multi-threaded JADE Logical Certifier will be included in the JADE 7.0.5 Consolidated Release.
This source bulletin article covers the main points relating to performance you need to consider when running the JADE Logical Certifier in a JADE 6.3 system
Hardware and Environmental Configuration
The hardware and environment can greatly impact the time taken when undertaking a JADE Logical Certify. The following points should be considered:
- Certify is I/O intensive and requires an I/O subsystem that is responsive.
- Low latency disks can provide a significant performance boost to the process.
- SSD disks as very low latency disk and can provide a significant performance boost to the process.
- JADE 6.3 and prior Certify benefits significantly from sufficient file system cache.
- Certify benefits from using the 64-Bit edition of the binaries.
- Certify benefits from ensuring writeenabled is set to false in the JadeInterpreter section of your initialisation file.
The full Logical Certify should be run on a recent backup of the system. If the JADE 6.3 system is running 32 bit binaries, but the server where the Logical Certify is being carried out supports 64 bit binaries, you should run the JADE Logical Certify with matching 64 bit binaries.
Setting writeenabled to false in the JadeInterpreter section of the initialisation file will prevent the output from the Logical Certify appearing in the Interpreter output window and thus remove a potential bottleneck.
Ideally the JADE Logical Certify should be run on a dedicated server with the appropriate performance characteristics rather than on the same server as the running Production system. In any event, the JADE Logical Certify should be run when the server and I/O system is quiet to minimize the I/O contention with other processes. Generally the more powerful the hardware and the fewer other processes running the faster the Logical Certify process will complete.
Multi-Worker Logical Certify
JADE Logical Certification is normally carried out with a single-worker checking each class sequentially. The JADE 7.0.5 Consolidated Release includes a multi-threaded logical certification process. This multi-threaded approach leverages the concurrency improvements included in the JADE 7.0 product.
To assist with the migration from JADE 6.3 to JADE 7.0, a multi-worker JADE Logical Certifier kit is available from http://www.jade.co.nz/downloads/jade/JA ... kerKit.zip. The kit comprises of a schema to be loaded into the JADE 6.3 system. The supplied application generates class lists that will be processed by the specified number of logical certifier workers at a later time when multiple instances of the database are run in ReadOnly Mode. When the class lists are generated a batch file is also created. These are located in the directory specified on the application form. The Logical Certify should be carried out on a recent backup of the JADE 6.3 system.
The supplied kit can not be run on JADE 7.0. It can also not be run on a Primary or Secondary SDS database, to run this tool on such a database, the database role be will need to 'cleared' - to clear the database role, use the jdbutilb command 'clearRole'.
The steps required are shown below:
1. In singleUser mode load the supplied LogicalCertifierMultiWorker schema files into the database to be certified.
2. Either run the LogicalCertifierMultiWorkerApp from the Development Environment or as a singleUser application:
3. Enter the number of JADE Logical Certifier workers to be used - defaults to 4.<bin>\jade.exe path=<system> ini=<ini> schema=LogicalCertifierMultiWorker app=LogicalCertifierMultiWorkerApp server=singleUser JadeInterpreter.WriteEnabled=false
4. Enter the path of the directory tree to be used by the JADE Logical Certifier workers - defaults to <JadeHome>\LogicalCertifierMultiWorker.
5.Click the Gen Class Lists button. This will create the directories, assign classes to the workers, create the _logcert.in files for the workers.
6. Shut down the database completely.
7. Run the LogicalCertifierMultiWorkerApp in readOnly mode:
Note: When restarting the Logical Certifier Startup form ensure the number of workers and the directory path are as used in steps 3 and 4 above before clicking the Start Workers button.<bin>\jade.exe path=<system> ini=<ini> schema=LogicalCertifierMultiWorker app=LogicalCertifierMultiWorkerApp server=localReadOnlyServer JadeInterpreter.WriteEnabled=false
8. Click the Start Workers button. This will start the workers, and will show when they finish. Any workers that have encountered errors will be shown in red as they finish. Workers without errors will show in green as they finish.
9. Review all the worker log files and run any _logcert.fix files produced
10. Repeat steps 8 and 9 until the system is clear of errors or contact JADE support for further assistance.
Comments
The number of workers can be tuned to best suit the environment’s IO/CPU capabilities. For example, setting copies to 4 on a dual core machine would be redundant as you now have four threads fighting for contention over two CPU cores. A good rule of thumb (on a machine where the application has exclusive access to the CPU) is to set the workers = (CPU cores - 1)
If you've previously tuned the number of reorg workers for the database, this is likely to be a good starting point for setting the number of workers used in the Multi-Worker Logical Certifier phase.Note: The term ‘cores’ refers to both physical and logical cores, e.g. those that support hyperthreading.
The _logcert.in files can be modified before clicking the Start Workers button. The files can be reviewed and where desired classes can be moved to different workers.
Any workers that have encountered errors will be shown in red in the displayed output as they finish. Workers without errors will show in green as they finish.
The logical certifier logs for each worker are stored in the directory for that individual worker. Any reported errors and fixes must be reviewed and the fix files run against the system. These can be run in any order.