#15 Why do files sometimes grow after an online compact?

[Jade Support]

While it might not be expected, a JADE database file can increase in size as the result of performing an online updating compaction. This can be seen in the following reorg.log snippet indicating that the _userscm.dat file has increased in size by approximately 200Mb.

ReorgJob=1 commenced: updating compact of file: _userscm [31], workfile cache size=131072, map changes, updates allowed

commencing reorg of D:/server/c_system/_userscm.dat, operation: compact - logical eof = 5317634801
5508469 objects processed
9556997 subobjects processed
0 errors
old file size 5317636096(5317634801 used), new file size 5552920576(5552919214 used)
synchronising file: _userscm [31] commenced

The method used for performing an online updating compaction is slightly different to that used when performing an offline compaction. This difference can lead to larger files in the online updating case.

In an offline compaction, the btree structure of the source file is replicated in the compacted file, so btree loading is maintained. Because deleted object space is cleared, the file size is reduced.

In an online updating compaction, the btree structure changes because of transaction activity. For this reason the objects are copied by sequentially reading through the file (as opposed to navigating the tree structures). In general the btree loading in the reorged file is not as efficient when this process is used to copy data - hence the behaviour observed.

When a Jade Database Utility certify of a file is performed, the subobject index utilization is evaluated and recorded in the certify log. In this example it was found that the subobject index utilisation was 71% before the compact and 30% after the compact. The extra disk space required for the modified btree structure has outweighed the space recovered by removing deleted object space, consequently the file consumes more disk space following the online updating compaction.

Please note that the Jade Technology Group have developed some clever enhancements to the online updating compact algorithm that will result in better btree loading in JADE 7.