#11 JADE 7.0 File Structure Changes and Memory Usage
Posted: Tue Sep 20, 2011 10:29 am
This bulletin is designed as a precursor to the full documentation included with JADE 7.0. It is not meant as replacement reading nor a white paper on memory management.
In earlier releases, the space to store a database entity (a JADE object or index block, for example) was allocated serially in a file. In this discussion, these entities are referred to as database objects. When a database object was deleted, its space was tracked and that space could be reused. The database engine made calls to the operating system to read or write individual database objects. The operating system buffered parts of database files in its own system-wide cache and it was responsible for all physical I/O to the database files.
The database engine retained copies of database objects in an object cache, to minimize calls to the operating system. The object cache employed a Least Recently Used (LRU) replacement policy to maintain a working set of objects. Only one thread at a time could access a database object in the object cache. Each database object contained a checksum field, updated when the database object was written to system cache and checked when the object was read from system cache.
New File Structure in JADE 7.0
Database files are now organized as a sequence of 8K byte disk blocks, which is the basic unit of I/O and free-space management. Database Btree index blocks fully occupy a disk block. Small objects are sub-allocated within a block and objects larger in size than 8K bytes span multiple contiguous disk blocks.
Disk Cache
A new block-level caching module, called disk cache, replaces the file caching and physical I/O functions previously performed by the operating system.
The content of a disk block is stored in a buffer; the working set of blocks buffered in memory is called the buffer pool. The buffer pool manager reads or writes blocks using direct I/O. Database objects are read and updated directly in the buffer pool, removing the need for an object caching layer and its data copying overheads.
What Does This Mean?
File data cached by the operating system is not included in any Process memory accounting. In prior JADE releases the memory consumed by operating system caching of database files wasn't attributed to the database process; It was part of the global total that can be seen as "Cached" under Physical Memory on the Task Manager Performance tab.
In JADE 7.0 the memory accounting for the database Process will have a larger in-use value than you have seen in prior releases. This is because the memory used for buffers holding the disk blocks (the file data) is now directly owned and attributed to the database process.
JADE 7.0 Migration Process
The memory usage you’ll see in Task Manager largely depends on your application. However, this change is seen mostly easily during the JADE 7.0 migration, specifically the ConvertDb phase.
The process allocates memory to the buffer pool until 256MB of memory is free. At first glance it might be disconcerting to see a single JADE process consuming many gigabytes of memory.
e.g.
The disk caching behaviour is controlled by the following INI parameter:
How is the Memory Released?
If free memory continually drops below the threshold JADE will continue releasing segments until it hits the configured minimum segments. But how does JADE calculate this using the DiskCacheFreeMemoryTarget value?
JADE calculates free memory as AvailableBytes + CacheBytes where AvailableBytes is the performance counter "\\Memory\\Available Bytes" and CacheBytes is the performance counter "\\Memory\\Cache Bytes".
At the time of writing the definitions of these counters were:
When memory is released you’ll see log entries like this:
In earlier releases, the space to store a database entity (a JADE object or index block, for example) was allocated serially in a file. In this discussion, these entities are referred to as database objects. When a database object was deleted, its space was tracked and that space could be reused. The database engine made calls to the operating system to read or write individual database objects. The operating system buffered parts of database files in its own system-wide cache and it was responsible for all physical I/O to the database files.
The database engine retained copies of database objects in an object cache, to minimize calls to the operating system. The object cache employed a Least Recently Used (LRU) replacement policy to maintain a working set of objects. Only one thread at a time could access a database object in the object cache. Each database object contained a checksum field, updated when the database object was written to system cache and checked when the object was read from system cache.
New File Structure in JADE 7.0
Database files are now organized as a sequence of 8K byte disk blocks, which is the basic unit of I/O and free-space management. Database Btree index blocks fully occupy a disk block. Small objects are sub-allocated within a block and objects larger in size than 8K bytes span multiple contiguous disk blocks.
Disk Cache
A new block-level caching module, called disk cache, replaces the file caching and physical I/O functions previously performed by the operating system.
The content of a disk block is stored in a buffer; the working set of blocks buffered in memory is called the buffer pool. The buffer pool manager reads or writes blocks using direct I/O. Database objects are read and updated directly in the buffer pool, removing the need for an object caching layer and its data copying overheads.
What Does This Mean?
File data cached by the operating system is not included in any Process memory accounting. In prior JADE releases the memory consumed by operating system caching of database files wasn't attributed to the database process; It was part of the global total that can be seen as "Cached" under Physical Memory on the Task Manager Performance tab.
In JADE 7.0 the memory accounting for the database Process will have a larger in-use value than you have seen in prior releases. This is because the memory used for buffers holding the disk blocks (the file data) is now directly owned and attributed to the database process.
JADE 7.0 Migration Process
The memory usage you’ll see in Task Manager largely depends on your application. However, this change is seen mostly easily during the JADE 7.0 migration, specifically the ConvertDb phase.
The process allocates memory to the buffer pool until 256MB of memory is free. At first glance it might be disconcerting to see a single JADE process consuming many gigabytes of memory.
e.g.
Above we see the persistent database allocating segment 93, taking the buffer pool size to 6.2GB.2011/09/19 10:03:52.076 017e4-0830 CvtDb: [187.4] projectData processing ...
2011/09/19 10:03:52.078 017e4-0830 PDB: projectData[51] added to control file
2011/09/19 10:04:13.126 017e4-041c PDB: DiskCache: segment allocated: _poolSegments= 7, pool size now 469762048 bytes (57344 blocks)
2011/09/19 10:04:24.636 017e4-041c PDB: DiskCache: segment allocated: _poolSegments= 8, pool size now 536870912 bytes (65536 blocks)
...
2011/09/19 10:25:17.649 017e4-041c PDB: DiskCache: segment allocated: _poolSegments= 93, pool size now 6241124352 bytes (761856 blocks)
- Example of the jadclient process during a JADE 7.0 migration.
- TaskManager.png (29.87 KiB) Viewed 5348 times
The default value is 256MB. During the migration phase it’s recommended to leave this unchanged for maximum performance. In fact, unless you are running JADE 7.0 on a dedicated server, don't modify any of the caching parameters. Let JADE manage this.[PersistentDb]
DiskCacheFreeMemoryTarget
How is the Memory Released?
If free memory continually drops below the threshold JADE will continue releasing segments until it hits the configured minimum segments. But how does JADE calculate this using the DiskCacheFreeMemoryTarget value?
JADE calculates free memory as AvailableBytes + CacheBytes where AvailableBytes is the performance counter "\\Memory\\Available Bytes" and CacheBytes is the performance counter "\\Memory\\Cache Bytes".
At the time of writing the definitions of these counters were:
- "\\Memory\Available Bytes" = the sum of memory assigned to the standby, free, and zero page lists
- "\\Memory\\Cache Bytes" = the sum of System Cache Resident Bytes, System Driver Resident Bytes, System Code Resident Bytes, and Pool Paged Resident Bytes
When memory is released you’ll see log entries like this:
2011/09/19 11:12:23.119 017e4-041c PDB: DiskCache:: system available bytes (261353472) less than DiskCacheFreeMemoryTarget (268435456), deallocating segment 94