getting an object's keys in a dictionary

[ghosttie]

Is this the only way to get the keys of an object in a dictionary or am I missing something?

Code: Select all

iter := dict.createIterator; iter.startAtObject(obj); iter.getCurrentKeys(aKey);

Follow up question: is it faster to do this or to get the key value from the object itself (which loads the object into cache). I ask because I've benchmarked it and they came out about the same, but I imagine it would depend on the size of the collection etc.

[JohnP]

There are two other ways of getting the keys. They are mentioned on page 28 of the JADE 6.3 "Design Tips for Better Performance" whitepaper (PerfDesign.pdf).

As mentioned in the whitepaper, the primary benefit in getting the keys in this way is the avoidance of the IO required to fetch the object. If you will be fetching the object anyway, getting it from the object is faster. To test the difference properly on 6.3, you need to select a test case large enough to take a significant amount of time - maybe a minute - and reboot the server machine between tests. The second run after a reboot will frequently be a lot faster than the first run, as the objects are already in cache. See the Micro-Benchmarking section on page 4 of the whitepaper. On JADE 7, you only need to restart the database server to get a good IO test, as JADE 7 uses its own cache rather than Window file-system cache.

The secondary performance impact is locking. Iterator::getCurrentKeys does not lock the collection. If the collection is already locked, eg if you are in transaction state, then the additional locking may not make a difference. If the collection is not locked, using Iterator::getCurrentKey on each of several keys can cost several locks per object. Locks are more expensive if they are done on a client node, especially if the node is connected to the server node via tcp.

[ghosttie]

Thanks, I wondered what the difference between Iterator::getCurrentKeys and Dictionary::getIteratorKeys was (apparently it's whether the collection gets locked or not).

However both of the alternative methods involve an Iterator - I was hoping to be able to just call something like Dictionary::getObjectKeys.

I did some benchmarks in JADE 7 (so I only had to restart the DB, not the computer) with over a million objects in the collection and got these results (single user):

Object::property = 00:02:51.506
Iterator::getCurrentKeys = 00:03:41.752
Iterator::getCurrentKey = 00:03:35.715
Dictionary::getIteratorKeys = 00:03:34.407

So at least in this case it seems that getting the key from the object itself is faster than getting the key from the dictionary. On the other hand I imagine the size of the object would affect this result, and filling up the cache with those objects could negatively affect other processes.

[murray]

I can't see how you could get keys from just a Dictionary, as it needs to know which object you are requesting the keys for.
The Dictionary does not maintain any state ("pointer") for your currently accessed object.
As I see it, your desire for "Dictionary::getObjectKeys" would be equivalent to "Array.getIndex" (without any parameters).
The purpose of the Iterator is to maintain a state with a pointer to a member within the dictionary, so it knows which object to reference.
You may have several Iterators referencing the same collection, how would the Dictionary know which object to return the keys for?

If you already have the object, then the keys are simply the relevant attributes of the object (defined as keys on the dictionary), accessed directly.
That would be faster in that case, but if you were iterating the collection without fetching objects, then it may be quicker to fetch the keys from the Iterator.

I guess what I am trying to say is: it depends what you are doing and what you are trying to achieve.

[ghosttie]

I can't see how you could get keys from just a Dictionary, as it needs to know which object you are requesting the keys for.

Sorry, I guess that wasn't clear - you'd pass the object you wanted the keys for, for example dict.getObjectKeys(obj, aKey). As you say, it would be like Array::getIndex. It just seems unnecessary to have to create an Iterator etc. when I have the dictionary and I have the object and I just want to get the key.

In this specific case I'm trying to sort a collection of objects by a derived value. I can avoid loading each object in the collection into the cache to get the key by using iter.startAtObject(obj); iter.getCurrentKeys(aKey); on a different collection that happens to be keyed on the right values. As I'm sure we're all aware, search-type code performs a lot better if we avoid loading all the objects into the cache, so I usually try to find a way to do what I need using Iterators, manipulating Collections of Object references without bringing the Objects into cache and getting keys from the Iterator. Unfortunately in this case it doesn't seem to be faster.

[murray]

Hi ghosttie,
I can see a bit clearer what you are wanting. I thought that your desired Dictionary::getKeys() method would have to fetch the object anyway to get its key values.
However, I can see that it would work (theoretically) if the Dictionary, given only the oid, could locate and return the key values that resolve to the object reference.
The Btrees used by Dictionaries are not structured to do this and so it would either be very inefficient (e.g. linear search) or would require a separate internal index structure keyed by oid (much additional overhead and space).

[ghosttie]

The Btrees used by Dictionaries are not structured to do this and so it would either be very inefficient (e.g. linear search) or would require a separate internal index structure keyed by oid (much additional overhead and space).

That would explain why it's slow to get the keys using iter.startAtObject(obj); iter.getCurrentKeys(aKey); - I assume startAtObject is getting to the object in a slow way.

[JohnP]

Iterator::startAtObject does a lookup similar to getAtKey, so you probably want to avoid that in this instance.

If the collection you were starting from had the right keys, you could use Iterator::getCurrentKeys to good effect. I sometimes add multiple low-order keys to a collection specifically so I can use Iterator::getCurrentKeys to retrieve them. Putting them after all the existing keys means they don't affect the order of the collection.

If you want to avoid flushing out all the cache with these objects, try using Process::adjustObjectCachePriority and Process::setObjectCachePriority. Specifically, this code will remove the object from the node's cache:

Code: Select all

process.setObjectCachePriority(obj1, 0);

[ghosttie]

Code: Select all

process.setObjectCachePriority(obj1, 0);

Ooo, I didn't know about that