ExtraHuge StringArray and performance?

[concord]

My Table control is subclassed and has a HugeStringArray attribute. Instances of the table populate the HugeStringArray with cell contents in the default order. I then re-implement the displayRow method on the table and pull values from the HugeStringArray in a user specified order and construct the tab delim displayRow string.

Very occasionally I bump up against the max membership size of 2047 for the array.

Ok, so this is easily addressed by using a StringArray with larger members. Before I go ahead and change this I'm wondering if there will be any measurable impact on performance, will my transient cache jump right up if I implement a MassiveStringArray with a membership size of 15999?

[allistar]

I would think you'd be pushing the string pool harder, and perhaps the transient cache too. You could easily prove this by profiling a script that creates a real world MassiveStringArray and check out the impact on the caches, in particular the string pool. You'll probably benefit from a larger StringPoolSize as defined in the ini file - up to 20M maybe?

[murray]

Ok, so this is easily addressed by using a StringArray with larger members. Before I go ahead and change this I'm wondering if there will be any measurable impact on performance, will my transient cache jump right up if I implement a MassiveStringArray with a membership size of 15999?

I think the degree of impact on your transient cache will depend on the number of very long strings in the array. You mention that the limit is reached only "Very occasionally", so it may not be too bad. However, it would allow a "worst case" situation to occur where you could run out of space.
It is my understandanding that the space for each String in a StringArray is individually allocated (someone please correct me if I am wrong), so you would be increasing the "possible maximum size", which may or may not be acheived in practice.

[concord]

I should stop being lazy and profile a test.

Generally this array would have around 10-40 entries and the members average 5-20 characters.
It's only on those very rare occasions that a very large string is encountered.

I suppose the key thing I'm trying to workout is... is it max size x entries that contribute to the cache or the the actual member size x entries?

e.g.
MassiveStringArray, membership 10,000 bytes.
10 entries, each 10 characters in length.
cache impact is 10x10=100 or 10x10,000=100,000

I'm not concerned about performance on those occasions when very large strings are encountered. It's the other 99.9% of the time that I don't want effected. I very rarely have a form without at least 1 of 2 tables somewhere. My application can easily have 15+ forms open at once.

[allistar]

Instead of a HugeStringArray, why not go with an ObjectArray whose member is a class which has a maximum length String property on it? If the array only has 10 to 40 entries in it, you aren't creating a lot more objects, yet the memory usage is kept down as the maximum length String only takes up the space used by it.

[murray]

Instead of a HugeStringArray, why not go with an ObjectArray whose member is a class which has a maximum length String property on it? If the array only has 10 to 40 entries in it, you aren't creating a lot more objects, yet the memory usage is kept down as the maximum length String only takes up the space used by it.

In my opinion (see above) this would achieve the same as I believe the Jade StringArray would do anyway, but with added overhead.
I can't set up a controlled test at the moment - on a single user system the transient cache is just too busy to get a reliable reading.
I was hoping that someone from Jade would reply with a definitive answer on how the implementation of StringArrays uses space.

[BeeJay]

If you had a StringArray defined as length 300, each entry in the array will have the full 300 allocated not just the amount required for each string.

ie: They're implemented as fixed length arrays.

I can't definitively say what happens when you go beyond the Slob size for the string length in a StringArray, but I'm pretty sure that it is still implemented as a fixed length array even once you go beyond 540 characters.

Cheers,
BeeJay.

[murray]

If you had a StringArray defined as length 300, each entry in the array will have the full 300 allocated not just the amount required for each string.

ie: They're implemented as fixed length arrays.

I can't definitively say what happens when you go beyond the Slob size for the string length in a StringArray, but I'm pretty sure that it is still implemented as a fixed length array even once you go beyond 540 characters.

Cheers,
BeeJay.

Thanks for that, BeeJay.
In that case Allistar's suggestion is worth investigating. As in all things, there are pros and cons to weigh up.

[torrie]

If you're interested in the space taken up by an object, see the Appendix in the Developers reference. Unfortunately it doesn't mention arrays

[JohnP]

Like BeeJay, I am pretty sure the max size is allocated even if it is not used. I would be tempted to use a StringArray, and handle overflow (> 62 bytes) as a special case.

It should not be difficult to test whether the max size is always allocated. Just make an array of size 10,000 and a JadeScript to add enough entries for the cache consumption to be noticeable in JadeMonitor's Cache Performance sheet. "Available Buffer Size" tells you how much remains. For best results, bounce the database before trying it. SingleUser should be fine.