Forums

by JADE Kid - Ray Hidayat >> Tue, 18 Jul 2000 6:37:20 GMT

If you saw all of my previous posts on instantaneous transactions and everything you will know that I have decided to load a file into a BinaryArray.

I set off to do this, when I found that BinaryArray only allows each part to be 128 bytes long. Now remember I want to load files that could be huge! So I set off to subclass it.

I made each section 65536 bytes long. That worked.
I made an attribute to use that. That worked.

But when I ran it:

Jade didn't like it. It said...
"Abnormal Program Termination"

Now I am fully aware of these BLOBS (Binary Large Objects).
If you didn't know already, any binary that has a length of over 540 bytes, is considered to have variable length. Those are Blobs. There are also Slobs (String Large Objects). Funny names huh?
So I made the size 256, which is less than 540. And the same error came up. I don't know why.
So I don't think that I can subclass BinaryArray.
The only exception to this exception (error, hahahahha) is when I load a file that is less than 65536 bytes long.

When I use the BinaryArray normally, it works.


--
Ray Hidayat
JADE Kid - 2000
www.jadekids.com

by bok >> Tue, 18 Jul 2000 11:51:29 GMT

Lo Ray,

I set off to do this, when I found that BinaryArray only allows each part to be 128 bytes long. Now remember I want to load files that could be huge! So I set off to subclass it.

I made each section 65536 bytes long. That worked.
I made an attribute to use that. That worked.

But when I ran it:

Jade didn't like it. It said...
"Abnormal Program Termination"

Firstly, there a couple of JADE buglets (that give rise to your abnormal program termination):
1) When you subclass BinaryArray and define entry size = 65536, notice that Jade leaves the "Maximum Block Size" attribute = 2048. The maximum block size needs to be at least as big as the entry size (JADE shouldn't let you subclass an array and increase the entry size in the subclass). If you want a Binary (or StringArray) with an entry size larger than the default blocksize (2048) you need to subclass Array not BinaryArray. If you do that JADE correctly sets the max block size (Coll class attribute), however there is another limit:

2) JOM currently has a 16K max buffer size limit for collection blocks, so JADE should not let you define an array that can result in a max block size

16K. (It does currently - so if you do you will crash).

Notwithstanding the above buglets, in general, I would advise against using binary or string arrays with large fixed entry sizes until JADE supports variable entry-size arrays (if you wish to store variable size data in each element, that is). In your case you would be ok, if you use fixed size chunks and stick to a chunksize < 16K (only the final chunk would result in wasted space). The main problem is that arrays currently allocate a fixed size slot for each element (the size you define for the your string/binary array). The stuff you mention below re blobs and slobs applies to binary and string attributes with a *schema defined* (not runtime) length > 540; it does not apply to elements in a string or binary array.

One way to provide variable length storage more efficiently is to define a 'variable storage class' with a variable length binary property and use an array of 'variable storage class' objects.

A fairly general solution, that I have recommended to some people in the past, is to implement a Variant class and an Array of Variant. This is relatively easy to do in JADE and provides a reusable implementation of an 'any array'. A skeletal Variant class definition is shown below:

Variant
(
myValue : Binary protected, subid = 1;
myType : Type protected;
)

We need to store the 'any value' in a binary because you can't currently define a real attribute of type Any. It would be nice to be able to define a virtual attribute of type Any, which we could then implement with a mapping method (BTW - That's an NFS). The alternative is to define get and set methods along the lines of the following:

getValue() : Any;begin

// return the internal value as an Any value of the actual type
return myValue.asType(myType);
end;

setValue(value : Any) updating;
// store the input value as a binarybegin

// NB: the intrinsic Binary conversion operator doesn't support coverting an object reference
myValue := value.Binary;
myType := value.getType;
end;

All you need then is to define a subclass of ObjectArray (VariantArray say) with membership Variant, simple eh?

The above methods use the following primitive methods: Binary::asType(type : Type) and
Any::getType() : Type

I have attached a sample schema, which provides an implementation of the Variant class and VariantArray along with a couple of test Jade scripts one of which loads a binary file into a variant array and then writes it back out from the array.

Now I am fully aware of these BLOBS (Binary Large Objects). If you didn't know already, any binary that has a length of over 540 bytes, is considered to have variable length. Those are Blobs. There are also Slobs (String Large Objects). Funny names huh?
So I made the size 256, which is less than 540. And the same error came up. I don't know why.
So I don't think that I can subclass BinaryArray.

Well, you can as long as you keep in mind the restrictions.

cheers

BOK
BokTek Services

by Bok >> Wed, 19 Jul 2000 1:49:18 GMT

I accidentally left some 5.1 schema syntax in the proto.scm in my prior posting; the attached version is 5.0.16 compatible.

cheers
bOK

by JADE Kid - Ray Hidayat >> Sat, 22 Jul 2000 0:11:44 GMT

Thank you very much. I will look into this.

--
Ray Hidayat
JADE Kid - 2000
www.jadekids.com