SHA-2 / SHA512 - tips anyone?

[concord]

I need to implement SHA512 within Jade.

SHA1 was simple and has been working well for years...

ExternalFunction
sha1(data: Binary; length: Integer; digest: Binary[20] output) is SHA1 in libeay32;

Any ideas for SHA-2 SHA512?

[murray]

I need to implement SHA512 within Jade.

SHA1 was simple and has been working well for years...

ExternalFunction
sha1(data: Binary; length: Integer; digest: Binary[20] output) is SHA1 in libeay32;

Any ideas for SHA-2 SHA512?

Hi concord,
I know I did the SHA-256 using libeay32 and that is working fine. I'll have to double check tomorrow and get back to you. Basically, you just need to have a look at the source code for libeay32 to see what's been implemented.
Stay tuned...
Murray.

[murray]

@concord, I just found the definition for SHA256...

Code: Select all

sha256(data: Binary; length: Integer64; digest: Binary[32] output) is SHA256 in libeay32;

I'm pretty sure I Haven't done SHA-512, but it should follow on from that.
Just bear in mind that the output size of the SHA-512 hash digest is larger at 512 bits which is 64 bytes (SHA-256 = 32 bytes).
When I'm talking of digest bytes, that is raw binary NOT hexadecimal or base 64 (some folks tend to get confused over that).
That is probably the only difference. It's quite likely that it's been implemented, as they're variations on the same algorithm.
Note: We had to adjust the length parameter from Integer type to Integer64 type when upgrading to 64 bit Jade.

Murray.

[murray]

Hi concord,

I had a look at SHA-512, and did a trial implementation today. As I suspected, it's just a small change on SHA-256.

Here's the external function definition:

Code: Select all

sha512(data: Binary; length: Integer64; digest: Binary[64] output) is "SHA512" in libeay32 applicationServerExecution;

The 'applicationServerExecution' clause is optional, but we use it to ensure the method runs on the server and not the client (mainly for 64-bit compatibility). You can leave it off if it causes no issues. I wrote some tests (of course) using some of NIST's test vectors and the results were all OK.

p.s. A SHA-512 digest is 64 bytes which expands out to 128 hexadecimal characters, not very "human readable"!

Murray.

[concord]

Brilliant, I'll give it a go today. Thanks!

[concord]

Murray I asked a similar question back in 2012 and you totally nailed it for me then too!

I have one final hurdle, I'm almost certain the HMAC wrapper I have in Jade for sha1 was probably written by you back in 2012, I'm sure I need to tweak a new version of this for sha512, I assume the BLOCK_SIZE is greater... 128?

Code: Select all

constants BLOCK_SIZE = 64; INNER_PAD = #[36 36 36 36 36 36 36 36 36 36 36 36 36 36 36 36 36 36 36 36 36 36 36 36 36 36 36 36 36 36 36 36 36 36 36 36 36 36 36 36 36 36 36 36 36 36 36 36 36 36 36 36 36 36 36 36 36 36 36 36 36 36 36 36]; OUTER_PAD = #[5C 5C 5C 5C 5C 5C 5C 5C 5C 5C 5C 5C 5C 5C 5C 5C 5C 5C 5C 5C 5C 5C 5C 5C 5C 5C 5C 5C 5C 5C 5C 5C 5C 5C 5C 5C 5C 5C 5C 5C 5C 5C 5C 5C 5C 5C 5C 5C 5C 5C 5C 5C 5C 5C 5C 5C 5C 5C 5C 5C 5C 5C 5C 5C]; vars keyI, keyO : Binary; bin1, bin2 : Binary; keyAdj : Binary; begin if key.length > BLOCK_SIZE then //--- key too big keyAdj := key.Binary.sha1 (true); keyI := INNER_PAD.bitXor (keyAdj); keyO := OUTER_PAD.bitXor (keyAdj); else keyI := INNER_PAD.bitXor (key.Binary); keyO := OUTER_PAD.bitXor (key.Binary); endif; bin1 := (keyI & data.Binary).sha1(true); bin2 := (keyO & bin1).sha1 (true); if raw_output then return bin2.String; else return bin2.toHexDigits.toLower; endif; end;

[concord]

Hmm.... the 512 digest I get for 'TEST' is

4A4144452028433A5C436F6E636F72645F6465765C73797374656D203A20416E64726577203A2073696E676C655573657229202D205B43534261736553636865

Using the following tool
http://www.freeformatter.com/sha512-gen ... #ad-output

I get the following digest 'TEST':
7bfa95a688924c47c7d22381f20cc926f524beacb13f84e203d4bd8cb6ba2fce81c57a5f059bf3d509926487bde925b3bcee0635e4f7baeba054e5dba696b2bf

[murray]

Make sure that your inputs are identical. I mean byte for byte. If you're sending strings to an online service it may use Unicode which is multiple bytes per character. Standard ANSI Jade is strictly one byte per character (as per ASCII coding). It only takes one bit difference to get a completely different output.

1. ASCII "TEST" = 54 45 53 54
2. Unicode "TEST" = 00 54 00 45 00 53 00 54

Testing the Jade call with ASCII "TEST" (no. 1 above, 4 bytes) I get

Code: Select all

00000001 7BFA 95A6 8892 4C47 C7D2 2381 F20C C926 {ú•¦.’LGÇÒ#.ò.É& 00000017 F524 BEAC B13F 84E2 03D4 BD8C B6BA 2FCE õ$¾¬±?„â.Ô½Œ¶º/Î 00000033 81C5 7A5F 059B F3D5 0992 6487 BDE9 25B3 .Åz_.›óÕ.’d‡½é%³ 00000049 BCEE 0635 E4F7 BAEB A054 E5DB A696 B2BF ¼î.5ä÷ºë TåÛ¦–²¿

Which matches the result you got from freeformatter.com.

Heh, believe it or not, the output you got from YOUR code is the hexadecimal represenation of the ASCII codes for the String...

Code: Select all

"JADE (C:\Concord_dev\system : Andrew : singleUser) - [CSBaseSche"

4A = "J", 41 = "A", 44 = "D", 45 = "E" , etc, etc
So I think a bit of checking is in order.

Here is the quick code I used to check the output of SHA_512("TEST")

Code: Select all

vars result : Binary[64]; begin call sha512( "TEST".Binary, 4, result ); write result.display; end;

[murray]

Murray I asked a similar question back in 2012 and you totally nailed it for me then too!

I have one final hurdle, I'm almost certain the HMAC wrapper I have in Jade for sha1 was probably written by you back in 2012, I'm sure I need to tweak a new version of this for sha512, I assume the BLOCK_SIZE is greater... 128?

Code: Select all

constants BLOCK_SIZE = 64; INNER_PAD = #[36 36 36 36 36 36 36 36 36 36 36 36 36 36 36 36 36 36 36 36 36 36 36 36 36 36 36 36 36 36 36 36 36 36 36 36 36 36 36 36 36 36 36 36 36 36 36 36 36 36 36 36 36 36 36 36 36 36 36 36 36 36 36 36]; OUTER_PAD = #[5C 5C 5C 5C 5C 5C 5C 5C 5C 5C 5C 5C 5C 5C 5C 5C 5C 5C 5C 5C 5C 5C 5C 5C 5C 5C 5C 5C 5C 5C 5C 5C 5C 5C 5C 5C 5C 5C 5C 5C 5C 5C 5C 5C 5C 5C 5C 5C 5C 5C 5C 5C 5C 5C 5C 5C 5C 5C 5C 5C 5C 5C 5C 5C];

Checking Wikidedia I see that SHA-224 and SHA-256 have a block size of 512 bits (64 bytes). SHA-384 and SHA-512 have a block size of 1024 bits (128 bytes). However, the block size of the HMAC function may be different. It will depend upon the block size of the HMAC algorithm, which may be tied to the hash size. I don't know what HMAC standard you are targetting. Whatever the block size, the padding is used to fill up to an entire block, as all data in the HMAC must be an exact multiple of the block size.

[concord]

Hi Murray, with "applicationServerExecution" my digest comes back totally empty. If I leave this out or insert "presentationClientExecution" I just end up with noise in the digest, e,g, data from the client shortcut or the JDE MDI caption.

e.g.
WORKSPACE

Code: Select all

vars bin : Binary; hash : Binary[64]; len : Integer64; begin hash := null; bin := 'TEST'.Binary; len := bin.length; call sha512 (bin, len, hash); write hash.display; write hash.toHexDigits.String; write 'Should be'; write '7bfa95a688924c47c7d22381f20cc926f524beacb13f84e203d4bd8cb6ba2fce81c57a5f059bf3d509926487bde925b3bcee0635e4f7baeba054e5dba696b2bf'.toUpper; end;

presentationClientExecution

Code: Select all

sha512(data: Binary; length: Integer64; digest: Binary[64] output) is "SHA512" in libeay32 presentationClientExecution;

output
00000001 4A41 4445 2028 433A 5C43 6F6E 636F 7264 JADE (C:\Concord
00000017 5F64 6576 5C73 7973 7465 6D20 3A20 416E _dev\system : An
00000033 6472 6577 203A 2073 696E 676C 6555 7365 drew : singleUse
00000049 7229 202D 205B 4353 4261 7365 5363 6865 r) - [CSBaseSche
4A4144452028433A5C436F6E636F72645F6465765C73797374656D203A20416E64726577203A2073696E676C655573657229202D205B43534261736553636865
Should be
7BFA95A688924C47C7D22381F20CC926F524BEACB13F84E203D4BD8CB6BA2FCE81C57A5F059BF3D509926487BDE925B3BCEE0635E4F7BAEBA054E5DBA696B2BF

applicationServerExecution

Code: Select all

sha512(data: Binary; length: Integer64; digest: Binary[64] output) is "SHA512" in libeay32 applicationServerExecution;

output

00000001 0000 0000 0000 0000 0000 0000 0000 0000 ................
00000017 0000 0000 0000 0000 0000 0000 0000 0000 ................
00000033 0000 0000 0000 0000 0000 0000 0000 0000 ................
00000049 0000 0000 0000 0000 0000 0000 0000 0000 ................

Should be
7BFA95A688924C47C7D22381F20CC926F524BEACB13F84E203D4BD8CB6BA2FCE81C57A5F059BF3D509926487BDE925B3BCEE0635E4F7BAEBA054E5DBA696B2BF