Anyone implemented a sha1 hash function?

[concord]

Had a crack at the hmac function, all I've managed is to take out my jade.exe:

Code: Select all

hmac(hash_func : String; // EVP_sha1 key: Binary; key_len: Integer; data: Binary; data_len : Integer; result:Binary[20] output; res_len : Integer output) is HMAC in libeay32; //TEST vars bin : Binary[20]; len : Integer; key : Binary; data : Binary; begin key := 'salt'.Binary; data := 'ABC'.Binary; bin := null; call hmac ('EVP_sha1', key, key.length, data, data.length, bin, len); write bin.String; write len.String; end;

[murray]

You have to get the parameters correct. It looks to me as though EVP_MD is a struct selecting the hash function, not a String. Passing bad data in pointers will cause bad things to happen.
There might be some help here http://www.openssl.org/docs/crypto/EVP_DigestInit.html, otherwise It might pay to download the source files and look at the header files.
You may have to call another function to initialise the hash structure first. I may not be much help with this unless I can get it to work myself.

[murray]

Ok Concord, I have had another look into this and worked out how to set up the hash data structure.
You need to call either EVP_sha1(), EVP_sha256(), or EVP_get_digestbyname( name );
The external function calls require many of the Jade Binaries to be declared of fixed size.
For that reason you need to code separate external calls for different hash sizes.
Here is a worked example for HMAC-SHA-1 and HMAC-SHA-256 using test vectors from RFCs 2202 and 4231.
It is just a quick test - not production code by any means...

External function definitions:

Code: Select all

evp_getDigestByName( pDigestName : String ) : Binary[72] is EVP_get_digestbyname in libeay32; get_evp_sha1() : Binary[72] is EVP_sha1 in libeay32; get_evp_sha256() : Binary[72] is EVP_sha256 in libeay32; hmac_sha1( hash_func : Binary[72] input; key : Binary; key_len : Integer; data : Binary; data_len : Integer; result : Binary[20] output; res_len : Integer output ) is HMAC in libeay32; hmac_sha256( hash_func : Binary[72] input; key : Binary; key_len : Integer; data : Binary; data_len : Integer; result : Binary[32] output; res_len : Integer output ) is HMAC in libeay32;

The JadeScript. You can either: (a) call evp_getDigestByName( "hash name" ), or (b) call get_evp_sha1() or get_evp_sha256(), as appropriate.

Code: Select all

test_libeay32_hmac(); constants // test vector from RFC2202 for HMAC-SHA-1 KEY1 = #[0b 0b 0b 0b 0b 0b 0b 0b 0b 0b 0b 0b 0b 0b 0b 0b 0b 0b 0b 0b]; // 20 bytes DATA1 = "Hi There".Binary; RESULT1 = #[b6 17 31 86 55 05 72 64 e2 8b c0 b6 fb 37 8c 8e f1 46 be 00]; //20 bytes // test vector from RFC4231 for HMAC-SHA-256 KEY2 = #[0b 0b 0b 0b 0b 0b 0b 0b 0b 0b 0b 0b 0b 0b 0b 0b 0b 0b 0b 0b]; // 20 bytes DATA2 = "Hi There".Binary; RESULT2 = #[b0 34 4c 61 d8 db 38 53 5c a8 af ce af 0b f1 2b 88 1d c2 00 c9 83 3d a7 26 e9 37 6c 2e 32 cf f7]; // 32 bytes vars evp_md : Binary[72]; result1 : Binary[20]; result2 : Binary[32]; resLen : Integer; begin // test 1 - HMAC-SHA-1 // evp_md := call evp_getDigestByName( "sha1" ); // either of these work evp_md := call get_evp_sha1(); call hmac_sha1( evp_md, KEY1, KEY1.length, DATA1, DATA1.length, result1, resLen ); write "HMAC-SHA-1: " & resLen.String & " bytes" & result1.display; write "result = " & (result1 = RESULT1).String & CrLf; // test 2 - HMAC-SHA-256 // evp_md := call evp_getDigestByName( "sha256" ); // either of these work evp_md := call get_evp_sha256(); call hmac_sha256( evp_md, KEY2, KEY2.length, DATA2, DATA2.length, result2, resLen ); write "HMAC-SHA-256: " & resLen.String & " bytes" & result2.display; write "result = " & (result2 = RESULT2).String; end;

p.s. I'm not 100% sure on the length of evp_md being 70 bytes. I originally figured 64 bytes, but that still caused a crash. It works for me.

(Edited on 16/08/2012 to correct size of evp_md structure from 70 to 72 bytes)

[concord]

Murry, you're a legend! Thanks.

The Openssl documentation was making my Jade head hurt, it's great to finally have something working...

Code: Select all

constants // test vector from RFC2202 for HMAC-SHA-1 KEY1 = 'salt'.Binary; // 4 bytes DATA1 = "ABC".Binary; RESULT1 = 'T_ø?úµš±Jð¿^(ª'; //20 bytes RESULT1a = '545f139df83ffab59a18b1144af01d15bf5e28aa'; //40 bytes vars evp_md : Binary[70]; result1 : Binary[20]; result2 : Binary[32]; resLen : Integer; begin evp_md := call get_evp_sha1(); call hmac( evp_md, KEY1, KEY1.length, DATA1, DATA1.length, result1, resLen ); write "HMAC-SHA-1: " & resLen.String & " bytes" & result1.String; write "result = " & (result1.String = RESULT1.String).String & CrLf; write "HMAC-SHA-1: " & resLen.String & " hex: " & result1.toHexDigits.toLower; write "result = " & (result1.toHexDigits.toLower = RESULT1a.String).String & CrLf;

[concord]

Wow the hmac external function is slow... 800 milliseconds!
I'm currently using the wininet dll to connect to one our websites, making a GET request to a php page, processing the response, closing the connection in 10-30 milliseconds!

Starting to wonder if I should press ahead with writing a bitXOr method on Binary and trying out your Jade based hmac solution.
I assume the bitXOr simply breaks the 2 Binary variables up into Bytes bitXOr's each byte and concatenates the results.

[concord]

Hmm.. It's highly dependant on the parameters, the params from your test Jade script take 3.9 seconds to execute the hmac call.

[murray]

I just ran a few timing checks on my PC, which is a 4 to 5 year old Pentium 4 running WinXP.
With a 1024 byte string I get the following results for HMAC-SHA-1 andHMAC-SHA-256, in microseconds
141, 129
81, 132
80, 128,
82, 121
The time includes both the call get_evp_sha1() and the call hmac_sha1(). I don't know why it takes so long for you, perhaps your data string is very large?

Also, bear in mind that the HMAC() function in the library is a top-level function that packages up the whole initialise, process and cleanup functions. It would not be efficient for computing many HMACs in sequence.
source from hmac.c ...

Code: Select all

unsigned char *HMAC(const EVP_MD *evp_md, const void *key, int key_len, const unsigned char *d, size_t n, unsigned char *md, unsigned int *md_len) { HMAC_CTX c; static unsigned char m[EVP_MAX_MD_SIZE]; if (md == NULL) md=m; HMAC_CTX_init(&c); HMAC_Init(&c,key,key_len,evp_md); HMAC_Update(&c,d,n); HMAC_Final(&c,md,md_len); HMAC_CTX_cleanup(&c); return(md); }

[concord]

Murray, thanks again, your assistance has been invaluable.

I knocked up a Binary::bitXor and then built an hmac method similar to yours.

I've thrown all sorts of tests at it, largish data chunks and a 64byte key and the longest it's taken is 2 milliseconds!

Cheers.

[murray]

That's good that you have got it working. The Jade code is a good way to see how the algorithm works.
Make sure you test with some approved 'known good" values, such as the test vectors in RFCs 2202 and 4231.

[concord]

My PHP solution has successfully authenticated and loaded about 30,000 auctions on TradeMe in the last month so I know it's rock solid. I've been testing against that.

I just don't like the fact that this system relied on a utility hosted on our website in order to communicate with TradeMe, so really wanted to phase it out and implement something more robust.

I've just loaded a couple of test auctions and gone ahead and pushed all this live. My live site is now happily communicating with TradeMe without the need for my php utlity page... I'm very happy. Thanks again.