Communication between separate JADE environments

[ConvertFromOldNGs]

by Eric Peachey >> Wed, 27 Oct 1999 21:24:32 GMT

Hello,

Imagine you have two separate JADE database systems, say a debtors system and some kind of booking system. When a client wants to book something in the booking system you want to check that client's debtor status before accepting the booking. How would you choose do this given the current mechanisms in JADE 5?

I suppose the options could be:

1) Have some kind of file export/import mechanism for loading files periodically from the debtors system.
2) Write applications in both systems that communicate using tcpip connection class objects to pass messages between the two systems.
3) That ActiveX Exposure thing. Is that designed for communication between JADE systems or is it really designed for enabling non-JADE systems to communicate with JADE without rolling your own C++ code to use the JOMAPI?
4) Define a relational view of the debtors system and then define an external database in the booking system!?
5) Roll your own C++ dll that somehow attaches to both databases at the same time using the JOM API (don't know if you can do this but sounds insane anyway).
6) Somebody suggested using LDAP but I no nothing about this.

None of these seem to be ideal.

I'd be interested in your thoughts.

Thanks,

Eric in Dunedin with a scintillating view down the harbour this morning.

[ConvertFromOldNGs]

by Craig Shearer >> Wed, 27 Oct 1999 21:50:00 GMT

Hi Eric

You're right that this is a problem that's not handled very well in JADE. My guess would be that the TCPConnection class idea is probably the best to go with for now.

One thing that needs to be considered when updating two databases is the possibility that they will get out of step with each other, with one database committing the transaction and the other aborting it. You need some sort of two-phase commit mechanism which JADE doesn't possess. Food for thought, I guess!

Rumour has it that the plant are looking at the very issue of how multiple independent JADE systems could communicate.

Craig.

[ConvertFromOldNGs]

by Krull >> Thu, 28 Oct 1999 1:56:32 GMT

One thing that needs to be considered when updating two databases is the possibility that they will get out of step with each other, with one database committing the transaction and the other aborting it. You need some sort of two-phase commit mechanism which JADE doesn't possess. Food for thought, I guess!

Two-phase commit transactions are sometimes used for synchronous transaction processing between multiple disparate systems. An alternative and currently more popular approach is to decouple transaction processing using an asynchronous messaging system that can provide guaranteed delivery. This style of asynchronous processing is generally supported by some of 'Message Queue' such as IBM's MQ Series or Micro$ofts Message Queue component object.

[ConvertFromOldNGs]

by Allistar Melville >> Wed, 27 Oct 1999 9:39:47 GMT

There are 3 ways this could work:
System 1 initiates all communication with system 2
Both systems commincate with the other system when they please
A third party system acts as an arbiter between both systems

From all of the possibilities you have given above I would either go
for tcp-ip communication or a custom made dll (option 5).

With the dll option you would need one dll for each different system
you are communicating with. If one system handles all communication
then you only need one dll, if both system talk to each other then two are needed.

As painful as it sounds writing a c++ dll to do just this is actually quite straight forward. If you want an example of how the dll connects
to a jade database and calls methods in it then email me and I'll see what I can do.

Thanks,

Eric in Dunedin with a scintillating view down the harbour this morning.

Regards,
Allistar in Auckland where the harbour bridge is covered in fog.

------------------------------------------------------------------
Allistar Melville (BSc) \_
Software Developer </'
Auckland, NEW ZEALAND /)
(/`
Greentree International,
Developer of Greentree Financial Software. ------------------------------------------------------------------

[ConvertFromOldNGs]

by Krull >> Thu, 28 Oct 1999 1:42:57 GMT

With the dll option you would need one dll for each different system you are communicating with. If one system handles all communication
then you only need one dll, if both system talk to each other then two are needed.

As painful as it sounds writing a c++ dll to do just this is actually quite straight forward. If you want an example of how the dll connects
to a jade database and calls methods in it then email me and I'll see what I can do.

The DLL interface approach described above works fine when you are interfacing from a non-JADE application, but strange as it may seem, it will not work in such a straightforward manner from a JADE application. There are some assumptions in the JOM kernel, which assume a single node instance per executable (Windows process). A JADE standard client, application server or database server host a JADE node; in order to connect to a different JADE system, the proposed DLL would need to first call the jomInititialize API, which creates another JADE node violating certain assumptions in the kernel.
In order to connect to a different JADE system using the JOM API from an existing JADE node you need to provide the equivalent of a COM 'out-of-process' activation. Without going into too many gory details, this requires initiating a separate JOM node from a separate host process and performing tha JOM API calls indirectly via an LPC (local procedure call) mechanism assuming the two nodes are on the same machine; or an RPC mechanism otherwise. This is not a trivial exercise but is currently being done in a third-party tool that uses the JADE Object Manager.

[ConvertFromOldNGs]

by Krull >> Thu, 28 Oct 1999 2:17:12 GMT

Eric Peachey wrote:

"I suppose the options could be:

1) Have some kind of file export/import mechanism for loading files periodically from the debtors system.
2) Write applications in both systems that communicate using tcpip connection class objects to pass messages between the two systems.
3) That ActiveX Exposure thing. Is that designed for communication between JADE systems or is it really designed for enabling non-JADE systems to communicate with JADE without rolling your own C++ code to use the JOMAPI?
4) Define a relational view of the debtors system and then define an external database in the booking system!?
5) Roll your own C++ dll that somehow attaches to both databases at the same time using the JOM API (don't know if you can do this but sounds insane anyway).
6) Somebody suggested using LDAP but I no nothing about this.

None of these seem to be ideal."

None of them are ideal but in current releases 3,4 and 5 won't work.

You might be able to use option 3) in JADE 5.1 if the JDC COM team support the 'out-of-process' activation of a JOM server component, but it won't work right now for similar reasons to why 5) doesn't work (described in separate post to this thread).
Option 4) will not work because the JADE ODBC driver only has ODBC core level 0 compliance, which is not sufficient to support JADE's external database connectivity. Even if it did work you really wouldn't want to do it this way - yech!

A variation on 5) that could work, is to 'roll your own' DLL interface to an asynchronous messaging system. This approach has some advantages over the simple TCP/IP connection, which will work; several JADE users are already using the TCP connection class to communicate between JADE systems.

[ConvertFromOldNGs]

by Graeme Joyce >> Thu, 28 Oct 1999 20:49:48 GMT

Imagine you have two separate JADE database systems, say a debtors system and some kind of booking system. When a client wants to book something in the booking system you want to check that client's debtor status before accepting the booking. How would you choose do this given the current mechanisms in JADE 5?

<snip>

6) Somebody suggested using LDAP but I no nothing about this.

Eric,

LDAP (Lightweight Directory Access Protocol) may or may not be a good solution to your problem, but here's how it works...

There seem to be two problems: a single entity (the client) is represented in more than one place, and there is no standard way of accessing the data.
Assume for the moment you have a working LDAP server available ("the directory"). You could use the Jade LDAP classes (soon to be released in CardSchema) to publish selected properties (eg debtor status) for each client in the directory. You can then query that data from any LDAP compliant system (eg your booking system). LDAP includes access controls to maintain data security.

There is the issue of having an additional service to operate, although it is possible the client will already have a directory server in place for other reasons (remember when web servers were a novelty?). There will also be issues of transaction control/ synchronisation to consider.

In general a directory is a good place to put data that needs to be visible in a lot of places, but that doesn't change too often (the server is optimised for read performance).

If you'd like to know more, I have a working Jade system here that integrates data from about 6 different databases and publishes it in an LDAP directory. We have also written some simple LDAP clients in Jade.

Graeme
Jade Utility - Jade Systems Group

[ConvertFromOldNGs]

by Robin Arzoni >> Fri, 29 Oct 1999 6:05:06 GMT

Hi Eric,

Re: your comment

">6) Somebody suggested using LDAP but I no nothing about this."

LDAP would be a good solution to this problem. Briefly, LDAP (Lightweight Directory Access Protocol) is an emerging Internet standard for directories. The idea is that you put all the client's data into one directory entry, and then multiple LDAP-enabled systems can view and update this single entry. The really neat thing is that you can just ask for the attribute you want: for example, 'debtor status': you don't have to fetch the whole entry.

We now have a set of JADE LDAP classes that make it easy for a JADE app to search and update any LDAP directory (assuming that the client has the appropriate permissions). These classes will be incorporated into CardSchema.

LDAP promises to put an end to the kind of problems that arise when your data is scattered over different repositories: for example, data being out of sync, the issues of communication between systems, and so on.

X500 should have solved these problems years ago, but it was too heavy and difficult to implement, and was OSI-based. LDAP is a lightweight version of X500, and is TCP/IP-based: hence its rapid adoption by the Internet community.

LDAP actually consists of three parts:

1. A standard directory structure.
2. A client-server protocol that searches and updates the directory structure.
3. An API that drives the client-server protocol.

Setting up the LDAP directory itself is a separate exercise. For more info on LDAP, see http://www.mjwilcox.com/ldap/.

We are now starting to use LDAP in Aoraki's internal systems, to great effect. For example, any JADE app that needs to know an Aoraki employee's email address or pager number (or access rights, or shoe size, or ...) can search the Aoraki LDAP server.

Hope this helps,
Regards,
Robin Arzoni
JADE Systems Group

[ConvertFromOldNGs]

by Wilfred Verkley >> Tue, 2 Nov 1999 1:08:32 GMT

We are beginning to use LDAP ourselves in some of our applications. In our last app we needed to share user and security information amongst different programs (not all of them Jade). At present we have written a C++ DLL to act as a LDAP client and to allow Jade to talk to LDAP through external function calls.

Its all fairly specific to our application though, it would be nice if we had something more standard and generic that was easier to adapt to all of our Jade stuff. Is there any chance this stuff in the CardSchema will be made available to the poor starving masses?

Wilfred.

[ConvertFromOldNGs]

by Robin Arzoni >> Tue, 2 Nov 1999 21:23:31 GMT

Wilfred,

Yes, CardSchema has no elitist agenda, and is available to the huddled masses for a fee of $NZ 0.00. However, it's worth considerably more than you pay for it. Richard Mitchell (rmitchell@jade.co.nz) is the contact for obtaining a copy.

As you point out, an LDAP server can provide information to both JADE and non-JADE apps. I also like to think of the LDAP server(s) as being a supplement to the JADE database.

I'll email a copy of the JADE LDAP documentation to you so that you can see if the classes do what you want.

Cheers
Robin