Load balancing

[ConvertFromOldNGs]

by johnmunro >> Thu, 14 Nov 2002 11:35:44 GMT

I've had this conversation with various Jade employees several times in the past, but I'm afraid I'm still rather in the dark, so I thought I'd have another go. I've searched the newsgroups but can't find a similar discussion.

We will limit this discussion to thin client connections, because the performance and requirements of the thick client have made us rule it out.

The problem is this: every Jade client needs to have the IP and port number of the application server it needs to connect to hard coded into its shortcut. It doesn't take much thought to see the problem - arranged in this way, it is likely that one application server will be overworked while another one sits idle. The bigger the client installation, the bigger the problem will be.

I am frankly surprised that Jade does not have a solution to this problem - I would have expected some sort of load balancing to be built into the basic Jade system. The one solution I know about was done by having all the clients connect to a name which is then resolved using DNS, and the DNS server returns a different IP address each time the DNS is resolved, so clients would connect to the app servers in round-robin order. There are a couple problems with this, first of all that it is round-robin and not intelligently assigning the least busy app server, and also it doesn't work if you have more than one app server on the same computer using different ports.

I wrote a solution, but ran into Jade problems and the project stalled there. First of all, clients do not run jade.exe, they run a VB app. The VB app connects to the database server through TCP (so all clients need to know the database server IP and the load balancing port number). There is a server application running on the database server which listens for these connections and replies to them with the IP and port of the least busy application server. The VB app then runs jade.exe connecting to the app server as normal. All of this works perfectly, except that the server application cannot get the port number from the application servers. Port isn't a property of Node (the app server), it's a property of Process (an application running on the app server). So if there is nothing running on the app server, we cannot get the port number at all, so cannot direct any clients to connect to it... Jade Support said that this could not be easily changed because it was a structural thing. This is not a huge problem at the moment though, because we have a server application which runs on each app server, so there is always at least one Process we can query for the port. The more serious problem is that Process.appServerPort always returns 0 when called from a server application running on the database server.

In communication with Jade Support, they suggested reading the app server IPs and ports from an ini file or retrieving them from the command line used to start each app server. The problem I had with this are that it always seemed to return the command line of the database server, not a particular application server. This may just be me getting confused about where the code is running, but I just couldn't ever get it to work. I think what I need to happen is for code running on the database server to cause some code to run on the application server which returns the port number back to the app running on the database server... Any help would be more than welcome.

Silvermoon Software's Jade Connection Manager product seems to do everything we want to be able to do (although I have no idea how they do it), but I'm afraid it'd be difficult/impossible to convince my superiors to add another external dependency/cost to our product.

John Munro

---
Synergist Limited - Home of FileVision
The Bioscience Innovation Centre
Cowley Road, Cambridge, UK
CB4 0DS

Telephone: +44 (0) 1223 478200
Fax: +44 (0) 1223 477969
Email: john.munro@filevision.com
Web: http://www.filevision.com

The contents of this communication are confidential and are only intended to be read by the addressee. We apologize if you receive this communication in error and ask that you contact Synergist Limited immediately to arrange for its return. The use of any information contained in this communication by an unauthorized person is strictly prohibited. Synergist Limited cannot accept responsibility for the accuracy or completeness of this communication as it is being transmitted over a public network. If you suspect this message may have been intercepted or amended, please inform Synergist Limited.

[ConvertFromOldNGs]

by barryg >> Thu, 14 Nov 2002 19:27:37 GMT

We dont have enough concurrent users to warrant a load balancing setup but we do
update each users shortcut from the NT logon script. Which helps if you want to manually
balance your users or move them to a different server in a DR situation ?

[ConvertFromOldNGs]

by allistar >> Thu, 14 Nov 2002 21:37:27 GMT

Hi John,
Allistar here (the developer of the Silvermoon Software "Connection Manager for Jade" utility).

It sounds like your solution is pretty much the same as what I have implemented in C++, using TCP/IP to connect to a server application
which tells the client which ip/port/schema/app/ini file to use.

Not being able to get the port off the application server is indeed
a pain, and it took me a few attempts to figure out how to get around that limitation (in my opinion it is a Jade bug, and things would be
much simpler if it were adressed).

Since I'm such a nice guy (:-p) here is how I got around it:

Let's say you have 3 app servers. I have a class in the database
called "ConnectionAppServer" that has a "myNode" reference and a "portNumber" and "ipAddress" property.
These get setup in an administrative screen, and the user specifies
the ip address and port number. Adding a new one is a simple matter of entering this information into a table on a form.

When the first clients connect, the "myNode" reference on the app
servers are all null. I have amethod called "getBestApplicationServer" and it does this:

-iterate through all "ConectionAppServer" objects
-if any of them have a "myNode" set to null, then pick that one. In
this situation the assumption is that this app server has not yet been connected to, so is a prime target for connecting to now.
-on the "initialise" method of the actual thin client, you can now get the port number of the application server, so get that and find the matching ConectionAppServer object, and set the "myNode" reference to
the current node.

So what you have is this happening:

- database server is started, app servers are started
- first client connects:
App1 (myNode = null) ***picks this one
App2 (myNode = null)
App3 (myNode = null)

-second client connects
App1 (myNode = not null)
App2 (myNode = null) ***picks this one
App3 (myNode = null)

-third client connects
App1 (myNode = not null)
App2 (myNode = not null)
App3 (myNode = null) ***picks this one

-forth client connects
App1 (myNode = not null)
App2 (myNode = not null)
App3 (myNode = not null)
***now it has a link between the "node" and the port/ip address it can now do offical load balancing by picking the node with the least
number of connected clients.

That's how I got around this limitation.

Obligatory advert for my product:
=================================
There are other things that the connection manager does other that
load balancing.
- it works with fat OR thin clients
- it synchronised bin directory files over the open TCP/IP connection
- all clients settigs are stored on the server and are passed to the client on each connection (including the ini file).

Take a closer look at:
www.silvermoon.co.nz/jcm.html

I would suggest that the price of my solution ($500 for unlimited
single database use) would be less expensive than developing and supporting it yourself (note that there are currently > 100 Jade sites using the connection manager utility).

I hope this information helps,
Allistar. ------------------------------------------------------------------
Allistar Melville
Software Developer
Auckland, NEW ZEALAND

Greentree International,
Developer of Greentree Financial Software. ------------------------------------------------------------------

[ConvertFromOldNGs]

by johnmunro >> Fri, 15 Nov 2002 12:07:46 GMT

Since I'm such a nice guy (:-p) here is how I got around it:

You are THE man Thanks a billion - anything I can do for you, just ask
John Munro

---
Synergist Limited - Home of FileVision
The Bioscience Innovation Centre
Cowley Road, Cambridge, UK
CB4 0DS

Telephone: +44 (0) 1223 478200
Fax: +44 (0) 1223 477969
Email: john.munro@filevision.com
Web: http://www.filevision.com

The contents of this communication are confidential and are only intended to be read by the addressee. We apologize if you receive this communication in error and ask that you contact Synergist Limited immediately to arrange for its return. The use of any information contained in this communication by an unauthorized person is strictly prohibited. Synergist Limited cannot accept responsibility for the accuracy or completeness of this communication as it is being transmitted over a public network. If you suspect this message may have been intercepted or amended, please inform Synergist Limited.

[ConvertFromOldNGs]

by johnmunro >> Fri, 15 Nov 2002 19:54:29 GMT

I had a thought - we're treating null (or invalid) myNode references as empty App Servers, but it could also mean that the App Server isn't there (service down, server switched off, network inaccessible or whatever). So we would route all new clients to an ip/port that doesn't have an App Server there to receive the connections, even thought there may be other App Servers which are running - black hole!

So I suppose what we need is for the load balancing application to check whether there actually is an App Server at the ip/port if the myNode reference is null/invalid. My first thought was to have the load balancing application spawn a thin client connection app pointed to the ip/port which would somehow report back whether it was able to connect or not. A smaller-footprint option might be to have the load balancing application just try to connect to the ip/port using TCP and assume that if there's someone there accepting connections it is probably an App Server.

As a further optimisation, it might be a good idea for the load balancing application to mark an appserver record as being dead (without actually deleting the record) to keep it from doing the "are you there" check every time a new client connection arrives and there's an appserver record with a null/invalid myNode reference. It could then be manually reactivated when the downed App Server is brought back up, or perhaps automatically reactivated (every few client connections it retries appserver records marked dead, or something).

John Munro

---
Synergist Limited - Home of FileVision
The Bioscience Innovation Centre
Cowley Road, Cambridge, UK
CB4 0DS

Telephone: +44 (0) 1223 478200
Fax: +44 (0) 1223 477969
Email: john.munro@filevision.com
Web: http://www.filevision.com

The contents of this communication are confidential and are only intended to be read by the addressee. We apologize if you receive this communication in error and ask that you contact Synergist Limited immediately to arrange for its return. The use of any information contained in this communication by an unauthorized person is strictly prohibited. Synergist Limited cannot accept responsibility for the accuracy or completeness of this communication as it is being transmitted over a public network. If you suspect this message may have been intercepted or amended, please inform Synergist Limited.

[ConvertFromOldNGs]

by allistar >> Fri, 15 Nov 2002 23:18:29 GMT

(My comments are inline below)

I had a thought - we're treating null (or invalid) myNode references as empty App Servers, but it could also mean that the App Server isn't there (service down, server switched off, network inaccessible or whatever). So we would route all new clients to an ip/port that doesn't have an App Server there to receive the connections, even thought there may be other App Servers which are running - black hole!

This is an unfortunate side effect of the Jade limitation. What the ConnectionManager software does is it allows an administrator to very easily remove an application server from the pool of available app servers (just by clicking a check box). It is equally easy to add another application server to the pool.

So I suppose what we need is for the load balancing application to check whether there actually is an App Server at the ip/port if the myNode reference is null/invalid. My first thought was to have the load balancing application spawn a thin client connection app pointed to the ip/port which
would somehow report back whether it was able to connect or not. A smaller-footprint option might be to have the load balancing application just try to connect to the ip/port using TCP and assume that if there's someone there accepting connections it is probably an App Server.

I decided not to go down this aproach and my software assumes that if the application server is ticked on then it is running. It becomes an administrative task to ensure this is the case. I really wanted to avoid any double conection strategy as this would delay startup time. The connection manager delays startup by about 2 seconds (less than that on a fast network) - including sending down the entire client.ini file.

As a further optimisation, it might be a good idea for the load balancing application to mark an appserver record as being dead (without actually deleting the record) to keep it from doing the "are you there" check every time a new client connection arrives and there's an appserver record with a
null/invalid myNode reference. It could then be manually reactivated when the downed App Server is brought back up, or perhaps automatically reactivated (every few client connections it retries appserver records marked dead, or something).

That is essentially what the check bo is I mentioned previously.

Another thing the connection manager software does is it keeps track of who has been logging (lat ip address, machine name, date, time, number of logins). That is handy for seeing how much the system is being logged into.
Another problem with the Jade limitation and the workaround we have is that Jade reuses oids for nodes when the database server is started. So if app server A has myNode refering to Node object 1234.2 - and the database server is restarted - then there is no guarantee that the same app servers "myNode" will be the same object. The problem with this is that the myNode reference is still valid, just not poiting to the app server it should. be. The way around this is to set all of the "myNode" references to null when the database server starts, and also when each application server starts.
Regards,
Allistar.