Comparing the source multiple JADE bases

[Dr Danyo]

Hi folks,

Does anyone have any code handy that will allow me to compare multiple JADE bases to each other In particular we need to know if any of the bases are missing any code (not brothered about persistent data, but forms would also need to be considered).

We have over 140K methods so any code would have to run nice and quick (fastest solution wins).

As a starter just identifying bases that don't match would be fine, but extra points for identifying the specific differences.

cheers - Dr Danyo.

[allistar]

Assuming both databases are on the same version of JADE, simply run the two schema extracts taken from them through a visual diff tool (I use one called Kompare quite often). This will make it clear pretty quickly what the differences are. This doesn't work so well if the timestamp/user for features in the databases are different, even though the source is the same. If this is the case, then another simple option is to output a text file from the databases which are in this format: SchemaName::ClassName::methodName::md5sum where md5sum is the md5sum of the method source. Then run those two files through a diff program. You could take a similar approach with forms (but instead

[torrie]

We've just done this on a one client's system. Unfortunately the source was encrypted so we were only able to compare the dates / times. In the end, we just compared the persistent data structures (Classes, properties and references) and then did a full schema load (with appropriate scripts.)

I've also done schema file comparisons between systems as Allistar suggested. (I use TextPad for this) Some times I've stripped the userModifiedTimestamps fom the schema file to remove errors due to different time stamps etc. (Code below)

If you have a copy of the Jade 6.0 documentation, the Developers Reference has the Schema File Syntax. It hasn't been published by Jade since, but is basically the same.

Code: Select all

cleanSchemaFile(); // Cleans a schema file removing all the modified time stamps vars oInputFile : File; oOutputFile : File; oFileOpenDlg : CMDFileOpen; sLine : String; begin create oFileOpenDlg transient; oFileOpenDlg.filter := "All Schema Files (*.scm, *.mth, *.cls)|*.scm;*.cls;*.mth|Schema Files (*.scm)|*.scm|Class Files (*.cls)|*.cls|Method Files (*.mth)|*.mth|All Files (*.*)|*.*"; oFileOpenDlg.dialogTitle := "Schema File Clean"; oFileOpenDlg.fileMustExist := true; oFileOpenDlg.validate := true; if oFileOpenDlg.open() = 0 then create oInputFile transient; oInputFile.fileName := oFileOpenDlg.fileName; oInputFile.shareMode := File.Share_Read; oInputFile.kind := File.Kind_Unknown_Text; oInputFile.mode := File.Mode_Input; oInputFile.open(); create oOutputFile transient; oOutputFile.fileName := oFileOpenDlg.fileName.getFileStem & "_Cleaned." & oFileOpenDlg.fileName.extractFileExtension; oOutputFile.shareMode := File.Share_Exclusive; oOutputFile.kind := File.Kind_Unknown_Text; oOutputFile.mode := File.Mode_Output; oOutputFile.open; while not oInputFile.endOfFile do sLine := oInputFile.readLine; if sLine[1:22] = " setModifiedTimeStamp" then else oOutputFile.writeLine( sLine ); endif; if sLine = "typeSources" or sLine = "externalFunctionSources" then break; endif; endwhile; // Got to the message sources so just write all the method source. while not oInputFile.endOfFile do oOutputFile.writeString( oInputFile.readString( 2048 ) ); endwhile; endif; epilog delete oFileOpenDlg; delete oInputFile; delete oOutputFile; end;

[shadowfax]

A variant on the extract a schema and compare is to use JadeBatchExtract to extract in sections. See the section "Extracting Schemas in Sections" in the UserGuide.

For example the following extracts every schema in the system to the directory C:\temp2\userSchemas

jadclient path=e:\jadesystems\jade6111\system app=JadeBatchExtract schema=JadeSchema server=singleUser endJade Every C:\temp2\userSchemas\schemas.mul "<sections>"

Because each Type (Class) is extracted to a separate file and the methods are in the same order (this is not true in a scm file) it is easier to compare for differences.

KDiff3 allows you to recursively compare the folders quickly.

This is also a good way to get files than can be banked into subversion.

[Dr Danyo]

Thanks for all the responses guys, that gives me plenty to work with.

Shadowfax, are you using subversion for your JADE source?

ps KDiff3 looks particularly useful.

[shadowfax]

In answer to Dr Danyo.

Yes we are using subversion for our JADE source. Every night we extract it and bank it.

I is used to answer questions such as
1. Who changed this method last (and how)
2. When did this line appear in this Jade method (and who added it)
3. When did the type of this property change.
4. etc.

Note questions such as 2 and 3 are easy to answer using the subversion "blame" feature

[BeeJay]

Presumably the person who gets the blame is the last person to compile the method on the day it was changed?

For example, suppose the following sequence of events occurs on the one day:

1. User A changes the method by adding two lines of code
2. User B also changes the method by updating a different line of code
3. The nightly extract runs and both lots of changes are now extracted & loaded into subversion

Does User B now get assigned the blame for both lots of changes, or are you somehow also using patch versioning and the method history to track both of these changes independently?

Cheers,
BeeJay.

[Dr Danyo]

Hi Beejay, For that level of detail they would need a commit strategy, but just having the daily source under source control must be giving loads of benefits.

Shadowfax, I presume that you are automating the extract and commit, any chance we could see your scripts etc

[shadowfax]

Here is our script. I have edited this so you could not see who the system belongs to
There may be some artifacts that result from this so be careful.
However the gist of the bat file should be ok.

Code: Select all

set HomeDrive=%1 if '%HomeDrive%' == '' goto usage set Project=%2 if '%Project%' == '' goto usage goto params :usage echo usage: extractSchema drive project goto end :params :start set DatabaseRoot=D:\Jade_Development if not exist %DatabaseRoot%\nul set DatabaseRoot=R:\Jade_Development set BuildType=ExtractSchema set Root=daily set BinDir=%DatabaseRoot%\%Project%\bin set DatabasePath=%DatabaseRoot%\%Project%\system set DistDatabasePath=%DatabaseRoot%\%Project%\systemd rem === Set up configuration variables === set Volume=%HomeDrive%: set User=build set TimeServer=\\machinename set Subversion="C:\Program Files (x86)\Subversion\bin\svn.exe" if not exist %Subversion% set Subversion="C:\Program Files\Subversion\bin\svn.exe" set Blat=%Volume%\%Root%\%Directory%\%Project%\overnight\Blat rem note: comma separate multiple address,*don't* include any spaces set MailTo=-t xxx@yyyy.com rem === Set up convenience variables === set SrcDir=%Volume%\%Root%\%Project% set SchemaDir=%SrcDir%\schema set LockFile=%Volume%\%Root%\%Project%_%BuildType%.lock set LogFile=%Volume%\%Root%\%BuildType%_%Project%.log rem === Change to the build directory === md "%Volume%\%Root%" pushd "%Volume%\%Root%" rem === Check for existing build session === if exist "%LockFile%" goto locked rem === Create lock file === net time %TimeServer% >> %LockFile% rem === Cleanup old logs === del "%LogFile%" rem === Log start timestamp == echo. > %LogFile% echo *** Extract Schema started *** >> %LogFile% net time %TimeServer% >> %LogFile% 2>&1 echo. >> %LogFile% echo. >> %LogFile% echo *** Deleting schema files *** >> %LogFile% del "%SchemaDir%\User1Schema.*" /q/f >> %LogFile% 2>&1 del "%SchemaDir%\User2Schema.*" /q/f >> %LogFile% 2>&1 "%BinDir%\jadclient.exe" displayTimes=true ini="%DatabasePath%\jade.ini" path="%DatabasePath%" server=multiUser schema=JadeSchema app=JadeBatchExtract endJade Every "%SchemaDir%\schemas.mul " "<sections>" echo. >> %LogFile% echo *** Checking in schema files *** >> %LogFile% echo. >> %LogFile% echo *** Add any new and Delete removed .cls files *** >> %LogFile% echo. >> %LogFile% pushd "%SrcDir%\schema" pushd "User1Schema" if exist deletedFiles %Subversion% delete --targets deletedFiles >> %LogFile% 2>&1 if exist addedFiles %Subversion% add --targets addedFiles >> %LogFile% 2>&1 popd pushd "User2Schema" if exist deletedFiles %Subversion% delete --targets deletedFiles >> %LogFile% 2>&1 if exist addedFiles %Subversion% add --targets addedFiles >> %LogFile% 2>&1 popd popd echo. >> %LogFile% %Subversion% --non-interactive commit -F"%SrcDir%\schema\PatchHistory_newlines.txt" "%SrcDir%\schema" >> %LogFile% 2>&1 echo. >> %LogFile% echo. >> %LogFile% %Blat% %LogMsgFile2% %MailTo% net time %TimeServer% >> %LogFile% 2>&1 echo *** Extract Schema Succeeded *** >> %LogFile% echo. >> %LogFile% goto unlock :locked echo *** Extract Schema is already in progress *** goto end :unlock del "%LockFile%" :end popd

[Dr Danyo]

Perfect, thank you.