dynamic methods

[ConvertFromOldNGs]

by John Munro >> Tue, 24 Jun 2008 21:41:06 GMT

I'm working on a feature that is (to some extent) similar to features of dynamic languages - I need to be able to add properties and methods to a class dynamically at runtime.

Between JadeDynamicObject and process.executeTransientMethod I can fake dynamic properties - the JadeDynamicObject can store dynamic property values, and to get the compiler to cooperate I insert code into the method just before it's executed.

For example if you had an Integer dynamic property called "count" with a value of 10 and the developer typed in the method

count := count + 1;

the compiler would complain because there isn't a static property called "count". So this code is inserted before the method:

vars
count : Integer;begin

count := jdo.getPropertyValue("count").Integer;

and this code is appended to the method:

epilog
jdo.setPropertyValue("count", count);
end;

and everything compiles and the dynamic property can be read and updated. Obviously it gets more complicated if the developer types in a method with a vars or epilog section, but it still works. An alternate strategy would be to pass the dynamic properties as io parameters to the developer's method:

temp(count : Integer io);

The problem I have is in faking dynamic methods. The reason the above strategies work is that properties, variables and io parameters act in an interchangeable manner, but methods don't really work interchangeably with anything else.

The only thing I can think of is introducing a non-Jade language element and replacing it with a sendMsg call just before execution. For example if you had a dynamic method called "myMeth" which returned an Integer, and the developer wrote this

write !myMeth + 1;

the method would be modified to this before execution

write process.executeTransientMethod(myDynamicMethods["myMeth"], self).Integer + 1;

and it would compile and run the dynamic method. It introduces some compiler-style work to make sure that the dynamic method exists, sorting out its parameters and return type etc, but it should work.

The thing I don't like about this solution is that it messes up the
code offsets in compiler errors and exceptions because what Jade is executing is no longer exactly what the developer wrote. Fixing that involves tracking all of the places where substitutions were made in order to translate the code offset back to a position in the original code...

Any thoughts? Other than "don't bother"?

John

[ConvertFromOldNGs]

by Allistar >> Wed, 25 Jun 2008 0:23:46 GMT

Interesting problem. Had you considered creating a .scm file of the method/property and loading it into the system at runtime? You'd have reorg/transition issue to deal with, but if the properties and methods are not added during normal system operation (i.e. they are only "deployed" during downtime), then this could be a solution.

I'd recommend fulling parsing the Jade code and getting some sort of AST (abstract syntax tree) of it to allow you to more easily securely modify the code. In a past project for a client I developed a full Jade code parser which generates an AST of Jade code. (This was developed in Jade and the AST is represented by Jade objects). You can then modify the AST (or run complicated queries over it) and get it to spit you back some source, which can then be compiled in a compiled. I'm happy to provide help with this - the most complicated part of the parser is the expression parser, which uses a modified shift-reduce style algorithm. Once you have an AST you have a lot of control over the source code for a method. This method also allows you to build in new language constructs pretty easily, which then get translated before giving you back some method source that Jade is happy with.

This was when you enter this code:

write myMeth(myOtherMeth("abcd"), "blah") + 2;

even though the Jade complier would complain, your parser would not as it would see that "myMeth" and "myOtherMeth" are in fact dynamic methods. It would then spit back Jade compliant source similar to this:

write executeDynamicMethod(executeDynamicMethod("abcd"), "blah").Integer + 2;

Couple this with the Jade editor control in Jade 6 and you have effectively extended the Jade language and simulated the normal Jade IDE method editor.

Regards,
Allistar.
--
A.

[ConvertFromOldNGs]

by John Munro >> Wed, 25 Jun 2008 16:47:18 GMT

Interesting problem. Had you considered creating a .scm file of the method/property and loading it into the system at runtime? You'd have reorg/transition issue to deal with, but if the properties and methods are not added during normal system operation (i.e. they are only "deployed" during downtime), then this could be a solution.

It needs to be a lot more immediate than that because properties and methods are added at runtime. Also this would require use of a command line running jadloadb - I don't think there's a way to load schemas code.

I'd recommend fulling parsing the Jade code and getting some sort of AST (abstract syntax tree) of it to allow you to more easily securely modify the code.

Wow, well that would certainly solve the problem but I hesitate to duplicate the Jade compiler. It seems a bit overkill compared to just searching for tokens starting with "!" that aren't inside quotes...

Also it doesn't solve the code offset problem in exceptions - it would give the code offset of the modified code, not the original.

John

[ConvertFromOldNGs]

by Allistar >> Wed, 25 Jun 2008 20:32:37 GMT

Interesting problem. Had you considered creating a .scm file of the method/property and loading it into the system at runtime? You'd have reorg/transition issue to deal with, but if the properties and methods are not added during normal system operation (i.e. they are only "deployed" during downtime), then this could be a solution.

It needs to be a lot more immediate than that because properties and methods are added at runtime. Also this would require use of a command line running jadloadb - I don't think there's a way to load schemas code.

jadloadb does work in multiuser mode, but you'd face production mode and versioning issues, so as you suggest it's not an appropriate solution in your case.

I'd recommend fulling parsing the Jade code and getting some sort of AST (abstract syntax tree) of it to allow you to more easily securely modify the code.

Wow, well that would certainly solve the problem but I hesitate to duplicate the Jade compiler. It seems a bit overkill compared to just searching for tokens starting with "!" that aren't inside quotes...

.....or comments, or quotes inside comments or comments inside quotes. Not overly difficult to handle though. Having your own AST of the method would allow you to be more flexible in how "dynamic" methods are called (i.e. you wouldn't need to prefix the with a "!"). Still some work involved, as you say. Took me about a solid week to get a full parser of Jade syntax working - the end result gives a powerful tool for analysing (aka Lint) and modifying code (cleaning up comments etc).

Also it doesn't solve the code offset problem in exceptions - it would give the code offset of the modified code, not the original.

Yes - that would be more difficult to overcome. If the method call took the same number of parameters I'd expect the byte code offsets to be the same, but each method call would have an additional parameter (namely the name of the method to call), so this wouldn't work. You could possibly make some assumptions about how an additional parameter into a method will effect byte positions, but I doubt it would be foolproof.

Allistar.
--
A.