Faster string concatenation for large string

[BeeJay]

It the JUG 2017 conference one of the slides I presented was showing how I significantly improved the building of a large resultant string, reducing the time to build the string from 30s to under 100ms, on average.

Attached is a sample schema that shows the timing difference for building the same string using standard string concatenation and then using the more optimised version. For 100,000 executions, the default, it takes between 7s and 8s using standard concatenation and < 100ms using the optimised version. Your mileage from this may be vary, particularly if the way you obtain a given string requires accessing multiple objects, but I thought it was still worthwhile sharing the sample code for the benefit of those who were unable to attend the JUG 2017 conference or those who weren't able to copy down the sample code quickly enough while I had it on-screen.

Disclaimer: I believe that this sample code is accurate and reliable and has been prepared with care but I give no guarantee that the sample code will be free of defects or errors. No responsibility or liability, financial or otherwise, is accepted for any consequences arising out of the use of this sample code including loss of profits, indirect, special or consequential losses or damages.

Cheers,
BeeJay.

[Stokes]

This is very interesting to me as we build up large HTML strings.

Have you been using a common method to do this for you or do you just implement in each method as required? I'm hoping that we could use this by rolling out a primitive method for example "String::concat(stringToAdd)", I understand that calling a method is inefficient but with the speed gains you are showing it could be worthwhile.

The way it is currently working would add a lot of repeated code to a lot of methods.

This could be a stupid question but couldn't Jade do something this clever behind the scenes?

[allistar]

Hi BeeJay,
Your solution is very interesting. The same strategy applies to Binary properties too. I would think this could be implemented "under the hood" in Jade though you'd need to specify that you were done with the concatenation so it could then remove the excess padding from the end. If we could subclass Types we could have a "BufferedString" type which has the behaviour you've shown built in. For now we could implement it using a transiently instantiated class which holds the string and the current cursor position. The additional method calls could be expensive though. In previous projects I've worked on we moved away from high volume methods on primitives because of the expense of calling them, it was very noticeable in the profiler. It would be great to have an "inline" method modifier; I imagine this would simply be compiler candy in Jade.

Cheers,
Allistar.

[BeeJay]

Stokes - I had to do it in-line with local variables to get the speed improvements I desired, so yes if I need to use it somewhere else I have to replicate the same code. ie: Using methods on string primitives didn't give the same performance gains and for smaller resulting strings actually performed worse than straight concatenation.

Allistar - If you put the string you're concatenating onto an object property, then you're crossing the jom/interpreter boundary and the performance degrades significantly. That's why in the sample code I presented at the JUG I had a single assignment from the local variable to the self.someStringProperty at the end of the method in question.

It would be nice if there was some syntatical sugar we could add in the Jade language to make this more 'automatic' to avoid having to repeat the code, and I have opened an NFS suggesting it would be nice if there could be an extension to the Jade language to support a more optimised way to perform large string concatenation operations.

Cheers,
BeeJay.