Six compiler is kinda slow.

[insidewhy]

My project "trek.js" is less than 2000 lines of code including the test framework.

Compiling it with six takes 2 whole seconds! 1 second per 1000 lines of code strikes me as slow.

I wonder what the cause of the slowness is...

Do you think maybe iterating the AST once per hook might be the cause? I was thinking maybe we could combine some of the hooks into a single pass over the AST?

[matthewrobb]

That my be the culprit although honestly it's a big improvement over the simple traverse being used in some other scripts. I want to focus on getting the core features to a functional state and simultaneously begin working on optimization as documentation.

[insidewhy]

Okay I applied an optimisation to reduce the number of traversals. Only sped up compilation by 20% though.

I thought maybe the node "children" property cloning the children every time might also be a drain and unnecessary... it's called frequently. I'll try changing it to return a reference.

[insidewhy]

Okay changing children to return a reference rather than a copy shaves off 0.05s. Doesn't cause any failures though.

[insidewhy]

I modified the binary to time the length of various phases:

28909: 0.040097868 - start
28909: 0.004540206 - begin rewrite
28909: 0.006303534 - create tree
28909: 0.009374042 - run hooks
28909: 0.008590053 - visit by type
28909: 0.013807254 - compiling modified code
28909: 0.000489437 - begin rewrite
28909: 0.055509359 - create tree
28909: 0.087714069 - run hooks
28909: 0.023649324 - visit by type
28909: 0.081594205 - compiling modified code
28909: 0.000266326 - begin rewrite
28909: 0.011619717 - create tree
28909: 0.014203699 - run hooks
28909: 0.025047763 - visit by type
28909: 0.040753196 - compiling modified code
28909: 0.000146625 - begin rewrite
28909: 0.027083906 - create tree
28909: 0.036599015 - run hooks
28909: 0.002911723 - visit by type
28909: 0.076925873 - compiling modified code
28909: 0.000307863 - begin rewrite
28909: 0.017398158 - create tree
28909: 0.023695877 - run hooks
28909: 0.017903667 - visit by type
28909: 0.029707038 - compiling modified code
28909: 0.000263719 - begin rewrite
28909: 0.201319319 - create tree
28909: 0.378699939 - run hooks
28909: 0.042116888 - visit by type
28909: 0.262141649 - compiling modified code
28909: 0.000197887 - begin rewrite
28909: 0.068399384 - create tree
28909: 0.157738098 - run hooks
28909: 0.007355023 - visit by type
28909: 0.076336350 - compiling modified code
28909: 0.000180994 - begin rewrite
28909: 0.012841982 - create tree
28909: 0.013851440 - run hooks
28909: 0.013189687 - visit by type
28909: 0.015398397 - compiling modified code
28909: 0.000165249 - begin rewrite
28909: 0.015522816 - create tree
28909: 0.024331442 - run hooks
28909: 0.001414304 - visit by type
28909: 0.031282832 - compiling modified code
28909: 0.000157675 - begin rewrite
28909: 0.002357620 - create tree
28909: 0.003547153 - run hooks
28909: 0.001859059 - visit by type
28909: 0.004885407 - compiling modified code
28909: 0.000116346 - begin rewrite
28909: 0.026229859 - create tree
28909: 0.049036515 - run hooks
28909: 0.014876754 - visit by type
28909: 0.084216974 - compiling modified code
28909: 0.001936387 - begin rewrite
28909: 0.037041274 - create tree
28909: 0.053832996 - run hooks
28909: 0.002468860 - visit by type
28909: 0.095954085 - compiling modified code
28909: 0.000201192 - begin rewrite
28909: 0.027678448 - create tree
28909: 0.026636120 - run hooks
28909: 0.001823856 - visit by type
28909: 0.140050135 - compiling modified code
28909: 0.000159847 - begin rewrite
28909: 0.010110160 - create tree
28909: 0.015718565 - run hooks
28909: 0.001062539 - visit by type
28909: 0.029931087 - compiling modified code
28909: 0.000152829 - begin rewrite
28909: 0.036135665 - create tree
28909: 0.068628303 - run hooks
28909: 0.006813074 - visit by type
28909: 0.166548196 - compiling modified code
28909: 0.000239399 - begin rewrite
28909: 0.024394362 - create tree
28909: 0.041201957 - run hooks
28909: 0.009335651 - visit by type
28909: 0.076419265 - compiling modified code
28909: 0.000182075 - begin rewrite
28909: 0.066318999 - create tree
28909: 0.132191987 - run hooks
28909: 0.005477510 - visit by type
28909: 0.308207505 - compiling modified code
28909: 0.000815381 - begin rewrite
28909: 0.009288184 - create tree
28909: 0.014197067 - run hooks
28909: 0.008615710 - visit by type
28909: 0.035863950 - compiling modified code
28909: 0.000933523 - begin rewrite
28909: 0.036304633 - create tree
28909: 0.105242725 - run hooks
28909: 0.005756485 - visit by type
28909: 0.164104687 - compiling modified code
28909: 0.000294268 - begin rewrite
28909: 0.087853196 - create tree
28909: 0.194935942 - run hooks
28909: 0.039450691 - visit by type
28909: 0.303947919 - compiling modified code```

[insidewhy]

So it seems it's the code compiliation step taking the most of the time. Maybe all the string concatenation? AST modification rather than reforming the tree out of source code could really help performance!

[matthewrobb]

I am actively working on rewriting a lot of stuff with the tree building/visiting/traversing. I imagine we will see some performance improvements when all is said and done. My last day with my current employer is tomorrow, movers come Saturday, then I drive 12 hours, then 3 days later I get on a plane in Michigan to fly to Seattle. My first day at the new job is Wednesday.

BUSY but at the same time once I am in Seattle I will have plenty of time to focus on this as my family won't be with me immediately. My only job outside of work will be finding us a house and working on rewriting the way all this works.

It's one of my primary goals to support using escodegen to do AST transforms and only do one operation to output a string.

I already have progress on porting class, quasi, and egal out of the doT.js templating system and removing that dependency as well.

I also am thinking about changing up the hook system to allow hook modules to pass out a single function which will get the entire tree passed in. Doing it this way will allow smarter caching of nodes. For example:

Object.define(exports, {

  ".ClassDeclaration": node => {
  },

  ".ClassExpression": node => {
  },

  ".CallExpression > .MemberExpression > .Identifier[name='super']": node => {
  },

  ".CallExpression > callee[name='super']": node => {
  }

})

Could optimize this by doing one query for all ".ClassDeclaration, .ClassExpression" and then doing all other queries using those cached results as the root.

There are more/better examples just this one of the top of my head.

[matthewrobb]

OH! - Also you may want to look in to the performance implications of using getters. If we're returning a reference then I don't see the harm in storing that reference on the object it's self. Accessors are really slow and they're being touched A LOT in the code atm.

[matthewrobb]

One more thing, can you integrate the benchmark setup you are using into the project?

[insidewhy]

If you check my branch "timing" I put the code there. Can open a pull request if you want.

[insidewhy]

I think.. I can apply all the hooks in a single traversal... by adjusting the compilation phase and ammending the visitor method slightly.

[insidewhy]

So for selectors with common parents.. it would compile them together and enable them then remove them from the type hash during traversal. I'll show you what I mean on Tuesday, going to see my family this weekend for fireworks night and then Godspeed you black emperor on monday night :)

[matthewrobb]

I tried to pull your branch but it's missing ./timer module

[matthewrobb]

Ah, the compiled file isn't in the repo

[insidewhy]

Oops, added it.

[matthewrobb]

Try pulling this and see what results you get as far as speed improvements, My results with your timing branch seemed pretty positive: https://github.com/matthewrobb/six/tree/optimize