What is an Abstract syntax tree (AST)?
Abstract syntax trees are data structures widely used in compilers, due to their property of representing the structure of program code. An AST is usually the result of the syntax analysis phase of a compiler.
To solve these inconsistencies, not that far way, it started the confluence of these differences on the ESTree organisation, creating (finally) the first AST common specification. Now we are talking ;)
Now we have a small overview about how AST becomes and where is used, let see how to use for our own benefit and the future.
Let’s analyse a basic example:
The whole block represents a VariableDeclaration, it what it is. The VariableDeclarator is the “var” keyword, inside of this node we will found the Identifier which represents the “a”. The tree view visualize walk thought our small sentence in a syntax way.
In AST all is a Node, any element extend from it and search for a specific node we will need to loop the whole tree and search exactly what we need. For instance, if our objective were to replace “a” variable by “b” then we are going to need some tools to modify, travel through and remove or add our original code.
AST can be malleable, we can transform, replace and even remove nodes
The following parsers are fully compatible with the ESTree spec and probably the most two popular ones.
Acorn claims to be faster, smaller and prettiest than Esprima. Personally I haven’t tried with it, but it’s the core of ASTExplorer and it seems to be quite good, I haven’t worked with this one personally but I’ll leave here the reference.
Whether Esprima or acorn, both do the work, and they do it very well. Up to you to choose.
Once we have our code in AST mode, we can loop the whole tree an change whatever we want. Estraverse is a tool to iterate through the tree and modify, skip or remove nodes. In the following example we are going to replace and long name Identifier by a short one.
Takes an Esprima syntax tree and returns theanalyzed scopes. It allows us, eg: go to an specific scope and do some transformations, detect all variables in the scipe and their references to other parts of the code, detect child scopes, get function parameters, etc etc.
You can practice in this demo page.
A tool to locate any identifier in your code, this is very useful if we would want to rename any variable or function.
The current version (on npm) it doesn’t have this patch, so, I managed only to rename variables.
Esmangle is a small minifier, one of many ECMAScript tools, this one in particular is has a interesting set of parameters to change the behaviour.
Esprima has actually an object accessible to compare node by Syntax type, this alternative is more suitable for Unit Testing.
There is a AST explorer where you can play around using different configurations.
This article is about to recap some of the most popular/useful tools I found in the learning process about AST, probably there is more (feel free to comment if you have more suggestions or changes), personally I think understand AST is one of the most important steps to comprehend the web these days, topics like WebAssembly, V8, Blink always mention AST as a main intermediary for parsing.