This removes usage of regexps from the parsing phase and splits the
program into three sub-packages: parser, database, and emitters.
The parser now generates an AST according to the formal grammar
described in the README:
https://github.com/google/filament/blob/pr/beamsplitter_rewrite2/tools/beamsplitter/README.md#grammar
This allows for nice readable error messages. More importantly, it
permits the C++ syntax to be less restrictive and paves the way for
possible expansion of the tool beyond `Options.h`.
I looked at the C++ AST generated by clang but it is huge and unwieldy.
For our purposes this simplified AST is much easier to work with.
The new lexer is inspired by the following Rob Pike talk.
- https://www.youtube.com/watch?v=HxaD_trXwRE
Beamsplitter does not use the state machine described in the above
prezo, but it does use a Go channel for separating the parser from the
lexer. In our case, the lexer is actually a recursive descent parser
with simple lookahead functionality. This made it easy for the "real"
parser to create an ergonomic coarse-grained AST.
This is a big change but it is a no-op in terms of the generated code.
c82ae6a3a7