Lexer2 Docs | Parsing via a custom programming language

Lexer2 introduces a new programming language built to parse code. The code parser itself is separate from the Lexer. Code you write will be executed by Tlf\Lexer2\Program. Lexer loops over input, moving the buffer forward one char on each loop, & calls $program->execute().

The Program does not know about the lexer or the buffer, or any specific implementation related to Lexing.

The Program is generic, and holds a list of objects, a list of Standard libraries, and a list of directives. (Your programming language code is grouped into "Directives" which are similar to functions)

There is an StdLib, which the Lexer adds to the Program.

Your code will be written in .cta files (its a custom extension that may change), and you'll also write an StdLib for your language, if built-in StdLib is not enough. For example, the PHP parser (under development), currently has four .cta files and WILL have its own standard lib of functions (implemented in PHP) that .cta code can call for complex operations that are specific to parsing PHP code.

There are also several objects that the Lexer adds to the Program. Currently, these are, an ASTLib, a BufferLib, the StdLib, and the Lexer itself. Each of these Libs define methods that are called by your .cta code. Those Libs then work with other objects, so that your .cta code is not working directly with objects, but instead with a clean interface designed for .cta code.

This means, the BufferLib has a method padLeft(int $num_spaces_to_add). Your .cta code can call buffer.padLeft 6 to add 6 spaces to the left-hand side of the buffer. The BufferLib will then call $this->buffer->prepend(str_repeat(" ", $num_spaces));. So the BufferLib is not the Buffer itself.

Instead BufferLib defines an API that your .cta code can call upon to interact with the actual Buffer.

TODO: Finish this documentation.

Notes:

• `

// declared command can be:
// - stdlib_method     # Note: You don't write stdlib_. Just the method name that is available in the stdlib.
// - object.method
// - object.property.method    # Can we go deeper? object.property.property.method ?
// - (MAYBE) object.!method.submethod     # Where object.!method returns an object & submethod is a method on that returned object.
// - :directive
// - namespace:directive
// - localvar= !object.method   # set a local variable
// - object.method $localvar    # pass a local variable to a function