Taeluf.com

Php Lexer

A declarative lexer seamlessly hooking into php functions, building ASTs for multiple languages.

Development Status / Roadmap

This branch (v0.8) is highly tested for the PhpGrammar. The PhpGrammar implements most php features, but is not totally complete. No php 8.0+ features have been added to lexing. The documentation is not great (it has not been updated for v0.8, but the generate an AST example should still be accurate)

Install

For development, it depends upon @easy_link(gitlab, taeluf/php/php-tests), which is installed via composer and @easy_link(gitlab, taeluf/php/CodeScrawl), which is NOT installed via composer because of circular dependency sometimes causing havoc. @template(php/composer_install,taeluf/lexer)

Generate an AST

See @see_file(doc/Examples.md) for more examples Example:

@import(Test.Doc.LexFile)

See @see_file(test/input/php/lex/SampleClass.php) for the input file and @see_file(test/output/php/tree/SampleClass.js) for the output $tree.

Status of Grammars

  • Php: Early implementation that catches most class information (in a lazy form) but may have bugs
  • Docblock: Currently handles /* style, cleans up indentation, removes leading * from each line, and processes simple attributes (start a line with * @something description).
    • Coming soon (maybe): Processsing of @‌method_attributes(arg1,arg2)
  • Bash: Coming soon, but will only catch function declarations & their docblocks.
    • the docblocks start with ## and each subsequent line must start with whitespace then # or just #.
    • I'm writing it so i can document @easy_link(tlf, git-bent)
  • Javascript: Coming soon, but will only catch docblocks, classes, methods, static functions, and mayyybee properties on classes.
    • I'm writing it so i can document @easy_link(tlf, js-autowire)

Write a Grammar

A Grammar is an array declaration of directives that define instructions. Those instructions may call built-in commands or may explicitly call methods on a grammar, the lexer, the token, or the head ast.

Writing a grammar is very involved, so please see @see_file(doc/GrammarWriting.md) for details.

Warning

  • Sometimes when you run the lexer, there will be echod output. Use output buffering if you want to stop this.
  • During onLexerEnd(...), Docblock does $ast->add('docblock', $lexer->previous('docblock')) IF there's a previous docblock set.

Contribute

  • Need features? Check out the Status.md document and see what needs to be done. Open up an issue if you're working on something, so we don't double efforts.