27 March 2000 Release 2.22 Notes for New Users of PCCTS Version 1.33MR22
47
#7. How to pass whitespace through
DLG
for pretty-printers
See files in notes/ws/*
This demonstrates how to combine several separate
DLG
tokens (whitespace for this example) into a single
ANTLR
token. It also demonstrates careful processing of tab characters to generate accurate column information even
within comments or other constructs which use more().
#8. How to prepend a newline to the
DLG
InputStream via derivation from
DLG
Lexer
See files in notes/prependnl/*
This demonstrates how to derive from
DLG
Lexer in order to replace a user-supplied
DLG
InputStream routine with
another which can perform additional processing on the input character stream before the characters are passed to
DLG
. In this case a single newline is prepended to the input. This is done to make it easier to treat the first non-
blank token on a line as a special case, even when it appears on the very first line of the input file.
#9. How to maintain a stack of #lexclass modes
See files in notes/modestack/*
This is based on routines written by David Seidel which allow the user to pass a a routine to be executed when the
mode is popped from the stack.
#10. When you want to change the token type just before passing the token to the parser
See files in notes/predbuf/*
This program shows how to reassign token codes to tokens at the time they are fetched by the parser by deriving
from class
ANTLR
TokenBuffer and changing the behavior of getToken().
#11. Rewriting a grammar to remove left recursion and perform left factoring
The original grammar:
command := SET var BECOMES expr
| SET var BECOMES QUOTE QUOTE
| SET var BECOMES QUOTE expr QUOT
| SET var BECOMES QUOTE command QUOTE
expr := QUOTE anyCharButQuote QUOTE
| expr AddOp expr
| expr MulOp expr
The repetition of "SET var BECOMES" for command would require k=4 to get to the interesting part. The first step
is to left-factor "command":
command := SET var BECOMES
( expr
| QUOTE QUOTE
| QUOTE expr QUOTE
| QUOTE command QUOTE
)
The definition of expr uses left recursion which must be eliminated when using
ANTLR
:
op := AddOp
| MulOp
expr := QUOTE anyCharButQuote QUOTE (op expr)*
Since expr begins with QUOTE and all the alternatives of the sub-rule of command also start with QUOTE. This
too can be left-factored:
command := SET var BECOMES QUOTE
( expr_suffix
| QUOTE
| expr QUOTE
| command QUOTE
)
expr_suffix := anyCharButQuote QUOTE (op expr)*
expr := QUOTE expr_suffix