John b5abd2bff1 Lexer rewrite: ... - Scan the input string *linearly*, without backtracking - Peek at most one character (unicode code-point) ahead - Store data (unescaped string literals and chars, identifiers, integers, floats) inside Token - This unfortunately makes tokens non-Copy - Refactor Parser to accommodate these changes - On the bright side, Parser no longer needs a reference to the text! - Write a new set of lexer tests - TODO: write a new set of token tests using tokendata Every day, we get closer to parsing `dummy.cl`!		2023-10-22 18:28:20 -05:00
..
examples	Lexer rewrite:	2023-10-22 18:28:20 -05:00
src	Lexer rewrite:	2023-10-22 18:28:20 -05:00
.gitignore	Initial Commit	2023-09-21 15:41:12 -05:00
Cargo.toml	constr: Misuse iterators to parse tokens	2023-10-20 15:33:02 -05:00