doughlang: Preserve errors through entire pipeline

lexer:
- Un-stringify errors
- Reserve more words
- Doc the comments

parser:
- MASSIVE changes to peek, peek_if, next_if, consume_if=>expect.
- Keep track of when EOF is allowable
- TKind is stupidly cheap with >100 niches, so we can fit like 4 of them in a single ParseError lmao
- TODO: make sure EOF/UnexpectedEOF propagation is correct. It seems... Kinda Not correct.
- Add meta-expressions
This commit is contained in:
2025-10-17 06:25:11 -04:00
parent c8f1f082c4
commit 6368e68941
6 changed files with 543 additions and 351 deletions

View File

@@ -9,6 +9,12 @@ pub struct Token {
pub span: Span,
}
impl Token {
pub fn kind(&self) -> TKind {
self.kind
}
}
#[derive(Clone, Debug)]
pub enum Lexeme {
String(String),
@@ -55,7 +61,9 @@ impl std::fmt::Display for Lexeme {
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
pub enum TKind {
Comment,
Comment, // Line or block comment
Doc, // Doc comment
And,
As,
Break,
@@ -67,6 +75,7 @@ pub enum TKind {
Fn,
For,
If,
Impl,
In,
Let,
Loop,
@@ -76,6 +85,7 @@ pub enum TKind {
Or,
Public,
Return,
Static,
Struct,
True,
While,