j/Conlang
1
1
Fork 0
Code Issues 5 Pull Requests Packages Projects 2 Releases Wiki Activity

Lexer: Rework Integer (and Float) literals to make them easier to parse #11

New Issue
Closed
opened 2023-10-17 04:22:14 +00:00 by j · 1 comment
j commented 2023-10-17 04:22:14 +00:00
Owner
Copy Link

Right now, the Lexer includes the base marker inside the string. However, it would likely be neater to match on the base marker and use that to determine whether the string is a valid literal at parse time.

#Requirements:

Grammar:

  • Update grammar with new production rules for each supported base

Tokens:

  • BaseHex := 0x: base-16 marker
  • BaseTen := 0d: base-10 marker (I find this one fun :P)
  • BaseOct := 0o: base-8 marker
  • BaseBin := 0b: base-2 marker

Lexer:

  • Remove base validation logic from Rule::integer
  • Add rules for each base

AST:

  • Preserve original number base within Int node

Parser:

  • Match (Base*, Integer), and convert from the appropriate base at parse-time
  • Implement digit aggregation
Right now, the Lexer includes the base marker inside the string. However, it would likely be neater to match on the base marker and use that to determine whether the string is a valid literal at parse time. #Requirements: ## Grammar: - [ ] Update grammar with new production rules for each supported base ## Tokens: - [ ] `BaseHex` := `0x`: base-16 marker - [ ] `BaseTen` := `0d`: base-10 marker (I find this one fun :P) - [ ] `BaseOct` := `0o`: base-8 marker - [ ] `BaseBin` := `0b`: base-2 marker ## Lexer: - [ ] Remove base validation logic from `Rule::integer` - [ ] Add rules for each base ## AST: - [ ] Preserve original number base within Int node ## Parser: - [ ] Match (Base*, Integer), and convert from the appropriate base at parse-time - [ ] Implement digit aggregation
j added the
bug
enhancement
 labels 2023-10-17 04:22:14 +00:00
j self-assigned this 2023-10-17 04:22:14 +00:00
j added this to the Basic Syntax project 2023-10-17 04:22:14 +00:00
j commented 2023-10-22 23:32:54 +00:00
Author
Owner
Copy Link

Issue has been obsoleted with the lexer rewrite

Issue has been obsoleted with the lexer rewrite
j closed this issue 2023-10-22 23:32:54 +00:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
bug
Something is not working
duplicate
This issue or pull request already exists
enhancement
New feature
help wanted
Need some help
invalid
Something is wrong
question
More information is needed
wontfix
This won't be fixed
No Label
bug
enhancement
Milestone
No items
No Milestone
Projects
Clear projects
No project Basic Syntax
Assignees
Clear assignees
No Assignees j
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: j/Conlang#11
No description provided.
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?