j/Conlang
1
1
Fork 0
Code Issues 5 Pull Requests Packages Projects 2 Releases Wiki Activity

Lexer: Rework Integer (and Float) literals to make them easier to parse #11

New Issue
Closed
opened 2023-10-17 04:22:14 +00:00 by j · 1 comment
j commented 2023-10-17 04:22:14 +00:00
Owner
Copy Link

Right now, the Lexer includes the base marker inside the string. However, it would likely be neater to match on the base marker and use that to determine whether the string is a valid literal at parse time.

#Requirements:

Grammar:

  • Update grammar with new production rules for each supported base

Tokens:

  • BaseHex := 0x: base-16 marker
  • BaseTen := 0d: base-10 marker (I find this one fun :P)
  • BaseOct := 0o: base-8 marker
  • BaseBin := 0b: base-2 marker

Lexer:

  • Remove base validation logic from Rule::integer
  • Add rules for each base

AST:

  • Preserve original number base within Int node

Parser:

  • Match (Base*, Integer), and convert from the appropriate base at parse-time
  • Implement digit aggregation
Right now, the Lexer includes the base marker inside the string. However, it would likely be neater to match on the base marker and use that to determine whether the string is a valid literal at parse time. #Requirements: ## Grammar: - [ ] Update grammar with new production rules for each supported base ## Tokens: - [ ] `BaseHex` := `0x`: base-16 marker - [ ] `BaseTen` := `0d`: base-10 marker (I find this one fun :P) - [ ] `BaseOct` := `0o`: base-8 marker - [ ] `BaseBin` := `0b`: base-2 marker ## Lexer: - [ ] Remove base validation logic from `Rule::integer` - [ ] Add rules for each base ## AST: - [ ] Preserve original number base within Int node ## Parser: - [ ] Match (Base*, Integer), and convert from the appropriate base at parse-time - [ ] Implement digit aggregation
j added the
bug
enhancement
 labels 2023-10-17 04:22:14 +00:00
j self-assigned this 2023-10-17 04:22:14 +00:00
j added this to the Basic Syntax project 2023-10-17 04:22:14 +00:00
j commented 2023-10-22 23:32:54 +00:00
Author
Owner
Copy Link

Issue has been obsoleted with the lexer rewrite

Issue has been obsoleted with the lexer rewrite
j closed this issue 2023-10-22 23:32:54 +00:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels   
bug

Something is not working

  
duplicate

This issue or pull request already exists

  
enhancement

New feature

  
help wanted

Need some help

  
invalid

Something is wrong

  
question

More information is needed

  
wontfix

This won't be fixed

No Label
bug
duplicate
enhancement
help wanted
invalid
question
wontfix
Milestone
Clear milestone
No items
No Milestone
Projects
Clear projects
No project
Basic Syntax
Assignees
Clear assignees
No Assignees
j
1 Participants
Notifications
Due Date
The due date is invalid or out of range. Please use the format 'yyyy-mm-dd'.

No due date set.

Dependencies

No dependencies set.

Reference: j/Conlang#11
No description provided.
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?