Commit Graph

213 Commits

Author SHA1 Message Date
e5cbd806c4 parser: refactor expectVarDeclExprStatement to match upstream
Restructure expectVarDeclExprStatement to match upstream Parse.zig's
approach: check for '=' first, then handle var decl init vs expression
statement separately. This fixes parsing of var decls with container
types (e.g., `const x: struct {} = val`), where the '}' of the type
was incorrectly treated as a block-terminated expression.

Also make container member parsing strict (longjmp on unexpected tokens
instead of recovery), and add for/while/labeled-block handling in
parseTypeExpr for function return types.

376/381 tests pass.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-02-11 14:40:31 +00:00
fdefdc98c2 parser: sync tests with upstream, fix tokenizer and parser
Sync parser_test.zig test section with upstream, adding ~40 new tests
(testError, testCanonical, testTransform). Remove extra blank lines
between tests to match upstream formatting.

Fix tokenizer keyword lookup bug: getKeyword() returned TOKEN_INVALID
when input was longer than a keyword prefix (e.g., "orelse" matched
"or" prefix then bailed out instead of continuing to find "orelse").

Fix parser to handle if/for/while expressions in type position (e.g.,
function return types like `fn foo() if (cond) i32 else void`). Add
labeled block support in parsePrimaryTypeExpr. Replace assert for
chained comparison operators with longjmp error.

365/381 tests pass. Remaining 16 failures are parser limitations for
specific syntax patterns and error recovery.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-02-11 14:40:31 +00:00
f5f54fcbe8 parser: propagate errors via setjmp/longjmp instead of exit(1)
Replace 32 parse-error exit(1) calls with longjmp to allow callers to
detect and handle parse failures. The OOM exit(1) in
astNodeListEnsureCapacity is kept as-is.

Add has_error flag to Ast, wrap parseRoot() with setjmp in astParse(),
and update test infrastructure to use the C parser for testError tests.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-02-11 14:40:31 +00:00
bda3329eee parser: refactor to match upstream Parse.zig structure
Extract shared helpers and fix error handling to align with upstream:

- Replace 7 assert() calls that crash on valid input with fprintf+exit
- Extract parsePtrModifiers() and makePtrTypeNode() to deduplicate
  pointer modifier parsing from 4 inline copies into 1 shared function
- Extract parseBlockExpr() and parseWhileContinueExpr() helpers
- Move comptime wrapping into expectVarDeclExprStatement() via
  comptime_token parameter
- Extract finishAssignExpr(), parseSwitchItem(), parseSwitchProng()

Net effect: 3233 → 3106 lines. All 298+ parser tests pass.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-02-11 14:40:30 +00:00
7193385f94 parser: reorder tests, fix check_test_order.py for new file layout
Update check_test_order.py to handle header/footer split correctly
when infrastructure code is at both start and end of file.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-02-11 14:40:30 +00:00
174a275c76 parser: port canonicalize and cast builtin tests
Port tests:
- "canonicalize symbols (simple)"
- "canonicalize symbols (character escapes)"
- "canonicalize symbols (asm)"
- "canonicalize cast builtins"
- "do not canonicalize invalid cast builtins"

Update test bodies to match upstream:
- "comptime block in container"
- "comment after empty comment"
- "comment after params"
- "decimal float literals with underscore separators"
- "container doc comments"

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-02-11 14:40:29 +00:00
801dfc6c6e parser: update test bodies to match upstream verbatim
Update test content to match upstream exactly:
- "comptime block in container"
- "comment after empty comment"
- "comment after params"
- "decimal float literals with underscore separators"
- "container doc comments"

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-02-11 14:40:29 +00:00
7a414e8731 parser: update defer test body to match upstream
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-02-11 14:40:29 +00:00
02621037a2 parser: fix duplicate while test content, match upstream verbatim
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-02-11 14:40:28 +00:00
83da26c183 parser: update test bodies to match upstream verbatim
Update test content to match upstream:
- "arrays" (full upstream test content)
- "blocks" (add labeled block and blk: variants)
- "comptime" (full upstream test with comptime var, expressions)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-02-11 14:40:28 +00:00
f3a2bb4451 parser: port destructure, infix, pointer/slice attribute tests
Port tests:
- "destructure" (implement assign_destructure in
  expectVarDeclExprStatement)
- "infix operators" (partial — orelse as discard target deferred)
- "pointer attributes" (fix ** to parse inner modifiers per upstream)
- "slice attributes" (fix sentinel+align to use ptr_type node)

Fix test bodies to match upstream verbatim:
- "block with same line comment after end brace"
- "comments before var decl in struct"
- "comments before global variables"
- "comments in statements"
- "comments before test decl"

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-02-11 14:40:27 +00:00
4c35471d46 parser: port pointer/slice attributes tests
Port tests:
- "pointer attributes"
- "slice attributes"

Fix ** pointer type to parse modifiers per upstream (no sentinel,
modifiers on inner pointer only).
Fix ptr_type selection when both sentinel and align are present
(use ptr_type with extra data instead of ptr_type_sentinel which
can't store alignment).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-02-11 14:40:27 +00:00
6bd0bdd7ed parser: update test bodies to match upstream verbatim
Update test content to match upstream exactly:
- "block with same line comment after end brace"
- "comments before var decl in struct"
- "comments before global variables"
- "comments in statements"
- "comments before test decl"

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-02-11 14:40:27 +00:00
885581323c reorder tests better 2026-02-11 14:40:26 +00:00
80f4342dd1 parser: port large batch of formatting tests (289/344)
Port 29 tests including:
- field access, multiline string, regression tests
- array formatting, function params, doc comments
- for loop payloads, switch items, saturating arithmetic
- inline for/while in expression context
- canonicalize symbols, pointer type syntax, binop indentation

Implement inline for/while in parsePrimaryExpr.
Remove unused tok variable from parsePrimaryExpr.

Deferred tests (need further work):
- "function with labeled block as return type"
- "Control flow statement as body of blockless if"
- "line comment after multiline single expr if"
- "make single-line if no trailing comma, fmt: off"
- "test indentation after equals sign" (destructuring)
- "indentation of comments within catch, else, orelse"

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-02-11 14:40:26 +00:00
bca46f7a02 parser: reorder tests to match upstream file order
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-02-11 14:40:25 +00:00
a3c1afac2c parser: port doc comment, literal, asm, ternary tests
Port tests:
- "remove newlines surrounding doc comment between members within container decl (1)"
- "remove newlines surrounding doc comment between members within container decl (2)"
- "remove newlines surrounding doc comment within container decl"
- "comments with CRLF line endings"
- "else comptime expr"
- "integer literals with underscore separators"
- "hex literals with underscore separators"
- "hexadecimal float literals with underscore separators"
- "C var args"
- "Only indent multiline string literals in function calls"
- "Don't add extra newline after if"
- "comments in ternary ifs"
- "while statement in blockless if"
- "test comments in field access chain"

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-02-11 14:40:25 +00:00
64ca07dbfd parser: reorder tests to match upstream file order
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-02-11 14:40:25 +00:00
ca3738bc3e parser: port asm, comment, doc comment tests
Port tests:
- "inline asm"
- "inline asm parameter alignment"
- "multiline string in array"
- "file ends with struct field"
- "line comment in array"
- "comment in array initializer/access"
- "comments at several places in struct init"
- "remove newlines surrounding doc comment"
- "remove newlines surrounding doc comment between members"
- "fix single statement if/for/while line breaks"
- "fn type"
- "nosuspend"
- "Block after if"
- "string identifier"
- "error return"

Add if/switch support in parsePrimaryTypeExpr.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-02-11 14:40:24 +00:00
958fbdfd12 update README 2026-02-11 14:40:24 +00:00
106430e6e9 parser: port misc formatting tests batch
Port tests:
- "fix single statement if/for/while line breaks"
- "fn type"
- "nosuspend"
- "Block after if"
- "string identifier"
- "error return"

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-02-11 14:40:23 +00:00
df10e2f885 parser: reorder tests to match upstream file order
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-02-11 14:40:23 +00:00
fdaeca84fe parser: port full "while" and "for" tests
Port tests:
- "while" (full test with all variants)
- "for" (full test with all variants including testTransform)

Fix in parser.c:
- comptime var decl: don't wrap in comptime node (renderer
  detects comptime from token positions)
- forPrefix: handle trailing comma in input list and capture list

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-02-11 14:40:23 +00:00
571fb20bb7 parser: port test "switch multiline string"
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-02-11 14:40:22 +00:00
9eebba10ea parser: port remaining asm tests
Port tests:
- "asm expression with comptime content"
- "simple asm"

Fix asm_output to handle (identifier) operand without arrow.
Fix asm_simple zigData mapping to use node_and_token.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-02-11 14:40:22 +00:00
383fe83626 parser: implement asm_legacy, port inline asm tests
Add AST_NODE_ASM_LEGACY for legacy string clobber format.
When asm clobbers use string literals ("clobber1", "clobber2"),
produce asm_legacy node instead of asm node.

Port tests:
- "preserves clobbers in inline asm with stray comma"
- "remove trailing comma at the end of assembly clobber"

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-02-11 14:40:21 +00:00
c5915c06fb add check_test_order.py script
Checks and fixes test ordering in parser_test.zig to match
upstream zig/lib/std/zig/parser_test.zig.

Usage:
  python3 check_test_order.py        # check only
  python3 check_test_order.py --fix  # reorder tests

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-02-11 14:40:21 +00:00
82d398cbb3 parser: reorder tests to match upstream file order
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-02-11 14:40:20 +00:00
3c83549f77 parser: port enum/union/catch/switch/for/if declaration tests
Port tests:
- "enum declaration"
- "union declaration"
- "catch"
- "switch"
- "for if", "if for", "while if", "if while", "while for", "for while"
- "if"

Fix in parser.c:
- parsePtrPayload: handle multi-capture |a, *b|
- parseWhileStatement/parseForStatement: separate statement-context
  body parsing (block or assign expr + semicolon)

Deferred tests (need further work):
- "while" (full test) - var decl in while body context
- "for" (full test) - capture parsing edge case

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-02-11 14:40:20 +00:00
7aa68ebbdb parser: port precedence, prefix, functions, values, indexing tests
Port tests:
- "precedence"
- "prefix operators"
- "functions"
- "multiline string"
- "multiline string with CRLF line endings"
- "values"
- "indexing"
- "struct declaration"

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-02-11 14:40:19 +00:00
97c9fb6378 parser: implement asm parsing, port formatting tests
Implement in parser.c:
- parseAsmExpr: asm_simple and asm nodes with outputs, inputs,
  clobbers (including legacy string clobber format)
- parseAsmOutputItem, parseAsmInputItem helper functions

Port tests:
- "preserve spacing"
- "return types"
- "imports"
- "global declarations"
- "extern declaration"
- "function attributes"
- "nested pointers with ** tokens"
- "test declaration"
- "top-level for/while loop"
- Various error set, switch prong, comment tests

Note: asm test cases that require asm_legacy AST node (not yet in
ast.h) are deferred.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-02-11 14:40:19 +00:00
a1fef56b95 parser: port formatting, declaration, fn attribute tests
Port tests:
- "preserve spacing"
- "return types"
- "imports"
- "global declarations"
- "extern declaration"
- "function attributes"
- "nested pointers with ** tokens"
- "test declaration"

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-02-11 14:40:19 +00:00
37ae8b01d1 parser: implement for loops, port for/while loop test
Implement in parser.c:
- forPrefix: parse for input expressions and capture variables
- parseForExpr: for_simple and for AST nodes with optional else
- Handle for and while in parsePrimaryTypeExpr for top-level usage

Remove stale cppcheck knownConditionTrueFalse suppression.

Port test "top-level for/while loop".

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-02-11 14:40:18 +00:00
40b7c19848 parser: port error set, suspend, switch prong comment tests
Port tests:
- "error set declaration"
- "union(enum(u32)) with assigned enum values"
- "resume from suspend block"
- "comments before error set decl"
- "comments before switch prong"
- "array literal with 1 item on 1 line"
- "comments in statements"

Implement in parser.c:
- suspend statement in expectStatement
- Fix error set decl to store lbrace token (not 0)
- Fix comptime statement to wrap inner expression

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-02-11 14:40:18 +00:00
d9ae83d1f6 parser: port comment, switch, error value tests batch
Port tests:
- "comment after if before another if"
- "line comment between if block and else keyword"
- "same line comments in expression"
- "add comma on last switch prong"
- "same-line comment after a statement"
- "same-line comment after var decl in struct"
- "same-line comment after field decl"
- "same-line comment after switch prong"
- "same-line comment after non-block if expression"
- "same-line comment on comptime expression"
- "switch with empty body"
- "line comments in struct initializer"
- "first line comment in struct initializer"
- "doc comments before struct field"

Implement in parser.c:
- error.Value and error{...} in parsePrimaryTypeExpr
- TOKEN_PERIOD_ASTERISK (deref) in parseSuffixOp
- Fix comptime statement to wrap inner expression in comptime node

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-02-11 14:40:17 +00:00
50ea349da4 parser: port switch, slice, array literal, formatting tests
Port tests:
- "switch cases trailing comma"
- "slice align"
- "add trailing comma to array literal"
- "first thing in file is line comment"
- "line comment after doc comment"
- "bit field alignment"
- "nested switch"
- "float literal with exponent"
- "if-else end of comptime"
- "nested blocks"
- "statements with comment between"
- "statements with empty line between"
- "ptr deref operator and unwrap optional operator"

Fix in parser.c:
- switch_case SubRange stored via addExtra (not inline)
- Switch case body uses parseAssignExpr (not expectExpr)
- TOKEN_PERIOD_ASTERISK for deref in parseSuffixOp

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-02-11 14:40:17 +00:00
64ce9659de parser: port array literal hint and vertical alignment tests
Port tests:
- "array literal with hint"
- "array literal vertical column alignment"

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-02-11 14:40:17 +00:00
70cbee78e4 parser: port test "struct literal containing a multiline expression"
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-02-11 14:40:16 +00:00
fe86388d1e parser: implement switch, port switch comment tests
Implement parseSwitchExpr in parser.c:
- switch(expr) { cases... } with case items, ranges, else
- switch_case_one and switch_case node types
- Proper scratch management for nested case items

Port tests:
- "switch comment before prong"
- "switch comment after prong"

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-02-11 14:40:16 +00:00
2dc5993a29 parser: port if-condition-wraps tests, implement catch payload
Port tests:
- "if condition has line break but must not wrap"
- "if condition has line break but must not wrap (no fn call comma)"

Implement catch payload (|err|) parsing in parseExprPrecedence.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-02-11 14:40:15 +00:00
6356c15c70 parser: port test "if condition wraps"
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-02-11 14:40:15 +00:00
a8bca43940 parser: implement while loops, port while test
Implement in parser.c:
- parseWhileExpr: while (cond) body, with optional payload,
  continue expression, and else clause
- while_simple, while_cont, while AST nodes

Port test "while else err prong with no block".

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-02-11 14:40:15 +00:00
26c73c4f87 parser: port test "2nd arg multiline string many args"
Split "2nd arg multiline string" to match upstream structure
(separate test for "many args" variant) and add missing
testTransform sub-case.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-02-11 14:40:14 +00:00
14fb82109b parser: port zig fmt on/off in middle of code tests
Port tests:
- "'zig fmt: (off|on)' works in the middle of code"
- "'zig fmt: on' indentation is unchanged"

Handle block-terminated expressions (if, while) that don't need
semicolons by checking if previous token was '}'.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-02-11 14:40:14 +00:00
2b48992a2f parser: port test "whitespace fixes"
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-02-11 14:40:14 +00:00
854f1157c4 parser: port test "rewrite callconv(.@\"inline\") to the inline keyword"
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-02-11 14:40:13 +00:00
ebda1c03bd parser: reorder tests to match upstream file order
Reorder all test blocks in parser_test.zig to match the order
they appear in the upstream zig/lib/std/zig/parser_test.zig.
Tests not in upstream ("Ast header smoke test", "my function")
are placed at the end.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-02-11 14:40:13 +00:00
ee955c8522 parser: port alignment and zig fmt whitespace tests
Port tests:
- "alignment in anonymous literal"
- "'zig fmt: (off|on)' can be surrounded by arbitrary whitespace"

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-02-11 14:40:13 +00:00
5d4e50075a parser: port zig fmt on/off, defer, block slice tests
Port tests:
- "line and doc comment following 'zig fmt: off'"
- "doc and line comment following 'zig fmt: off'"
- "line comment following 'zig fmt: on'"
- "doc comment following 'zig fmt: on'"
- "line and doc comment following 'zig fmt: on'"
- "doc and line comment following 'zig fmt: on'"
- "block in slice expression"
- "defer"

Fix defer node data: body goes in lhs (not rhs) to match .node
union variant.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-02-11 14:40:12 +00:00
26725d687a parser: port fn param and container initializer tests
Port tests:
- "doc comments on param decl"
- "pointer of unknown length"
- "call expression"
- "anytype type"
- "container initializers"

Handle anytype keyword in function parameter lists.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-02-11 14:40:12 +00:00