Commit Graph

126 Commits

Author SHA1 Message Date
08f46bb10b update AGENTS.md 2026-02-11 18:37:25 +00:00
5fb7a1ab9c Add astgen scaffolding with ZIR data structures and first passing test
Introduce zir.h/zir.c with ZIR instruction types (269 tags, 56 extended
opcodes, 8-byte Data union) ported from lib/std/zig/Zir.zig, and
astgen.h/astgen.c implementing the empty-container fast path that produces
correct ZIR for empty source files.

The test infrastructure in astgen_test.zig compares C astGen() output
field-by-field against Zig's std.zig.AstGen.generate() using tag-based
dispatch, avoiding raw byte comparison since Zig's Data union has no
guaranteed in-memory layout.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-11 18:37:07 +00:00
a3e8198477 valgrind 2026-02-11 18:27:44 +00:00
d6e65fe565 update LICENSE again 2026-02-11 18:14:32 +00:00
ab77c21bcb Update README 2026-02-11 18:10:49 +00:00
9d15552f1c parser: align structural logic with upstream Parse.zig
Fix 11 divergences where parser.c differed from Parse.zig in logic or
structure, not justified by C vs Zig language differences:

- parseContainerMembers: set trailing=false after test decl, add
  field_state tracking (A1, A2)
- expectStatement: guard defer/errdefer behind allow_defer_var (A3)
- expectVarDeclExprStatement: wrap assignment in comptime node when
  comptime_token is set (A4)
- parseBlock: guard semicolon check with statements_len != 0 (A5)
- parseLabeledStatement: add parseSwitchExpr call (A6)
- parseWhileStatement: restructure with else_required and early
  returns to match upstream control flow (A7)
- parseForStatement: restructure with else_required/has_else and
  early returns to match upstream control flow (A8)
- parseFnProto: fail when return_type_expr is missing (A9)
- expectTopLevelDecl: track is_extern, reject extern fn body (A10)
- parsePrefixExpr: remove TOKEN_KEYWORD_AWAIT case (A11)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-11 18:10:26 +00:00
67706f86f3 more debugging tips 2026-02-11 14:54:18 +00:00
2929100c15 parser: reorder function definitions to match upstream Parse.zig
Reorder function definitions so they follow the same order as upstream
zig/lib/std/zig/Parse.zig, making cross-referencing easier. Move
OperInfo and NodeContainerField typedefs to the header section, and add
forward declarations for parseParamDeclList and operTable that are now
needed due to the new ordering.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-11 14:40:34 +00:00
57e033e4b3 parser: align function names with upstream Parse.zig
Rename assignOpTag to assignOpNode to match upstream. Extract inlined
code into separate functions to match upstream's structure:
expectTestDecl, expectIfStatement, expectParamDecl, parseSwitchProngList.
Add parseSingleAssignExpr for upstream API surface alignment.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-11 14:40:33 +00:00
237a05a2fc sort forward declarations by name 2026-02-11 14:40:33 +00:00
5bd533d40c parser: replace fprintf+longjmp with fail(), add forward declarations
Introduce a fail(p, "msg") inline function that stores the error message
in a buffer and longjmps, replacing ~52 fprintf(stderr,...)+longjmp pairs.
The error message is propagated through Ast.err_msg so callers can decide
whether/how to display it. Also add forward declarations for all static
functions and move PtrModifiers typedef to the type definitions section.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-11 14:40:33 +00:00
cd07751d13 parser: port upstream error detection, unskip all 14 tests
Mechanically port error handling patterns from upstream Parse.zig:
- &&/whitespace validation in binary operator parsing
- varargs state tracking in parameter lists
- invalid_bit_range check for slice types
- same-line doc comment detection in eatDocComments
- required for-loop payload validation
- error keyword requiring '.' for error values
- expected_semi_or_else checks in if/for/while statements
- labeled for/while/inline expressions in parsePrimaryExpr
- doc comment validation for test/comptime blocks
- EOF check in parseRoot
- comptime handling in else-branch context

All 381/381 tests pass with 0 skipped.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-11 14:40:32 +00:00
f3e70a0568 parser: skip 14 tests, replace assertToken assert with longjmp
Skip 14 tests that require unimplemented parser features:
- 5 testCanonical/testTransform (primitive type symbols, invalid bit
  range, doc comment validation, multiline string in blockless if)
- 9 testError/recovery (error detection for comptime, varargs,
  semicolons, brackets, whitespace, ampersand)

Replace assert() in assertToken with longjmp to prevent crashes on
malformed input during testError tests.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-02-11 14:40:32 +00:00
e5cbd806c4 parser: refactor expectVarDeclExprStatement to match upstream
Restructure expectVarDeclExprStatement to match upstream Parse.zig's
approach: check for '=' first, then handle var decl init vs expression
statement separately. This fixes parsing of var decls with container
types (e.g., `const x: struct {} = val`), where the '}' of the type
was incorrectly treated as a block-terminated expression.

Also make container member parsing strict (longjmp on unexpected tokens
instead of recovery), and add for/while/labeled-block handling in
parseTypeExpr for function return types.

376/381 tests pass.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-02-11 14:40:31 +00:00
fdefdc98c2 parser: sync tests with upstream, fix tokenizer and parser
Sync parser_test.zig test section with upstream, adding ~40 new tests
(testError, testCanonical, testTransform). Remove extra blank lines
between tests to match upstream formatting.

Fix tokenizer keyword lookup bug: getKeyword() returned TOKEN_INVALID
when input was longer than a keyword prefix (e.g., "orelse" matched
"or" prefix then bailed out instead of continuing to find "orelse").

Fix parser to handle if/for/while expressions in type position (e.g.,
function return types like `fn foo() if (cond) i32 else void`). Add
labeled block support in parsePrimaryTypeExpr. Replace assert for
chained comparison operators with longjmp error.

365/381 tests pass. Remaining 16 failures are parser limitations for
specific syntax patterns and error recovery.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-02-11 14:40:31 +00:00
f5f54fcbe8 parser: propagate errors via setjmp/longjmp instead of exit(1)
Replace 32 parse-error exit(1) calls with longjmp to allow callers to
detect and handle parse failures. The OOM exit(1) in
astNodeListEnsureCapacity is kept as-is.

Add has_error flag to Ast, wrap parseRoot() with setjmp in astParse(),
and update test infrastructure to use the C parser for testError tests.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-02-11 14:40:31 +00:00
bda3329eee parser: refactor to match upstream Parse.zig structure
Extract shared helpers and fix error handling to align with upstream:

- Replace 7 assert() calls that crash on valid input with fprintf+exit
- Extract parsePtrModifiers() and makePtrTypeNode() to deduplicate
  pointer modifier parsing from 4 inline copies into 1 shared function
- Extract parseBlockExpr() and parseWhileContinueExpr() helpers
- Move comptime wrapping into expectVarDeclExprStatement() via
  comptime_token parameter
- Extract finishAssignExpr(), parseSwitchItem(), parseSwitchProng()

Net effect: 3233 → 3106 lines. All 298+ parser tests pass.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-02-11 14:40:30 +00:00
7193385f94 parser: reorder tests, fix check_test_order.py for new file layout
Update check_test_order.py to handle header/footer split correctly
when infrastructure code is at both start and end of file.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-02-11 14:40:30 +00:00
174a275c76 parser: port canonicalize and cast builtin tests
Port tests:
- "canonicalize symbols (simple)"
- "canonicalize symbols (character escapes)"
- "canonicalize symbols (asm)"
- "canonicalize cast builtins"
- "do not canonicalize invalid cast builtins"

Update test bodies to match upstream:
- "comptime block in container"
- "comment after empty comment"
- "comment after params"
- "decimal float literals with underscore separators"
- "container doc comments"

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-02-11 14:40:29 +00:00
801dfc6c6e parser: update test bodies to match upstream verbatim
Update test content to match upstream exactly:
- "comptime block in container"
- "comment after empty comment"
- "comment after params"
- "decimal float literals with underscore separators"
- "container doc comments"

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-02-11 14:40:29 +00:00
7a414e8731 parser: update defer test body to match upstream
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-02-11 14:40:29 +00:00
02621037a2 parser: fix duplicate while test content, match upstream verbatim
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-02-11 14:40:28 +00:00
83da26c183 parser: update test bodies to match upstream verbatim
Update test content to match upstream:
- "arrays" (full upstream test content)
- "blocks" (add labeled block and blk: variants)
- "comptime" (full upstream test with comptime var, expressions)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-02-11 14:40:28 +00:00
f3a2bb4451 parser: port destructure, infix, pointer/slice attribute tests
Port tests:
- "destructure" (implement assign_destructure in
  expectVarDeclExprStatement)
- "infix operators" (partial — orelse as discard target deferred)
- "pointer attributes" (fix ** to parse inner modifiers per upstream)
- "slice attributes" (fix sentinel+align to use ptr_type node)

Fix test bodies to match upstream verbatim:
- "block with same line comment after end brace"
- "comments before var decl in struct"
- "comments before global variables"
- "comments in statements"
- "comments before test decl"

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-02-11 14:40:27 +00:00
4c35471d46 parser: port pointer/slice attributes tests
Port tests:
- "pointer attributes"
- "slice attributes"

Fix ** pointer type to parse modifiers per upstream (no sentinel,
modifiers on inner pointer only).
Fix ptr_type selection when both sentinel and align are present
(use ptr_type with extra data instead of ptr_type_sentinel which
can't store alignment).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-02-11 14:40:27 +00:00
6bd0bdd7ed parser: update test bodies to match upstream verbatim
Update test content to match upstream exactly:
- "block with same line comment after end brace"
- "comments before var decl in struct"
- "comments before global variables"
- "comments in statements"
- "comments before test decl"

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-02-11 14:40:27 +00:00
885581323c reorder tests better 2026-02-11 14:40:26 +00:00
80f4342dd1 parser: port large batch of formatting tests (289/344)
Port 29 tests including:
- field access, multiline string, regression tests
- array formatting, function params, doc comments
- for loop payloads, switch items, saturating arithmetic
- inline for/while in expression context
- canonicalize symbols, pointer type syntax, binop indentation

Implement inline for/while in parsePrimaryExpr.
Remove unused tok variable from parsePrimaryExpr.

Deferred tests (need further work):
- "function with labeled block as return type"
- "Control flow statement as body of blockless if"
- "line comment after multiline single expr if"
- "make single-line if no trailing comma, fmt: off"
- "test indentation after equals sign" (destructuring)
- "indentation of comments within catch, else, orelse"

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-02-11 14:40:26 +00:00
bca46f7a02 parser: reorder tests to match upstream file order
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-02-11 14:40:25 +00:00
a3c1afac2c parser: port doc comment, literal, asm, ternary tests
Port tests:
- "remove newlines surrounding doc comment between members within container decl (1)"
- "remove newlines surrounding doc comment between members within container decl (2)"
- "remove newlines surrounding doc comment within container decl"
- "comments with CRLF line endings"
- "else comptime expr"
- "integer literals with underscore separators"
- "hex literals with underscore separators"
- "hexadecimal float literals with underscore separators"
- "C var args"
- "Only indent multiline string literals in function calls"
- "Don't add extra newline after if"
- "comments in ternary ifs"
- "while statement in blockless if"
- "test comments in field access chain"

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-02-11 14:40:25 +00:00
64ca07dbfd parser: reorder tests to match upstream file order
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-02-11 14:40:25 +00:00
ca3738bc3e parser: port asm, comment, doc comment tests
Port tests:
- "inline asm"
- "inline asm parameter alignment"
- "multiline string in array"
- "file ends with struct field"
- "line comment in array"
- "comment in array initializer/access"
- "comments at several places in struct init"
- "remove newlines surrounding doc comment"
- "remove newlines surrounding doc comment between members"
- "fix single statement if/for/while line breaks"
- "fn type"
- "nosuspend"
- "Block after if"
- "string identifier"
- "error return"

Add if/switch support in parsePrimaryTypeExpr.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-02-11 14:40:24 +00:00
958fbdfd12 update README 2026-02-11 14:40:24 +00:00
106430e6e9 parser: port misc formatting tests batch
Port tests:
- "fix single statement if/for/while line breaks"
- "fn type"
- "nosuspend"
- "Block after if"
- "string identifier"
- "error return"

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-02-11 14:40:23 +00:00
df10e2f885 parser: reorder tests to match upstream file order
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-02-11 14:40:23 +00:00
fdaeca84fe parser: port full "while" and "for" tests
Port tests:
- "while" (full test with all variants)
- "for" (full test with all variants including testTransform)

Fix in parser.c:
- comptime var decl: don't wrap in comptime node (renderer
  detects comptime from token positions)
- forPrefix: handle trailing comma in input list and capture list

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-02-11 14:40:23 +00:00
571fb20bb7 parser: port test "switch multiline string"
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-02-11 14:40:22 +00:00
9eebba10ea parser: port remaining asm tests
Port tests:
- "asm expression with comptime content"
- "simple asm"

Fix asm_output to handle (identifier) operand without arrow.
Fix asm_simple zigData mapping to use node_and_token.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-02-11 14:40:22 +00:00
383fe83626 parser: implement asm_legacy, port inline asm tests
Add AST_NODE_ASM_LEGACY for legacy string clobber format.
When asm clobbers use string literals ("clobber1", "clobber2"),
produce asm_legacy node instead of asm node.

Port tests:
- "preserves clobbers in inline asm with stray comma"
- "remove trailing comma at the end of assembly clobber"

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-02-11 14:40:21 +00:00
c5915c06fb add check_test_order.py script
Checks and fixes test ordering in parser_test.zig to match
upstream zig/lib/std/zig/parser_test.zig.

Usage:
  python3 check_test_order.py        # check only
  python3 check_test_order.py --fix  # reorder tests

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-02-11 14:40:21 +00:00
82d398cbb3 parser: reorder tests to match upstream file order
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-02-11 14:40:20 +00:00
3c83549f77 parser: port enum/union/catch/switch/for/if declaration tests
Port tests:
- "enum declaration"
- "union declaration"
- "catch"
- "switch"
- "for if", "if for", "while if", "if while", "while for", "for while"
- "if"

Fix in parser.c:
- parsePtrPayload: handle multi-capture |a, *b|
- parseWhileStatement/parseForStatement: separate statement-context
  body parsing (block or assign expr + semicolon)

Deferred tests (need further work):
- "while" (full test) - var decl in while body context
- "for" (full test) - capture parsing edge case

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-02-11 14:40:20 +00:00
7aa68ebbdb parser: port precedence, prefix, functions, values, indexing tests
Port tests:
- "precedence"
- "prefix operators"
- "functions"
- "multiline string"
- "multiline string with CRLF line endings"
- "values"
- "indexing"
- "struct declaration"

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-02-11 14:40:19 +00:00
97c9fb6378 parser: implement asm parsing, port formatting tests
Implement in parser.c:
- parseAsmExpr: asm_simple and asm nodes with outputs, inputs,
  clobbers (including legacy string clobber format)
- parseAsmOutputItem, parseAsmInputItem helper functions

Port tests:
- "preserve spacing"
- "return types"
- "imports"
- "global declarations"
- "extern declaration"
- "function attributes"
- "nested pointers with ** tokens"
- "test declaration"
- "top-level for/while loop"
- Various error set, switch prong, comment tests

Note: asm test cases that require asm_legacy AST node (not yet in
ast.h) are deferred.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-02-11 14:40:19 +00:00
a1fef56b95 parser: port formatting, declaration, fn attribute tests
Port tests:
- "preserve spacing"
- "return types"
- "imports"
- "global declarations"
- "extern declaration"
- "function attributes"
- "nested pointers with ** tokens"
- "test declaration"

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-02-11 14:40:19 +00:00
37ae8b01d1 parser: implement for loops, port for/while loop test
Implement in parser.c:
- forPrefix: parse for input expressions and capture variables
- parseForExpr: for_simple and for AST nodes with optional else
- Handle for and while in parsePrimaryTypeExpr for top-level usage

Remove stale cppcheck knownConditionTrueFalse suppression.

Port test "top-level for/while loop".

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-02-11 14:40:18 +00:00
40b7c19848 parser: port error set, suspend, switch prong comment tests
Port tests:
- "error set declaration"
- "union(enum(u32)) with assigned enum values"
- "resume from suspend block"
- "comments before error set decl"
- "comments before switch prong"
- "array literal with 1 item on 1 line"
- "comments in statements"

Implement in parser.c:
- suspend statement in expectStatement
- Fix error set decl to store lbrace token (not 0)
- Fix comptime statement to wrap inner expression

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-02-11 14:40:18 +00:00
d9ae83d1f6 parser: port comment, switch, error value tests batch
Port tests:
- "comment after if before another if"
- "line comment between if block and else keyword"
- "same line comments in expression"
- "add comma on last switch prong"
- "same-line comment after a statement"
- "same-line comment after var decl in struct"
- "same-line comment after field decl"
- "same-line comment after switch prong"
- "same-line comment after non-block if expression"
- "same-line comment on comptime expression"
- "switch with empty body"
- "line comments in struct initializer"
- "first line comment in struct initializer"
- "doc comments before struct field"

Implement in parser.c:
- error.Value and error{...} in parsePrimaryTypeExpr
- TOKEN_PERIOD_ASTERISK (deref) in parseSuffixOp
- Fix comptime statement to wrap inner expression in comptime node

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-02-11 14:40:17 +00:00
50ea349da4 parser: port switch, slice, array literal, formatting tests
Port tests:
- "switch cases trailing comma"
- "slice align"
- "add trailing comma to array literal"
- "first thing in file is line comment"
- "line comment after doc comment"
- "bit field alignment"
- "nested switch"
- "float literal with exponent"
- "if-else end of comptime"
- "nested blocks"
- "statements with comment between"
- "statements with empty line between"
- "ptr deref operator and unwrap optional operator"

Fix in parser.c:
- switch_case SubRange stored via addExtra (not inline)
- Switch case body uses parseAssignExpr (not expectExpr)
- TOKEN_PERIOD_ASTERISK for deref in parseSuffixOp

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-02-11 14:40:17 +00:00
64ce9659de parser: port array literal hint and vertical alignment tests
Port tests:
- "array literal with hint"
- "array literal vertical column alignment"

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-02-11 14:40:17 +00:00