Commit Graph

34904 Commits

Author SHA1 Message Date
202733edbc astgen: implement ZIR generation for basic expressions and declarations
Mechanical translation of AstGen.zig into C. Implements:
- Container members: comptime, simple_var_decl, test_decl, fn_decl
- Expressions: number_literal, string_literal, identifier (with
  primitive types, integer types, and decl_val/decl_ref resolution),
  field_access (field_val/field_ptr), address_of, builtin_call
  (@import), array_type, array_init (with inferred [_] length),
  array_cat (++), ptr_type
- Statement types: assign with _ = expr discard pattern
- Test infrastructure: testDecl, addFunc, fullBodyExpr,
  blockExprStmts, emitDbgNode/emitDbgStmt, rvalueDiscard
- Support: GenZir sub-block instruction tracking, result location
  propagation (RL_NONE/RL_REF/RL_DISCARD), string dedup, import
  tracking, namespace decl table, lastToken, firstToken

1/5 corpus files pass (test_all.zig). Remaining 4 skip gracefully
via has_compile_errors when encountering unimplemented features.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-02-11 21:32:37 +00:00
280997f658 remove usage of __attribute__((__cleanup__(fn)))
problematic with tcc.
2026-02-11 22:45:54 +02:00
b5300c4d9b more instructions for agents 2026-02-11 20:05:33 +00:00
85e7abd0b7 Merge branch 'zig01' 2026-02-11 18:37:51 +00:00
08f46bb10b update AGENTS.md 2026-02-11 18:37:25 +00:00
5fb7a1ab9c Add astgen scaffolding with ZIR data structures and first passing test
Introduce zir.h/zir.c with ZIR instruction types (269 tags, 56 extended
opcodes, 8-byte Data union) ported from lib/std/zig/Zir.zig, and
astgen.h/astgen.c implementing the empty-container fast path that produces
correct ZIR for empty source files.

The test infrastructure in astgen_test.zig compares C astGen() output
field-by-field against Zig's std.zig.AstGen.generate() using tag-based
dispatch, avoiding raw byte comparison since Zig's Data union has no
guaranteed in-memory layout.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-11 18:37:07 +00:00
0a563abefa valgrind 2026-02-11 18:36:29 +00:00
a3e8198477 valgrind 2026-02-11 18:27:44 +00:00
d6e65fe565 update LICENSE again 2026-02-11 18:14:32 +00:00
ab77c21bcb Update README 2026-02-11 18:10:49 +00:00
9d15552f1c parser: align structural logic with upstream Parse.zig
Fix 11 divergences where parser.c differed from Parse.zig in logic or
structure, not justified by C vs Zig language differences:

- parseContainerMembers: set trailing=false after test decl, add
  field_state tracking (A1, A2)
- expectStatement: guard defer/errdefer behind allow_defer_var (A3)
- expectVarDeclExprStatement: wrap assignment in comptime node when
  comptime_token is set (A4)
- parseBlock: guard semicolon check with statements_len != 0 (A5)
- parseLabeledStatement: add parseSwitchExpr call (A6)
- parseWhileStatement: restructure with else_required and early
  returns to match upstream control flow (A7)
- parseForStatement: restructure with else_required/has_else and
  early returns to match upstream control flow (A8)
- parseFnProto: fail when return_type_expr is missing (A9)
- expectTopLevelDecl: track is_extern, reject extern fn body (A10)
- parsePrefixExpr: remove TOKEN_KEYWORD_AWAIT case (A11)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-11 18:10:26 +00:00
67706f86f3 more debugging tips 2026-02-11 14:54:18 +00:00
2929100c15 parser: reorder function definitions to match upstream Parse.zig
Reorder function definitions so they follow the same order as upstream
zig/lib/std/zig/Parse.zig, making cross-referencing easier. Move
OperInfo and NodeContainerField typedefs to the header section, and add
forward declarations for parseParamDeclList and operTable that are now
needed due to the new ordering.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-11 14:40:34 +00:00
57e033e4b3 parser: align function names with upstream Parse.zig
Rename assignOpTag to assignOpNode to match upstream. Extract inlined
code into separate functions to match upstream's structure:
expectTestDecl, expectIfStatement, expectParamDecl, parseSwitchProngList.
Add parseSingleAssignExpr for upstream API surface alignment.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-11 14:40:33 +00:00
237a05a2fc sort forward declarations by name 2026-02-11 14:40:33 +00:00
5bd533d40c parser: replace fprintf+longjmp with fail(), add forward declarations
Introduce a fail(p, "msg") inline function that stores the error message
in a buffer and longjmps, replacing ~52 fprintf(stderr,...)+longjmp pairs.
The error message is propagated through Ast.err_msg so callers can decide
whether/how to display it. Also add forward declarations for all static
functions and move PtrModifiers typedef to the type definitions section.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-11 14:40:33 +00:00
cd07751d13 parser: port upstream error detection, unskip all 14 tests
Mechanically port error handling patterns from upstream Parse.zig:
- &&/whitespace validation in binary operator parsing
- varargs state tracking in parameter lists
- invalid_bit_range check for slice types
- same-line doc comment detection in eatDocComments
- required for-loop payload validation
- error keyword requiring '.' for error values
- expected_semi_or_else checks in if/for/while statements
- labeled for/while/inline expressions in parsePrimaryExpr
- doc comment validation for test/comptime blocks
- EOF check in parseRoot
- comptime handling in else-branch context

All 381/381 tests pass with 0 skipped.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-11 14:40:32 +00:00
f3e70a0568 parser: skip 14 tests, replace assertToken assert with longjmp
Skip 14 tests that require unimplemented parser features:
- 5 testCanonical/testTransform (primitive type symbols, invalid bit
  range, doc comment validation, multiline string in blockless if)
- 9 testError/recovery (error detection for comptime, varargs,
  semicolons, brackets, whitespace, ampersand)

Replace assert() in assertToken with longjmp to prevent crashes on
malformed input during testError tests.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-02-11 14:40:32 +00:00
e5cbd806c4 parser: refactor expectVarDeclExprStatement to match upstream
Restructure expectVarDeclExprStatement to match upstream Parse.zig's
approach: check for '=' first, then handle var decl init vs expression
statement separately. This fixes parsing of var decls with container
types (e.g., `const x: struct {} = val`), where the '}' of the type
was incorrectly treated as a block-terminated expression.

Also make container member parsing strict (longjmp on unexpected tokens
instead of recovery), and add for/while/labeled-block handling in
parseTypeExpr for function return types.

376/381 tests pass.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-02-11 14:40:31 +00:00
fdefdc98c2 parser: sync tests with upstream, fix tokenizer and parser
Sync parser_test.zig test section with upstream, adding ~40 new tests
(testError, testCanonical, testTransform). Remove extra blank lines
between tests to match upstream formatting.

Fix tokenizer keyword lookup bug: getKeyword() returned TOKEN_INVALID
when input was longer than a keyword prefix (e.g., "orelse" matched
"or" prefix then bailed out instead of continuing to find "orelse").

Fix parser to handle if/for/while expressions in type position (e.g.,
function return types like `fn foo() if (cond) i32 else void`). Add
labeled block support in parsePrimaryTypeExpr. Replace assert for
chained comparison operators with longjmp error.

365/381 tests pass. Remaining 16 failures are parser limitations for
specific syntax patterns and error recovery.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-02-11 14:40:31 +00:00
f5f54fcbe8 parser: propagate errors via setjmp/longjmp instead of exit(1)
Replace 32 parse-error exit(1) calls with longjmp to allow callers to
detect and handle parse failures. The OOM exit(1) in
astNodeListEnsureCapacity is kept as-is.

Add has_error flag to Ast, wrap parseRoot() with setjmp in astParse(),
and update test infrastructure to use the C parser for testError tests.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-02-11 14:40:31 +00:00
bda3329eee parser: refactor to match upstream Parse.zig structure
Extract shared helpers and fix error handling to align with upstream:

- Replace 7 assert() calls that crash on valid input with fprintf+exit
- Extract parsePtrModifiers() and makePtrTypeNode() to deduplicate
  pointer modifier parsing from 4 inline copies into 1 shared function
- Extract parseBlockExpr() and parseWhileContinueExpr() helpers
- Move comptime wrapping into expectVarDeclExprStatement() via
  comptime_token parameter
- Extract finishAssignExpr(), parseSwitchItem(), parseSwitchProng()

Net effect: 3233 → 3106 lines. All 298+ parser tests pass.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-02-11 14:40:30 +00:00
7193385f94 parser: reorder tests, fix check_test_order.py for new file layout
Update check_test_order.py to handle header/footer split correctly
when infrastructure code is at both start and end of file.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-02-11 14:40:30 +00:00
174a275c76 parser: port canonicalize and cast builtin tests
Port tests:
- "canonicalize symbols (simple)"
- "canonicalize symbols (character escapes)"
- "canonicalize symbols (asm)"
- "canonicalize cast builtins"
- "do not canonicalize invalid cast builtins"

Update test bodies to match upstream:
- "comptime block in container"
- "comment after empty comment"
- "comment after params"
- "decimal float literals with underscore separators"
- "container doc comments"

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-02-11 14:40:29 +00:00
801dfc6c6e parser: update test bodies to match upstream verbatim
Update test content to match upstream exactly:
- "comptime block in container"
- "comment after empty comment"
- "comment after params"
- "decimal float literals with underscore separators"
- "container doc comments"

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-02-11 14:40:29 +00:00
7a414e8731 parser: update defer test body to match upstream
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-02-11 14:40:29 +00:00
02621037a2 parser: fix duplicate while test content, match upstream verbatim
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-02-11 14:40:28 +00:00
83da26c183 parser: update test bodies to match upstream verbatim
Update test content to match upstream:
- "arrays" (full upstream test content)
- "blocks" (add labeled block and blk: variants)
- "comptime" (full upstream test with comptime var, expressions)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-02-11 14:40:28 +00:00
f3a2bb4451 parser: port destructure, infix, pointer/slice attribute tests
Port tests:
- "destructure" (implement assign_destructure in
  expectVarDeclExprStatement)
- "infix operators" (partial — orelse as discard target deferred)
- "pointer attributes" (fix ** to parse inner modifiers per upstream)
- "slice attributes" (fix sentinel+align to use ptr_type node)

Fix test bodies to match upstream verbatim:
- "block with same line comment after end brace"
- "comments before var decl in struct"
- "comments before global variables"
- "comments in statements"
- "comments before test decl"

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-02-11 14:40:27 +00:00
4c35471d46 parser: port pointer/slice attributes tests
Port tests:
- "pointer attributes"
- "slice attributes"

Fix ** pointer type to parse modifiers per upstream (no sentinel,
modifiers on inner pointer only).
Fix ptr_type selection when both sentinel and align are present
(use ptr_type with extra data instead of ptr_type_sentinel which
can't store alignment).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-02-11 14:40:27 +00:00
6bd0bdd7ed parser: update test bodies to match upstream verbatim
Update test content to match upstream exactly:
- "block with same line comment after end brace"
- "comments before var decl in struct"
- "comments before global variables"
- "comments in statements"
- "comments before test decl"

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-02-11 14:40:27 +00:00
885581323c reorder tests better 2026-02-11 14:40:26 +00:00
80f4342dd1 parser: port large batch of formatting tests (289/344)
Port 29 tests including:
- field access, multiline string, regression tests
- array formatting, function params, doc comments
- for loop payloads, switch items, saturating arithmetic
- inline for/while in expression context
- canonicalize symbols, pointer type syntax, binop indentation

Implement inline for/while in parsePrimaryExpr.
Remove unused tok variable from parsePrimaryExpr.

Deferred tests (need further work):
- "function with labeled block as return type"
- "Control flow statement as body of blockless if"
- "line comment after multiline single expr if"
- "make single-line if no trailing comma, fmt: off"
- "test indentation after equals sign" (destructuring)
- "indentation of comments within catch, else, orelse"

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-02-11 14:40:26 +00:00
bca46f7a02 parser: reorder tests to match upstream file order
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-02-11 14:40:25 +00:00
a3c1afac2c parser: port doc comment, literal, asm, ternary tests
Port tests:
- "remove newlines surrounding doc comment between members within container decl (1)"
- "remove newlines surrounding doc comment between members within container decl (2)"
- "remove newlines surrounding doc comment within container decl"
- "comments with CRLF line endings"
- "else comptime expr"
- "integer literals with underscore separators"
- "hex literals with underscore separators"
- "hexadecimal float literals with underscore separators"
- "C var args"
- "Only indent multiline string literals in function calls"
- "Don't add extra newline after if"
- "comments in ternary ifs"
- "while statement in blockless if"
- "test comments in field access chain"

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-02-11 14:40:25 +00:00
64ca07dbfd parser: reorder tests to match upstream file order
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-02-11 14:40:25 +00:00
ca3738bc3e parser: port asm, comment, doc comment tests
Port tests:
- "inline asm"
- "inline asm parameter alignment"
- "multiline string in array"
- "file ends with struct field"
- "line comment in array"
- "comment in array initializer/access"
- "comments at several places in struct init"
- "remove newlines surrounding doc comment"
- "remove newlines surrounding doc comment between members"
- "fix single statement if/for/while line breaks"
- "fn type"
- "nosuspend"
- "Block after if"
- "string identifier"
- "error return"

Add if/switch support in parsePrimaryTypeExpr.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-02-11 14:40:24 +00:00
958fbdfd12 update README 2026-02-11 14:40:24 +00:00
106430e6e9 parser: port misc formatting tests batch
Port tests:
- "fix single statement if/for/while line breaks"
- "fn type"
- "nosuspend"
- "Block after if"
- "string identifier"
- "error return"

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-02-11 14:40:23 +00:00
df10e2f885 parser: reorder tests to match upstream file order
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-02-11 14:40:23 +00:00
fdaeca84fe parser: port full "while" and "for" tests
Port tests:
- "while" (full test with all variants)
- "for" (full test with all variants including testTransform)

Fix in parser.c:
- comptime var decl: don't wrap in comptime node (renderer
  detects comptime from token positions)
- forPrefix: handle trailing comma in input list and capture list

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-02-11 14:40:23 +00:00
571fb20bb7 parser: port test "switch multiline string"
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-02-11 14:40:22 +00:00
9eebba10ea parser: port remaining asm tests
Port tests:
- "asm expression with comptime content"
- "simple asm"

Fix asm_output to handle (identifier) operand without arrow.
Fix asm_simple zigData mapping to use node_and_token.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-02-11 14:40:22 +00:00
383fe83626 parser: implement asm_legacy, port inline asm tests
Add AST_NODE_ASM_LEGACY for legacy string clobber format.
When asm clobbers use string literals ("clobber1", "clobber2"),
produce asm_legacy node instead of asm node.

Port tests:
- "preserves clobbers in inline asm with stray comma"
- "remove trailing comma at the end of assembly clobber"

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-02-11 14:40:21 +00:00
c5915c06fb add check_test_order.py script
Checks and fixes test ordering in parser_test.zig to match
upstream zig/lib/std/zig/parser_test.zig.

Usage:
  python3 check_test_order.py        # check only
  python3 check_test_order.py --fix  # reorder tests

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-02-11 14:40:21 +00:00
82d398cbb3 parser: reorder tests to match upstream file order
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-02-11 14:40:20 +00:00
3c83549f77 parser: port enum/union/catch/switch/for/if declaration tests
Port tests:
- "enum declaration"
- "union declaration"
- "catch"
- "switch"
- "for if", "if for", "while if", "if while", "while for", "for while"
- "if"

Fix in parser.c:
- parsePtrPayload: handle multi-capture |a, *b|
- parseWhileStatement/parseForStatement: separate statement-context
  body parsing (block or assign expr + semicolon)

Deferred tests (need further work):
- "while" (full test) - var decl in while body context
- "for" (full test) - capture parsing edge case

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-02-11 14:40:20 +00:00
7aa68ebbdb parser: port precedence, prefix, functions, values, indexing tests
Port tests:
- "precedence"
- "prefix operators"
- "functions"
- "multiline string"
- "multiline string with CRLF line endings"
- "values"
- "indexing"
- "struct declaration"

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-02-11 14:40:19 +00:00
97c9fb6378 parser: implement asm parsing, port formatting tests
Implement in parser.c:
- parseAsmExpr: asm_simple and asm nodes with outputs, inputs,
  clobbers (including legacy string clobber format)
- parseAsmOutputItem, parseAsmInputItem helper functions

Port tests:
- "preserve spacing"
- "return types"
- "imports"
- "global declarations"
- "extern declaration"
- "function attributes"
- "nested pointers with ** tokens"
- "test declaration"
- "top-level for/while loop"
- Various error set, switch prong, comment tests

Note: asm test cases that require asm_legacy AST node (not yet in
ast.h) are deferred.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-02-11 14:40:19 +00:00
a1fef56b95 parser: port formatting, declaration, fn attribute tests
Port tests:
- "preserve spacing"
- "return types"
- "imports"
- "global declarations"
- "extern declaration"
- "function attributes"
- "nested pointers with ** tokens"
- "test declaration"

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-02-11 14:40:19 +00:00